Get Started

Run benchmarks on your Apple Silicon Mac and contribute to the community dataset.

1

Install fusion-mlx

Install the fusion-mlx inference engine for Apple Silicon:

$pip install fusion-mlx
2

Run the Server

Start the fusion-mlx server with your chosen model:

$fusion-mlx serve --model Qwen3-4B --quantization Q4_K_M

Or use the admin panel to browse and launch models:

$fusion-mlx serve --admin
3

Submit Benchmarks

Benchmarks run automatically when you use the admin panel. Results are submitted to bench.dpdns.org after each run.

Each benchmark measures:

  • tg TPS — Token generation throughput (tokens/sec)
  • pp TPS — Prompt processing throughput (tokens/sec)
  • TTFT — Time to first token (milliseconds)
  • Peak Memory — Peak GPU memory usage (GB)

API Reference

POST/api/benchmarks
{
  "chip_name": "M2 Ultra",
  "chip_variant": "",
  "memory_gb": 192,
  "gpu_cores": 76,
  "os_version": "macOS 15.3",
  "omlx_version": "0.9.8",
  "model_name": "Qwen3-32B",
  "quantization": "Q4_K_M",
  "context_length": 4096,
  "pp_tps": 128.5,
  "tg_tps": 24.3,
  "ttft_ms": 320.1,
  "peak_memory_gb": 18.2,
  "submission_group": "uuid-string",
  "owner_hash": "optional-anonymous-id",
  "batching_results": []
}

201 Created

Benchmark saved successfully.

View at/benchmarks/42

409 Duplicate

Same chip + model + quant + context already exists.

Existing at/benchmarks/42

Owner Hash

The owner_hash field is an optional anonymous identifier. It allows you to:

  • View all your submissions at /my/{your_hash}
  • Deduplicate submissions — same owner + model + chip + quant + context = skipped
  • Track your contribution history without creating an account

Generate one with: python -c "import uuid; print(uuid.uuid4().hex[:16])"

View Community Benchmarks