GitHub hosts HunyuanVideo, Tencent's open-source framework for large-scale video generation models, enabling AI-driven video creation.
oMLX
167KMonthly visits
37Popularity
About oMLX
oMLX is a macOS-native machine learning server designed to run large language models locally with optimized performance. It implements a two-tier paged SSD KV cache that persists cache blocks to disk in safetensors format, allowing previously seen prefixes to be restored in milliseconds rather than recomputed. This addresses the issue of frequent KV cache invalidation common in agent workflows, reducing time-to-first-token from 30–90 seconds to under 5 seconds on long contexts. The system supports continuous batching through mlx-lm’s BatchGenerator, enabling concurrent requests with up to 4.14× generation speedup at 8× concurrency. oMLX serves as a drop-in backend for OpenAI and Anthropic-compatible clients including Claude Code, OpenClaw, and Cursor, providing both OpenAI-compatible and Anthropic-compatible API endpoints. A native macOS menu bar app allows starting, stopping, and monitoring the server, alongside a web dashboard for model management, chat, and real-time metrics. The application reads standard Hugging Face and LM Studio model caches, eliminating the need for re-downloading models.
Key features
- Paged SSD KV cache with LRU policy
- Continuous batching for concurrent requests
- Native macOS menu bar application
- Web dashboard for model management and monitoring
- OpenAI and Anthropic API compatibility
- Multi-model serving with LRU eviction
- Hugging Face cache integration
- MCP tool integration support
Use cases
- Local LLM inference for coding agents
- Offline AI development on macOS
- Model serving for OpenAI-compatible clients
Pros
- Paged SSD KV caching preserves cache across sessions and restarts
- Native macOS app with menu bar control and web dashboard
- OpenAI and Anthropic-compatible API endpoints for tool integration
- Continuous batching for concurrent request handling
- Supports MLX-format models including Qwen, LLaMA, Mistral, and VLM models
Cons
- Requires Apple Silicon Mac with macOS 15+
- Minimum 16GB RAM recommended, 64GB+ preferred for larger models
- No Windows or Linux support
- Models must be in MLX format
Frequently asked questions about oMLX
How is oMLX different from Ollama or LM Studio?
oMLX persists every KV cache block to SSD in safetensors format, allowing previously cached portions to be restored in milliseconds rather than recomputed when context shifts. This reduces time-to-first-token from 30–90 seconds to under 5 seconds on long contexts, unlike Ollama or LM Studio which invalidate the entire cache in memory.
What hardware do I need to run oMLX?
Apple Silicon (M1 or later) with macOS 15+ is required. 16GB RAM is the minimum, but 64GB+ is recommended for comfortable use with larger models. The sweet spot for daily coding work is an M-series Pro/Max with 64GB+.
Does oMLX work with Claude Code, OpenClaw, and Cursor?
Yes. oMLX provides both OpenAI-compatible and Anthropic-compatible API endpoints, functioning as a drop-in backend for all three tools. The web dashboard includes a one-click config generator for easy setup.
Do I need to re-download my models to use oMLX?
No. oMLX reads the standard Hugging Face cache shared by Transformers, MLX, vLLM, and llama.cpp, so models already downloaded just work without re-downloading. It also supports LM Studio folders and includes a built-in Hugging Face downloader.
What models are supported by oMLX?
oMLX supports any MLX-format model from Hugging Face, including Qwen, LLaMA, Mistral, Gemma, DeepSeek, MiniMax, GLM, and more. Vision-Language Models are supported since v0.2.0, and reasoning models get automatic tag handling.
How do I get started with oMLX?
Download the DMG or install from source. The macOS app walks you through model directory setup, and the server can be started, stopped, and monitored via a native menu bar app or web dashboard. No re-download of models is required.
oMLX Website Engagement
Last Update: 9 days ago
Total Monthly Visits
0
Bounce Rate
0%
Visit Duration (avg)
0.00s
Pages Per Visit
0
Country Rank
0
India
Global Rank
0
Category Rank
#0
Computer Hardware
Monthly Traffic
Traffic Sources
Traffic Share By Country
- United States19.7%
- China19.5%
- Taiwan18.5%
- South Korea8.9%
- Singapore7.9%
