GitHub hosts HunyuanVideo, Tencent's open-source framework for large-scale video generation models, enabling AI-driven video creation.
ARBR
0Popularity
About ARBR
ARBR is an open-source gateway and control plane that sits between applications and AI model providers. It normalizes access across major providers and any OpenAI-compatible endpoint through a single OpenAI-compatible interface. The system classifies each incoming request, applies routing rules, enforces budgets and rate limits, and can block or redirect traffic before execution. Every request and response passes through ARBR, generating structured events that include cost, latency, tokens and routing decisions, which can be visualized in local dashboards or exported to OpenTelemetry. The platform supports streaming chat completions, embeddings and realtime voice proxy, with SDKs available for JavaScript, Python and cURL. It enables teams to deploy new models safely using canary or shadow testing with instant rollback, and to evaluate model quality using LLM judges before routing production traffic. ARBR also captures logged traffic for fine-tuning datasets, though this feature is not yet shipped.
Key features
- OpenAI-compatible endpoint for seamless integration
- Difficulty-aware model selection with fallback rules
- Pre-execution policy enforcement (budgets, rate limits, kill switches)
- Structured event logging for cost, latency and routing decisions
- LLM-based quality evaluation and model routing
- Canary and shadow deployment with instant rollback
- Multi-provider support (OpenAI, Anthropic, Google, AWS, Groq, DeepSeek, others)
- JavaScript, Python and cURL SDKs
Use cases
- Govern AI spend and enforce usage policies across teams
- Route requests to the most cost-effective model that meets quality thresholds
- Monitor and audit AI usage patterns for compliance and optimization
Pros
- Single OpenAI-compatible endpoint across all providers
- Real-time cost, latency and usage observability by model, app and team
- Policy enforcement before request execution (budgets, rate limits, guardrails)
- Open-source MIT license with no vendor lock-in
- Supports streaming, embeddings and realtime voice proxy
Cons
- Feature catalog indicates traffic capture for fine-tuning is not yet shipped
- Requires self-hosting or use of hosted option (no fully managed cloud service mentioned)
Frequently asked questions about ARBR
What is ARBR and what does it do?
ARBR is an open-source gateway and control plane that normalizes access to AI models across multiple providers through a single OpenAI-compatible interface. It classifies requests, applies routing rules, enforces budgets and rate limits, and provides observability into cost, latency, tokens, and routing decisions.
Who should use ARBR?
ARBR is designed for teams and organizations that use multiple AI model providers and need centralized control over routing, governance, and observability. It suits developers, data scientists, and operations teams managing AI workloads.
How does ARBR integrate with existing applications?
ARBR integrates seamlessly by acting as a single OpenAI-compatible endpoint. Existing applications can point to ARBR’s gateway without modifying their code, gaining routing, logging, authentication, and budget enforcement immediately.
Does ARBR support real-time voice proxy?
Yes, ARBR supports real-time voice proxy alongside streaming chat completions and OpenAI-compatible embeddings, enabling a wide range of AI-driven applications.
Can ARBR help evaluate and deploy new AI models safely?
ARBR enables safe model deployment through canary or shadow testing on real traffic, with instant rollback and regression gates. It also supports evaluating model quality using LLM judges before routing production traffic.
What kind of observability does ARBR provide?
ARBR provides detailed observability by capturing structured events for every request and response, including cost, latency, tokens, and routing decisions. This data can be visualized in local dashboards or exported to OpenTelemetry for further analysis.
