OpenLLM

Local LLM routing layer for connecting models and tools

Coding & Development AI Models & APIsFree Trial

Visit website ↗

$30/moStarting price

3.3KMonthly visits

9Popularity

About OpenLLM

OpenLLM acts as a routing layer that connects local and remote large language models to developer tools without requiring API keys or subscriptions upfront. It supports major providers like OpenAI, Anthropic, Google, and local models via Ollama, LM Studio, or other servers, allowing a single endpoint to manage inference across multiple sources. The tool integrates with agents such as Claude Code, Codex, OpenCode, and others, enabling shared memory and context through Context for seamless project continuity. Users can download a Mac app or install via a terminal command, then route requests through OpenLLM with minimal setup. Encryption ensures provider keys remain secure, with AES-256-GCM protection and zero-knowledge principles preventing decryption even under subpoena. The platform includes hosted inference for the first 50 million tokens per month on the free tier, with additional features like analytics, voice transcription, and media libraries available in paid plans.

Key features

Local and API model routing
Zero-knowledge encrypted key vault
Shared memory across agents
Context web data integration
Hosted inference included
Multi-provider compatibility
Usage analytics and cost tracking
Media library for images, audio, and video

Use cases

Routing LLM requests across multiple providers and local models
Enabling shared context for AI agents in development workflows
Securely managing provider keys without exposing them to tools

Pros

Supports local and remote models through a single endpoint
Includes hosted inference for free tier users
Encrypts provider keys with zero-knowledge architecture
Integrates with multiple developer tools and agents
Shared memory and context across agents via Context

Cons

Free tier limited to 50 million tokens per month
Requires installation of a desktop app or terminal command
No explicit support for mobile or browser-only use

Frequently asked questions about OpenLLM

What does included inference cover?

Included inference provides a set of tokens per month at no cost, allowing users to start making requests immediately without connecting external providers or local models. The free tier includes a baseline amount of hosted inference, with additional usage available in paid plans.

What is the integrations marketplace?

The integrations marketplace allows users to add tools and services once, making them available across all supported agents and clients. This includes integrations for tools like Claude Code, Codex, and OpenCode, as well as additional features like Context.dev for web data.

Do I need a Context.dev account?

No, a Context.dev account is not required. Context.dev features, such as shared memory and web data capabilities, are included for free on every OpenLLM plan without needing a separate account.

Can you read my provider keys?

No, provider keys are encrypted using AES-256-GCM and stored as ciphertext. OpenLLM does not store the actual keys, and the encryption keys are held only in memory for the duration of a request before being zeroed out.

Is this local model inference?

OpenLLM supports both local models (via Ollama, LM Studio, or other servers) and remote models from major providers like OpenAI, Anthropic, and Google. It acts as a routing layer to manage inference across all these sources.

Do I have to change my code?

No, OpenLLM is designed to work with existing tools and workflows. Users can route requests through OpenLLM with minimal setup, and the configuration does not require modifying the code of supported agents or clients.