Inference providers

Eigin treats inference as a utility. The agent doesn't care which model is on the other end, as long as it can hold a conversation and call tools. All provider communication flows through a single client protocol, with a few implementations covering the landscape, so switching providers is easy. Three kinds sit behind that protocol: the Eigin Relay, third-party cloud providers, and on-device local models.

Eigin Relay

A built-in provider backed by Eigin's cloud inference proxy. It's the easiest way to connect cloud AI: buy credits with an in-app purchase and start chatting. Behind the scenes it forwards requests to a vetted upstream provider at cost. See Eigin Relay.

Cloud providers

Bring your own key to a third-party provider. Built-in presets: OpenAI, Anthropic, Google (AI Studio), Mistral, OpenRouter, EUrouter, Berget, Infercom. Each preset bundles a base URL, logo, help text, and a link to get an API key. You can also add a custom provider with any OpenAI-compatible base URL.

Local providers

Inference can run entirely on-device: offline, on your own hardware. The local provider is a built-in preset, picked like any other. Once downloaded, a model stands in for cloud models in normal chats.

Clients

Rather than a client per provider, Eigin uses a small set:

  • OpenAI-compatible client: any provider implementing the OpenAI REST API (/models, /chat/completions). Covers OpenAI, Google, Mistral, OpenRouter, and most others.
  • Relay client: the OpenAI-compatible client pointed at the Eigin Relay, plus credit-wallet management. No API key: each request is authorized with a credit token instead.
  • Anthropic client: a native implementation for the Messages API, which differs enough to warrant its own client (auth headers, content blocks, typed SSE events, system prompt handling).
  • Local client: bridges the agent loop to the on-device model, holding session state per chat.

A factory selects the client from the base URL: the relay URL routes to the relay client, other cloud URLs to OpenAI or Anthropic, and local:// to the on-device client. Adding an OpenAI-compatible provider is just adding a preset.

Provider adapters

Some providers are mostly OpenAI-compatible but have quirks. A provider adapter handles the adjustments instead of branching inside the client:

  • Gemini: sends stop as the finish reason even with tool calls present. The adapter checks for pending tool calls on stop and injects Gemini-specific metadata.
  • Mistral: requires tool call IDs to be alphanumeric and at most 9 characters. The adapter sanitizes them.
  • Anthropic on OpenRouter: injects a cache_control hint so OpenRouter passes prompt caching through.

Model catalog

A curated list of known models with metadata: display names, context windows, costs, and capabilities (tool calls, reasoning, vision), ranked by fit. When a user adds a provider, Eigin auto-selects the best-fit model; the user can override in the model selector.

Extending

  • New OpenAI-compatible provider: add a preset.
  • Provider with quirks: add a preset and an adapter.
  • Completely different API: implement the client protocol and add a URL check in the factory.
  • New models or re-ranking: update the catalog.