AI gateway FAQ

LLMRegistry is a gateway and control plane between your application and AI model providers. Your app sends one OpenAI-compatible request; the gateway handles provider credentials, model selection, failover, and normalized usage data before returning the response.

Applications using OpenAI-compatible clients can usually retain much of their existing integration. The work involved depends on the chosen models and features. Integration guidance is available after sign-in.

The available catalog changes as providers add, rename, or retire models. Treat the authenticated model list as the source of truth for callable IDs, context limits, pricing, and capabilities instead of hardcoding a marketing list.

Teams have access to model comparisons, organization controls, usage reporting, and support when preparing for production. Available features depend on provider support and organization policies.

Use a direct model ID when the exact provider and model are part of your product contract. Use a route alias when resilience, latency, cost, or provider portability matters more than pinning one upstream. A route lets you change candidates and guardrails without shipping application code.

A cascade can advance when the current candidate is rate-limited, unavailable, too slow to start, returns a provider fault, or cannot accept the request within the configured context policy. Invalid client requests are returned to the caller instead of being hidden behind repeated provider attempts.

A fallback cascade tries candidates in order and advances after a qualifying failure. Speculative racing starts approved candidates together and streams the first valid result, then cancels unfinished work. Cascades minimize duplicate work; racing can bill both provider attempts in exchange for lower tail latency.

Provider selection and pre-stream failover happen before the response is committed to the client. Once tokens have started, silently switching models could produce an incoherent answer, so a mid-stream upstream failure ends that stream and remains visible in request telemetry.

When enabled by policy, LLMRegistry can match a request to an approved prior result and return it without another provider call. Similarity, lifetime, bypass, and invalidation controls determine when that is safe. Response headers distinguish a hit, miss, or bypass so cached work never disappears from your ledger.

The request ledger records routing metadata such as the resolved route, provider, region, token counts, cost, time to first token, and request ID. The observability and routing views add provider health, decision events, alerts, and webhook delivery without requiring prompt content.

Yes. BYOK connections are stored in the encrypted provider vault and can be managed per upstream. Provider availability and regional options vary, so the authenticated provider list is the current source for supported connections and validation status.

Your connection policy decides. You can keep a strict BYOK boundary, move to another configured model, or allow pooled capacity as a safety net. Health and quota signals are evaluated before routing so the failover policy is explicit rather than an unexpected billing surprise.

Prompt and completion payloads stay in memory for request processing and are not stored to disk by the current gateway. Operational metadata—such as model, provider, tokens, cost, latency, route, region, status, and request identifiers—can still be retained for billing, reliability, and audit workflows.

Semantic caching is an explicit exception that stores encrypted response content for reuse. Each upstream provider has its own data-handling terms, so provider choice remains part of the complete privacy boundary.

Issue a separate key for each workload or environment, then set an expiry, hard spend limit, quota period, model allowlist, and optional IP allowlist. Secrets are shown once, and rotation can keep the previous secret valid briefly to avoid a deployment interruption.

Regional controls depend on the selected provider, model, endpoint, and organization policy. LLMRegistry exposes US and EU boundaries for supported configurations and keeps payload content out of metadata-only evidence exports. For a regulated workload, ask for the current control mapping and attestations rather than inferring certification from a product screen.

Billing depends on the traffic path. Pooled credits use LLMRegistry-managed provider access, while BYOK traffic is billed by your provider and carries the published platform terms. Provider cost, platform fees, funding fees, and applicable tax are shown separately. Use the pricing page for current rates rather than relying on an old invoice or screenshot.

Pooled access is useful when you want one balance, quick provider onboarding, or emergency capacity. BYOK is useful when you already have negotiated pricing, committed capacity, or a provider-specific data boundary. Many teams combine them: BYOK for the normal path and pooled access for an explicitly approved fallback.

Set hard budgets at the organization or key level, isolate workloads with separate credentials, restrict expensive models, and enable threshold alerts. The control plane supports early warnings before the hard stop, while the usage ledger shows cost down to the request and route.

The reseller organization is designed for isolated organization accounts, per-organization wallets and keys, configurable margins, model controls, spend limits, and white-label invoice workflows. Live commercial and tax configuration should be reviewed with the LLMRegistry team before customer billing begins.

Send the LLMRegistry request ID, approximate timestamp, route or model ID, HTTP status, and whether the request was streaming. Do not email API keys, provider secrets, prompts, or full customer payloads. A request ID is usually enough to correlate the metadata trail.

The public status page shows gateway and service health. Signed-in organizations add provider-level latency, error budgets, incidents, route behavior, and alert policies so you can separate an application problem from an upstream event.

Email support@llmregistry.com. Include the organization name and the outcome you need. For security-sensitive issues, describe the impact first and share secrets only through a channel the support team explicitly approves.