Enterprise Private Deployment · Inference Space
Deploy the unified model gateway on enterprise infrastructure to centrally manage model routing, Keys, usage, audits, and security policies.
Inference Space is the enterprise private deployment of the unified model gateway. Applications access only the internal gateway address, while administrators manage organizations, gk_ API Keys, model providers, usage, and audits in one console.
Inference Space private deployment guide
Docker Compose, internal images, first login, upgrades, backups, and troubleshooting.
Chat API
Connect applications with the OpenAI-compatible /v1/chat/completions endpoint.
Models and capabilities
Review available capabilities, model integration methods, and request constraints.
Private deployment is suitable for teams that need to keep data inside the enterprise network, centrally govern calls to multiple models, or connect self-hosted inference services. For high availability, multi-region, or GPU inference clusters, complete capacity and network planning before deployment.
Why deploy inside the enterprise
- Controlled data boundary: Gateway configuration, API Keys, request records, and audit data remain inside the enterprise network; whether request content is sent to an external upstream depends on the model providers enabled by the enterprise.
- Unified model entry point: Applications maintain one Base URL and one gateway Key, while administrators can change model providers in the console.
- Centralized governance: Configure routing, rate limits, quotas, usage, audits, and security moderation in one control plane.
- Smooth migration: The data plane remains OpenAI-compatible, so existing applications usually need only a new Base URL, Key, and model configuration.
Capabilities
Inference Space provides unified access to chat, image generation, speech recognition, speech synthesis, OCR, and vision segmentation. The models that can actually be called depend on the providers and models enabled in the private instance.
| Capability | Enterprise use |
|---|---|
| Model routing | Coordinate upstream models, self-hosted services, and different model versions from one place. |
| API Keys and organizations | Issue a separate gk_ Key for each application or team to simplify authorization, quotas, and revocation. |
| Usage and audits | Record usage by model, capability, request, and billing item for reconciliation and troubleshooting. |
| Security policies | Configure input and output moderation and PII redaction independently, by organization in the console. |
| Observability | View gateway, usage-ingestion, and basic runtime metrics in Grafana. |
Delivery model
Customers deploy the released public multi-architecture images or a complete image set in an enterprise-internal registry. The delivery package's ai-gateway/onprem/ directory contains Docker Compose, .env.example, Vector, and Grafana configuration; use these files as the source of truth for deployment parameters.
For environment variables, the image inventory, startup order, and upgrade and recovery steps, see the Inference Space private deployment guide.
Integration recommendations
- Keep API Keys only on the server or in the enterprise secrets-management system; do not expose them to browsers or clients.
- Use a separate
gk_Key for each application to simplify individual audits, rate limits, and revocation. - Treat the gateway Base URL, model ID, and Key name as application configuration rather than hard-coding them in application code.
- Place production instances behind an enterprise HTTPS reverse proxy; do not expose the database, Redis, GreptimeDB, or internal usage endpoints externally.
For architecture planning or deployment support, contact deployment support.
Client Integrations (Claude Code / Codex CLI / SDKs)
One-click scripts for pointing Claude Code and Codex CLI at the gateway, plus OpenAI and Anthropic SDK configuration.
Inference Space Private Deployment
Deploy Inference Space inside an enterprise network with Docker Compose, including image preparation, environment configuration, first login, model integration, upgrades, and recovery.