Self-hosted API gateway & AI gateway, in Rust

Tygress is pre-release. The gateway is built and in active pre-1.0 development, but it is not publicly available yet. Join the early-access waitlist for beta availability and product updates.

Tygress is pre-1.0, and some things are not done. There is no published container image, Helm chart, or Kubernetes manifest, and the operator console, Kubernetes discovery, Kafka logging, and the local embedder for the semantic cache are opt-in builds. HTTP/3, WebSocket over HTTP/2, SAML, post-quantum certificates and signatures, and Windows are not supported. The OpenAI Realtime API returns 501, failover between A2A agents is not wired, and some proxy-wasm hostcalls (HTTP callouts, shared data and queues, timers) are not implemented. Benchmarks will be published with their methodology. The configuration schema and admin API may change before 1.0.

Tygress is a self-hosted API gateway and AI gateway written in Rust on its own purpose-built async transport. It routes REST, gRPC and WebSocket APIs, LLM requests, MCP tools and A2A agents through one data plane with shared identity, rate limiting, cost and guardrail policies, and also proxies TCP/UDP traffic at L4.

Yes, if it understands both. An API gateway manages identity and traffic for REST and gRPC services; an AI gateway adds model routing, provider credentials, token budgets, and AI guardrails. Tygress runs both in one self-hosted data plane, so one identity, rate-limit, and logging model covers APIs, models, and agents. Whether you should consolidate depends on your workloads and existing infrastructure.

Tygress runs as one binary, or as separate control-plane and data-plane roles that sync configuration over NATS. The source includes a Dockerfile for a distroless, non-root container image and a Docker Compose reference stack; no image is published yet. An opt-in Kubernetes build watches EndpointSlices to track upstream pods. It is designed to run in a private VPC or on-premises; fully air-gapped operation also requires local model endpoints, identity services, and other dependencies.

Self-hosting the gateway gives you control of routing and policy. Requests sent to external model APIs still leave your network; keeping inference local requires local model endpoints and control of logs, caches, and telemetry.

Tygress connects to OpenAI, Anthropic, Google Gemini, Google Vertex AI (Gemini models), Azure OpenAI, AWS Bedrock, Mistral, Groq, and any OpenAI-compatible endpoint. Clients can call any of them through the OpenAI Chat Completions, OpenAI Responses, or Anthropic Messages API: natively where the provider supports that API, and bridged through chat completions otherwise. Chat Completions reaches Anthropic, Gemini, and Vertex through each vendor's own OpenAI-compatible endpoint. Some provider-specific features, such as stored conversations, hosted tools, prompt caching, or image input, need a native endpoint.

Tygress reads token usage from provider replies, including cached and reasoning tokens where the provider reports them, and prices it from a bundled 80-model catalog or your own pricing. Token limits and USD budgets run over fixed, sliding, rolling-anchor, or token-bucket windows keyed per consumer, route, or any request variable. Durable daily, monthly, or lifetime caps apply per consumer and per team, with webhook alerts at thresholds you set. Semantic caching can answer a similar non-streaming prompt without calling the chat model, and semantic routing, once switched from shadow mode to enforce, can send simpler prompts to cheaper models. Actual savings depend on your models, traffic, and cache settings.

Authentication plugins cover OIDC, JWT with remote JWKS, LDAP and Active Directory, mTLS, OAuth2 token introspection, JWE, HMAC, basic auth, and API keys, with OPA and ACLs for authorization. AI controls include PII and secret redaction, a prompt-injection guard, model and MCP tool allowlists, and human approvals for MCP tool calls and A2A methods, with each decision recorded in an append-only PostgreSQL table when a database is configured. These controls support your security policies; deployment choices and upstream services also affect how data is handled.

An open-source core is planned at launch. Release and license details will be published with the release.

No. Earlier builds used a vendored fork of Cloudflare's Pingora. Since September 2026, Tygress runs on tygress-proxy, its own async transport, and its own request engine, written in Rust. Owning the transport lets DLP and prompt guards inspect every chunk of a streamed reply, and runs provider translation and request signing inside each retry and failover attempt, after every plugin edit.

Current capabilities include: API gateway for HTTP/1.1, HTTP/2, gRPC (including gRPC-Web and JSON transcoding), WebSocket, and TCP/UDP/TLS passthrough, with load balancing, active and passive health checks, retries, and circuit breaking; 60+ built-in plugins for authentication (OIDC, JWT, LDAP, mTLS, OAuth2 introspection), authorization (ACL, OPA), traffic control, and logging, plus sandboxed proxy-wasm plugins and inline Rhai or JavaScript; LLM routing across nine provider types with failover, virtual keys, token limits, USD budgets for keys and teams, and a bundled model-pricing catalog; AI guardrails: prompt-injection detection, DLP that redacts, blocks, or audits PII and secrets in prompts and can redact them in buffered and streamed replies, model allowlists, and semantic caching; MCP tool gating and human approvals, REST-to-MCP toolsets, and A2A agent routing; Prometheus metrics, OpenTelemetry tracing with GenAI semantic conventions, Langfuse export, and append-only PostgreSQL records of config revisions, approval decisions, and autopilot changes; Hybrid post-quantum TLS key exchange (X25519MLKEM768), offered by default and enforceable per listener, upstream, or route; ACME certificates; and zero-downtime binary upgrades.