Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Onwards

Crates.io Documentation GitHub

A Rust-based AI Gateway that provides a unified interface for routing requests to OpenAI-compatible targets. The goal is to be as “transparent” as possible.

Features

  • Unified routing to any OpenAI-compatible provider
  • Hot-reloading configuration with automatic file watching
  • Authentication with global and per-target API keys
  • Rate limiting per-target and per-API-key with token bucket algorithm
  • Concurrency limiting per-target and per-API-key
  • Load balancing with weighted random and priority strategies
  • Automatic failover across multiple providers
  • Strict mode for request validation, response sanitization, and error standardization
  • Response sanitization for OpenAI schema compliance
  • Prometheus metrics for monitoring
  • Custom response headers for pricing and metadata
  • Upstream auth customization for non-standard providers