Getting Started

This guide will help you get started with Inference Lab.

Installation

Install from crates.io:

cargo install --locked inference-lab

Or build from source:

cargo build --release

Running Your First Simulation

From a checkout of the repository:

inference-lab --config configs/llama-3-70b.toml --workload workloads/quick.toml

configs/ holds one file per model (each with its hardware entries) and workloads/ the arrival patterns and request shapes; pass --hardware <name> when a model config has more than one entry.

Next Steps

• Learn about configuration options
• Explore running simulations