Project System

The project system lets you define multi-step workflows in a dw.toml manifest file and run them with dw project commands.

Creating a Project

From a Template

dw project init my-project

Interactive prompts let you choose a template:

Template	Description
`single-batch`	Prepare, submit, and analyze a single batch
`pipeline`	Multi-stage pipeline (prepare -> run stages -> analyze)
`shell`	Shell-script steps instead of Python
`minimal`	Just a dw.toml with no scaffolding

Or specify directly:

dw project init my-project --template pipeline

From an Example

dw examples clone model-evals
cd model-evals

Manually

Create a dw.toml in your project directory.

The dw.toml Manifest

[project]
name = "my-project"
setup = "uv sync"
workflow = [
    "dw project setup",
    "dw project run prepare -- -n 100",
    "dw files stats batches/batch.jsonl",
    "dw files prepare batches/batch.jsonl --model Qwen/Qwen3-VL-30B-A3B-Instruct-FP8",
    "dw batches run batches/batch.jsonl --watch --output-id .batch-id",
    "dw batches results $(cat .batch-id) -o results.jsonl",
    "dw project run analyze -- -r results.jsonl",
    "dw batches analytics $(cat .batch-id)",
]

[steps.prepare]
description = "Download dataset and generate batch JSONL"
run = "uv run my-project prepare"

[steps.analyze]
description = "Score results against ground truth"
run = "uv run my-project score"

Fields

Field	Description
`project.name`	Project name
`project.setup`	Command to run on `dw project setup` (e.g., `uv sync`)
`project.workflow`	Ordered list of commands for `dw project run-all` and `dw project info`
`steps.<name>.run`	Shell command to execute for this step
`steps.<name>.description`	Human-readable description shown in `dw project info`

Running Steps

Setup

Install dependencies:

dw project setup

Runs the project.setup command from dw.toml.

Individual Steps

dw project run prepare

Pass extra arguments after --:

dw project run prepare -- -n 100 --output custom.jsonl

Full Workflow

Run all workflow steps sequentially:

dw project run-all

This executes every command in the workflow array, skipping dw project setup (run it separately first).

Resume After Failure

If a step fails, fix the issue and continue from where you left off:

dw project run-all --continue

Or start from a specific step:

dw project run-all --from 3

Inspecting State

Project Info

dw project info

Shows available steps, descriptions, and the full workflow.

Run Status

dw project status

Shows the current run state: which steps have completed, which failed, and where the run left off.

Cleanup

dw project clean

Removes batches/, results/, and the run state file (.dw-run.json).

Workflow Comments

Lines in the workflow starting with # are comments. They are displayed in dw project info but skipped during dw project run-all and excluded from step numbering for --from:

workflow = [
    "dw project setup",
    "# Download the dataset from: https://example.com/data",
    "dw project run prepare",
]