Train a model

Training uses a ready spec and its rewards. Start from an evaluation baseline.

1. Choose a model

from trajectory import Client

client = Client()
bench_id = "YOUR_BENCHMARK_ID"
catalog = client.training.list_options(bench_id=bench_id)

for model in catalog.models:
    print(model.base_model_slug, model.options)

Use a returned base_model_slug. Each entry describes that model’s supported options, bounds, and defaults. A computed default has default_kind="computed" and a null default; leave the option unset to let the server calculate it.

2. Start

run = client.training.create(
    bench_id=bench_id,
    base_model_slug="YOUR_SUPPORTED_TRAINING_MODEL_SLUG",
    options={"num_steps": 20},
)
print(run.training_run_id)

Training and evaluation take RunOptions fields through options; a standalone evaluation accepts a subset. Omit options, or pass {}, to use the selected model’s defaults. Only set options advertised by that mode’s list_options response; training and evaluation can have different bounds. batch_size counts eligible task groups per optimizer step, and group_size counts trajectories within a group. For a short test run, lower both; default concurrency scales with them. Nothing here caps total trajectories, so plan a budget before you start. See training options for every setting and reduce trajectory spend for the ones that control token use.

3. Watch progress and pick a checkpoint

import time

while True:
    training = client.training.runs.retrieve(run.training_run_id)
    progress = client.training.runs.progress(run.training_run_id)
    print(training.status, progress.completed_steps, progress.total_steps)
    if training.status in {"succeeded", "failed", "cancelled"}:
        break
    time.sleep(30)

if training.status != "succeeded":
    raise RuntimeError(f"Training {training.status}: {training.failure}")

An ID means the run was accepted. A pending run with zero completed steps is waiting; increasing step counts show progress. Watch attempted and graded trajectories as well. If trajectories accumulate without an update, inspect their endings and reward components; cancel a test run that has exceeded its planned budget. Check completed steps and checkpoints before concluding that training updated the model. See termination reasons for which endings training admits. For a small test run, inspect trajectories and group counts:

trajectories = list(client.trajectories.list(training_run_id=run.training_run_id))
print(len(trajectories), [(item.trajectory_id, item.status) for item in trajectories])
for step in client.training.runs.list_step_summaries(run.training_run_id).items:
    print(step.step, step.no_signal_groups, step.used_groups, step.pending_groups)

no_signal_groups counts completed groups with identical rewards, which give zero advantage in group-relative training. When these groups are filtered out, training needs additional trajectories to fill the batch. After the run succeeds, inspect its rewards and choose a checkpoint. list_held_out_rewards returns the validation reward for each step:

for point in client.training.rewards.list_history(run.training_run_id):
    print(point.step, point.avg_reward)
for point in client.training.rewards.list_held_out_rewards(run.training_run_id):
    print(point.step, point.held_out_reward, point.round_completed,
          point.completed_samples, point.expected_samples)

checkpoints = client.training.checkpoints.list(run.training_run_id)
checkpoint = max(checkpoints, key=lambda c: c.step_index)
print(checkpoint.step_index, checkpoint.checkpoint_id, checkpoint.base_model_slug)

Check round_completed, and completed_samples against expected_samples, before comparing validation means. The agent’s Training tab shows run progress and checkpoints. Evaluate that checkpoint before you deploy it.

4. Resume from a checkpoint

resumed = client.training.create(
    bench_id=bench_id,
    base_model_slug=checkpoint.base_model_slug,
    parent_checkpoint_id=checkpoint.checkpoint_id,
    options={"num_steps": 20},
    display_name="Resumed training",
)
print(resumed.training_run_id)

The checkpoint must match the selected base model. Use parent_checkpoint_id for both resumed training and checkpoint evaluation. Resuming restores the optimizer, LR schedule, and global step; set load_weights_only to true to start from the checkpoint’s weights with fresh training state.

Training options

Pass these keys in options. The defaults below apply to every supported model unless noted; list_options reports the bounds and defaults for the model you select. A default written as null → a value is computed when you leave the option unset. With the default batch_size of 32 and group_size of 8, that resolves to 512 concurrent training trajectories and a minimum of 4 eligible trajectories per group.

Optimization

Option	Default	What it controls
`num_steps`	`100`	Number of optimizer steps to run (1–100,000).
`algorithm`	`gspo`	Training algorithm; omit to preserve the model default. One of `gspo` or `grpo`.
`batch_size`	`32`	Eligible task-instance groups per optimizer update (1–256).
`group_size`	`8`	Trajectories generated for each training instance; GSPO and GRPO require at least 2 (at most 32).
`min_trajectories_per_group`	`null` → `group_size // 2`	Minimum eligible trajectories per training group. `null` uses half of `group_size`, rounded down (1–32).
`learning_rate`	`1e-4`	Base optimizer learning rate (1e-8–1e-2).
`lr_warmup_steps`	`15`	Reported training steps used for warmup, including skipped optimizer updates.
`lr_hold_steps`	`135`	Training steps to hold the base rate after warmup, including skipped updates, before cosine decay.
`lr_min_ratio`	`0.1`	Cosine decay floor as a fraction of the base learning rate (0–1).
`max_staleness`	`3`	Maximum optimizer steps a training trajectory’s starting policy may trail the current policy (0–100). Older in-flight trajectories are cancelled and older completed trajectories are discarded; `0` trains only on trajectories started from the current policy.
`load_weights_only`	`false`	Load parent checkpoint weights with fresh training state. Defaults to `false`, which resumes the optimizer, LR schedule, and global step. With validation enabled, evaluate the starting policy, run validation when `global_step % val_interval_steps == 0`, and evaluate the final policy on normal completion. Resuming preserves the global step and validation cadence.
`density_aware_training`	`false`	Train each task at four output-token budgets, with `group_size` trajectories per budget. Turns on `train_on_truncation`, which must stay `true` except when resuming a checkpoint with its training state. `batch_size` sets the total number of task groups per batch; each budget counts as a separate group. Learn more.

The learning rate rises linearly to learning_rate over lr_warmup_steps, holds until step lr_warmup_steps + lr_hold_steps, then follows a cosine curve down to learning_rate × lr_min_ratio at the final step.

Learning rate by training step: a linear rise to the base rate by step 15, flat until step 150, then a cosine curve down to 10% of the base rate at the final step of a 400-step example. A default 100-step run ends during the flat section.

With the defaults, decay starts at step 150, so a default 100-step run ends at the base rate. To use the decay, set num_steps above 150 or lower lr_hold_steps.

Training filters

Option	Default	What it controls
`train_on_truncation`	`false`; `true` with `density_aware_training`	Include trajectories that hit token or turn limits in training, with reward 0. Defaults to `false`, or `true` with `density_aware_training`; set to `true` to discourage reward hacking through truncated submissions.
`train_on_timeout`	`false`	Include timed-out trajectories in training, with reward 0. Defaults to `false`; set to `true` to discourage reward hacking through timed-out submissions.
`train_on_error`	`false`	Train on errored trajectories, including execution errors, grading errors, and missing grades, with reward 0.
`train_on_all_fail`	`false`	Train on groups whose eligible rewards are all at or below zero; `false` excludes them.
`train_on_all_pass`	`false`	Train on groups whose eligible rewards are all at or above one; `false` excludes them.
`train_on_zero_advantage`	`false`	Train on groups with identical eligible rewards; `false` excludes them.

Reward 0 applies to training only; the recorded trajectory reward stays as logged. See termination reasons.

Trajectories

Option	Default	What it controls
`max_concurrent_trajectories`	`null` → `batch_size × group_size × 2`	Maximum active training trajectories (1–1,024). `null` uses `batch_size × group_size × 2`, allowing 2× batch oversampling for throughput and asynchronous RL. Recommended: set above `batch_size × group_size` to keep collection ahead of training.
`max_turns_per_trajectory`	`200`	Maximum agent and environment turns allowed in each trajectory (1–500).
`max_output_tokens_per_turn`	`null` → model maximum	Maximum output tokens per model response. `null` uses the model’s maximum, bounded by remaining context.
`max_output_tokens_per_trajectory`	`null` → model maximum context length	Maximum output tokens across one trajectory, including after context compaction (1–2,097,152). `null` uses the model’s maximum supported context length.
`trajectory_timeout_seconds`	`7200`	Maximum execution seconds per trajectory, including grading (1–36,000). Excludes queueing and sandbox startup. Collection timeouts must cover this budget.
`batch_timeout_seconds`	`9600`	Maximum wall-clock time to assemble one training batch (1–38,400). The clock starts when collection begins and does not reset as groups or in-flight trajectories arrive. Must cover trajectory execution and is not adjusted automatically.

Sampling

Option	Default	What it controls
`reasoning_effort`	`null` → lowest supported thinking level	Model reasoning effort. `null` selects the lowest supported thinking level; `off` disables thinking where supported. Levels: `minimal`, `low`, `medium`, `high`, `xhigh`, `max`, depending on the model. Qwen3.5-4B, Qwen3.6-35B-A3B, Qwen3.5-397B-A17B, and Nemotron-3.5-Lightning-30B-A3B support only on/off; `null` selects `on`. `list_options` reports each model’s `choices`; other values return 422.
`temperature`	`1.0`	Sampling temperature; defaults to 1.0 (0–2).
`top_p`	`1.0`	Nucleus sampling probability; `null` preserves the model default (0–1).
`top_k`	`-1`	Sample from the top k tokens; `-1` disables the limit, and `null` preserves the model default.

Validation during training

Option	Default	What it controls
`val_enabled`	`true`	Enable validation during training.
`val_interval_steps`	`1`	Training steps between validations; `0` disables validation (0–100). When using `0`, omit `val_enabled`; an explicit `true` is rejected.
`val_num_tasks`	`50`	Maximum distinct tasks per evaluation. Standalone evals run every task in the selected scope when omitted.
`val_trajectories_per_task`	`1`	Number of trajectories generated for each task in a validation round.
`val_max_concurrent_trajectories`	`50`	Maximum number of validation trajectories running concurrently.
`val_final_wait_timeout_seconds`	`7200`	Maximum seconds to wait for outstanding validation rounds at training end; `0` stops waiting immediately.
`val_reasoning_effort`	`null` → training setting	Validation reasoning effort. `null` inherits training defaults. Supports the same model-specific levels and on/off options as `reasoning_effort`.
`val_temperature`	`null` → training setting	Validation sampling temperature (0–2). `null` inherits training defaults.
`val_top_p`	`null` → training setting	Validation nucleus sampling probability (0–1). `null` inherits training defaults.
`val_top_k`	`null` → training setting	Validation top-k token limit. `null` inherits training defaults; `-1` disables it.
`val_max_output_tokens_per_turn`	`null` → training setting	Maximum output tokens per turn in each validation trajectory. `null` inherits training defaults.
`val_max_output_tokens_per_trajectory`	`null` → training setting	Maximum output tokens across one validation trajectory. `null` inherits training defaults.
`val_max_turns_per_trajectory`	`null` → training setting	Maximum agent and environment turns per validation trajectory. `null` inherits training defaults.
`val_trajectory_timeout_seconds`	`null` → training setting	Maximum execution seconds per validation trajectory, including grading. `null` inherits training defaults.

Reduce trajectory spend

Training is asynchronous: trajectories keep running up to max_concurrent_trajectories, including while an optimizer step is in progress. Each step trains on batch_size complete groups that pass filtering. Groups whose trajectories all pass, all fail, or tie carry no learning signal, so by default they are discarded after using tokens.

Concurrency. The default oversamples by 2×. Lower max_concurrent_trajectories to spend less on trajectories the trainer may not use, but keep it above batch_size × group_size so collection stays ahead of training.
Staleness. Trajectories that started more than max_staleness steps before the current step are discarded: running ones are cancelled and finished ones are skipped. Lower values keep training closer to on-policy but discard more work. 0 cancels every trajectory still running when a step finishes.
Validation. With the defaults, a 100-step run validates 101 times on up to 50 tasks, about 5,000 extra trajectories. Set val_interval_steps to 5 or 10, lower val_num_tasks, or set val_interval_steps to 0 and evaluate checkpoints after training.

run = client.training.create(
    bench_id=bench_id,
    base_model_slug="YOUR_SUPPORTED_TRAINING_MODEL_SLUG",
    options={
        "num_steps": 50,
        "max_concurrent_trajectories": 384,
        "val_interval_steps": 10,
        "val_num_tasks": 25,
    },
)

Get started

Build and improve

Production learning

Reference

1. Choose a model

2. Start

3. Watch progress and pick a checkpoint

4. Resume from a checkpoint

Training options

Optimization

Training filters

Trajectories

Sampling

Validation during training

Reduce trajectory spend

​1. Choose a model

​2. Start

​3. Watch progress and pick a checkpoint

​4. Resume from a checkpoint

​Training options

​Optimization

​Training filters

​Trajectories

​Sampling

​Validation during training

​Reduce trajectory spend

1. Choose a model

2. Start

3. Watch progress and pick a checkpoint

4. Resume from a checkpoint

Training options

Optimization

Training filters

Trajectories

Sampling

Validation during training

Reduce trajectory spend