Quickstart - Trajectory

This starts an evaluation on a spec (benchmark) you already uploaded. To upload one, choose or create an agent, then follow Bring your spec. The cookbook has worked spec integrations.

1. Install

You need Python 3.11 or later and an API key from Settings → API keys.

pip install --upgrade trajectory-sdk

uv add trajectory-sdk

export TRAJECTORY_API_KEY="YOUR_TRAJECTORY_API_KEY"

from trajectory import Client

client = Client()
print(client.organizations.retrieve_current())

2. Choose a spec and model

Use the agent and spec names from your upload. The pair selects the latest uploaded version under that agent. List the available base models:

from trajectory import Client

client = Client()
for model in client.evals.list_models().items:
    print(model.base_model_slug)

3. Start the evaluation

from uuid import uuid4

from trajectory import Client

client = Client()
agent_name = "YOUR_AGENT_NAME"
bench_name = "YOUR_BENCHMARK_NAME"
base_model_slug = "Qwen/Qwen3.5-4B"

run = client.evals.create(
    agent_name=agent_name,
    benchmark_name=bench_name,
    base_model_slug=base_model_slug,
    display_name="Baseline evaluation",
    idempotency_key=str(uuid4()),
)
print(run.eval_run_id)
print(client.evals.runs.progress(run.eval_run_id).status)

Poll progress until the evaluation is completed, failed, or cancelled. After completion, retrieve the recorded score:

result = client.evals.runs.retrieve(run.eval_run_id)
print("Mean reward:", result.reward_mean)

Finish a baseline/checkpoint comparison before uploading another version under the same name. Use bench_id instead of the name pair when selecting a specific spec version. Open the run in the platform to read trajectories; Use the platform shows the run URL. Next: Train a model, then Deploy a model.