Skip to main content
The Trajectory CLI shares credentials and workflows with the Python SDK. Start with a small benchmark and a recorded grade, then use the same commands from a coding agent or a script. The primary command is tj; trajectory remains available as an alias.

​
Install and sign in

You need Python 3.11 or later. Install the executable with either command:
If you installed with uv add trajectory-sdk in a project, use uv run tj. See Authentication for profiles, SSH sessions, and CI credentials.

​
Prepare a project

Interactive setup starts with two choices:
  • Try an open-source benchmark: choose a cookbook example and follow its diagnostic, evaluation, and training steps where supported.
  • Bring your own benchmark: use the current directory, choose another local directory, or provide a repository URL. The current directory is selected by default.
Install trajectory-sdk[tui] for the arrow-key menus. Use --directory PATH to select a project directly. When this directory has saved setup, Resume saved setup appears first. Press Esc to cancel without changing project settings. Explicit source or assistant flags skip the first menu; scripts continue to use saved setup without prompting. For a script, supply the source explicitly:
Setup creates trajectory.toml and starts no remote work. You can provide --agent-id to save the owner for subsequent uploads.

​
Bring a customer benchmark

Point setup at the real benchmark and choose the assistant you want to work with:
A local source becomes the working directory, so you are not asked to select it again. If the agent and benchmark live in separate projects, use --directory ./my-agent --repo ./customer-benchmark. The benchmark directory does not need to be a Git checkout. For a repository link, select a working directory after choosing the assistant. The assistant looks for an existing checkout or prepares one in a new subdirectory using your existing Git authentication. HTTPS links and Git SSH addresses are supported; use links without embedded tokens, query parameters, or fragments. A link to a branch or subdirectory stays in the prompt so the assistant can resolve the intended revision and path. Setup saves the source in trajectory.toml; tj init offers to resume it. It does not clone or run repository code itself. Use a separate setup directory to switch sources. For a prompt you can use in an existing assistant session:
The assistant inspects your task loader, agent entry point, tools, runtime, grader, and result reader before asking for missing inputs. It helps reproduce one native task, package the real harness, compare a managed diagnostic with the native grade, and run a small held-out evaluation. It preserves your agent and grading behavior, reviews remote submissions with you, and reports observed results and commands to resume. Private data access or missing runtime dependencies may still need your input.

​
Continue with a coding assistant

Choose Claude Code, Codex, or Gemini CLI. If you chose the current directory or a local benchmark directory, the assistant opens there. For a cookbook example or repository URL, select a project directory after choosing the assistant; --directory PATH skips that prompt. Copy/manual setup uses the current directory unless you supply --directory. Installed assistants appear first. Trajectory opens the selected assistant in your project directory with the bundled integration skill, the setup task, and the selected profile and API origin. The assistant uses its existing login and normal permission settings. Your Trajectory credential is not included in the prompt. Select an assistant directly:
The assistant inspects your code, helps adapt and validate the integration, and asks before submitting remote work. Closing its session returns to the CLI; finished means the assistant exited successfully, not that an evaluation passed. Run tj init again to continue, including when the project already has a saved setup choice. Choose Copy setup prompt to use another assistant or an existing session. If the local clipboard is unavailable, the CLI prints the prompt. Missing assistants show their installation link and the same prompt. Continue manually (or --assistant manual) keeps the command-based instructions. Interactive assistants never launch with --json, --non-interactive, or piped input/output. To generate a prompt for automation:
The JSON result includes assistant.prompt. It does not access the clipboard or start an assistant. A failed assistant launch retains the project files so you can retry; a successful login remains saved even if its subsequent setup is interrupted. For an existing repository, you can also ask your coding agent to follow the bundled integration skill:
Your agent still needs to adapt and verify the benchmark’s native tools and grader. The setup command does not automatically convert an arbitrary repository.

​
Start from the cookbook

Choose Try an open-source benchmark in tj init, or select an example directly:
Choose your assistant before selecting the project directory. Setup saves the example in trajectory.toml, so tj init can resume it. Use another directory to start a different example. The assistant locates an existing cookbook checkout or helps clone one into a new subdirectory, then reads the example’s current guide and code. To get the same example-specific prompt for another coding assistant:
Read assistant.prompt in the JSON result. This command prepares local settings and guidance; it does not clone a repository, access the clipboard, or submit remote work. Use --assistant manual --non-interactive for a list of next steps instead. The prompt guides you through prerequisites, a graded diagnostic, a small baseline evaluation, and, for examples with training support, a training run and checkpoint comparison. It asks you to review the model, task selection, run settings, and budget before submitting work. Some cookbook scripts combine ingestion with diagnostics or evaluation with training; the assistant checks those side effects before recommending a command. Setup and assistant output link to the relevant Trajectory guides as you progress. Training is opt-in, and a small smoke run does not establish that a model improved.

​
Upload and validate

For cookbook examples, follow the selected example’s guide to create and upload its benchmark. If it already returned a benchmark ID, use that ID for the readiness and diagnostic commands below. For a customer benchmark, prepare a BenchmarkSpec from your real tasks, runtime, harness, and grader. Ask your coding assistant to save it as a JSON manifest, or serialize your BenchmarkSpec with spec.model_dump_json(indent=2). tj init saves project settings; it does not generate this manifest for cookbook or customer benchmarks. The upload command below assumes you saved it as manifest.json. List agents and reuse a suitable one, or create an agent for this project:
Replace AGENT_ID with the returned agent ID and manifest.json with your manifest’s path:
The response includes an operation_id. Wait for registration and image building, then inspect readiness using the operation’s bench_id:
Use the returned benchmark_diagnostic_id to inspect the diagnostic:
Check recorded outcomes and failures before scaling up. A successful image build alone does not verify the harness or grader. Diagnostics are explicit submissions and do not automatically retry a lost submission response.

​
Set diagnostic limits

Discover diagnostic models and options, excluding the task counts controlled by the service:
Save diagnostic.json with a model from discovery and limits appropriate to your harness:
diagnostic.json
Validate locally, then submit when ready:
The example limits above are a small smoke budget; preserve your source agent’s budgets for a comparable evaluation. Diagnostics run up to ten tasks with one attempt each. Omit val_num_tasks and val_trajectories_per_task. You can set val_max_concurrent_trajectories to limit concurrency; the service caps it at the number of diagnostic tasks. This requires an API release that supports diagnostic concurrency. A dry run validates the local schema without contacting the API; model availability, option bounds, and benchmark readiness are checked by the service. An HTTP 409 from diagnostics results means the diagnostic has not finished. Reconnect with watch, then request results; do not submit another diagnostic to poll.

​
Run a baseline evaluation

Discover models and the selected benchmark’s supported options:
Create eval.json. Replace BENCH_ID and MODEL_FROM_OPTIONS with values from those responses and use options within the returned bounds:
eval.json
Validate the local schema without submitting, then launch:
--dry-run checks the local request schema. It does not prove that a benchmark is ready, a model is available, or your account has sufficient capacity or credits.

​
Train and inspect

Start from a successful baseline and discover training-specific models and options:
Prepare a training JSON file using that schema and the returned option bounds. Training is a separate explicit action:
Run lists require a benchmark selection:
Use --cursor with the returned pagination cursor for later pages. To request cancellation:
Stopping a local watch command or closing the terminal does not cancel the remote run.

​
Recover a submission

The CLI records a request ID before submitting an eval, training run, or benchmark upload. Repeat the same command to recover that submission, including after a lost connection. An identical run configuration reuses its receipt even after the submission succeeded. To intentionally launch another run with the same config, choose a new request ID:
Do not reuse that ID for a different configuration. Benchmark upload identity also includes packaged file contents, so a changed harness creates a new upload. Receipts are scoped to the credential and API destination and live outside the project. tj operations list shows local submission receipts; get and watch retrieve remote ingestion operations by ID.

​
Coding agents and scripts

Use --json for one JSON result on stdout. Progress and login instructions go to stderr. Ordinary commands never prompt when stdin is not a terminal; --non-interactive also disables prompts explicitly. Authentication approval still requires a person.
The directory field from docs --topic index locates bundled guides in both wheel and editable installs. In the cookbook picker, Esc returns to source selection; Ctrl+C exits without writing setup files.

​
Capture traces or generate a test fixture

To instrument an existing agent for production trace capture, start that separate flow explicitly:
Capture requires private-beta access. The assistant checks availability and helps verify one captured trace; selecting this flow does not automatically connect or evaluate the agent. For a small local test fixture, tj init --template example --non-interactive generates an addition benchmark under trajectory-example/. Repeating setup preserves existing files. These specialized flows are available through flags rather than the benchmark selection menu.