> ## Documentation Index
> Fetch the complete documentation index at: https://docs.trajectory.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Run Trajectory from your terminal

> Use the Trajectory CLI to sign in, prepare a benchmark, validate tasks, launch evaluations and training, and reconnect to work from scripts or agents.

The Trajectory CLI shares credentials and workflows with the Python SDK. Start with a
small benchmark and a recorded grade, then use the same commands from a coding agent
or a script. The primary command is `tj`; `trajectory` remains available as an alias.

## Install and sign in

You need Python 3.11 or later. Install the executable with either command:

```bash theme={null}
uv tool install trajectory-sdk
```

```bash theme={null}
pip install --upgrade trajectory-sdk
```

```bash theme={null}
tj login
tj whoami
```

If you installed with `uv add trajectory-sdk` in a project, use `uv run tj`.
See [Authentication](https://docs.trajectory.ai/authentication.md) for profiles, SSH sessions, and CI credentials.

## Prepare a project

```bash theme={null}
tj init
```

Interactive setup starts with two choices:

* **Try an open-source benchmark**: choose a cookbook example and follow its diagnostic,
  evaluation, and training steps where supported.
* **Bring your own benchmark**: use the current directory, choose another local
  directory, or provide a repository URL. The current directory is selected by default.

Install `trajectory-sdk[tui]` for the arrow-key menus. Use `--directory PATH` to select a
project directly. When this directory has saved setup, **Resume saved setup** appears
first. Press Esc to cancel without changing project settings. Explicit source or assistant
flags skip the first menu; scripts continue to use saved setup without prompting.

For a script, supply the source explicitly:

```bash theme={null}
tj init --cookbook gsm8k --assistant copy --json
```

Setup creates `trajectory.toml` and starts no remote work. You can provide `--agent-id`
to save the owner for subsequent uploads.

### Bring a customer benchmark

Point setup at the real benchmark and choose the assistant you want to work with:

```bash theme={null}
tj init --repo ./customer-benchmark --assistant codex
tj init --repo https://github.com/organization/benchmark --assistant claude
tj init --repo git@github.com:organization/private-benchmark.git --assistant gemini
```

A local source becomes the working directory, so you are not asked to select it
again. If the agent and benchmark live in separate projects, use
`--directory ./my-agent --repo ./customer-benchmark`. The benchmark directory does
not need to be a Git checkout.

For a repository link, select a working directory after choosing the assistant.
The assistant looks for an existing checkout or prepares one in a new subdirectory
using your existing Git authentication. HTTPS links and Git SSH addresses are
supported; use links without embedded tokens, query parameters, or fragments.
A link to a branch or subdirectory stays in the prompt so the assistant can resolve
the intended revision and path.

Setup saves the source in `trajectory.toml`; `tj init` offers to resume it. It does not clone
or run repository code itself. Use a separate setup directory to switch sources.
For a prompt you can use in an existing assistant session:

```bash theme={null}
tj init --repo ./customer-benchmark --assistant copy --json
```

The assistant inspects your task loader, agent entry point, tools, runtime, grader,
and result reader before asking for missing inputs. It helps reproduce one native
task, package the real harness, compare a [managed diagnostic](https://docs.trajectory.ai/guides/spec.md) with the
[native grade](https://docs.trajectory.ai/guides/source-graders.md), and run a small
[held-out evaluation](https://docs.trajectory.ai/guides/evaluations.md). It preserves your agent and grading
behavior, reviews remote submissions with you, and reports observed results and
commands to resume. Private data access or missing runtime dependencies may still
need your input.

### Continue with a coding assistant

Choose **Claude Code**, **Codex**, or **Gemini CLI**. If you chose the current directory
or a local benchmark directory, the assistant opens there. For a cookbook example or
repository URL, select a project directory after choosing the assistant;
`--directory PATH` skips that prompt. Copy/manual setup uses the current directory unless
you supply `--directory`. Installed assistants appear first. Trajectory opens the selected assistant in your
project directory with the bundled integration skill, the setup task, and the
selected profile and API origin. The assistant uses its existing login and normal
permission settings. Your Trajectory credential is not included in the prompt.

Select an assistant directly:

```bash theme={null}
tj init --assistant codex
tj init --directory ./my-agent --template benchmark --assistant claude --profile staging
tj init --directory ./my-agent --assistant gemini
```

The assistant inspects your code, helps adapt and validate the integration, and
asks before submitting remote work. Closing its session returns to the CLI;
`finished` means the assistant exited successfully, not that an evaluation passed.
Run `tj init` again to continue, including when the project already has a saved
setup choice.

Choose **Copy setup prompt** to use another assistant or an existing session.
If the local clipboard is unavailable, the CLI prints the prompt. Missing
assistants show their installation link and the same prompt. **Continue manually**
(or `--assistant manual`) keeps the command-based instructions.

Interactive assistants never launch with `--json`, `--non-interactive`, or piped
input/output. To generate a prompt for automation:

```bash theme={null}
tj init --directory ./my-agent --template benchmark --assistant copy --json
```

The JSON result includes `assistant.prompt`. It does not access the clipboard or
start an assistant. A failed assistant launch retains the project files so you can
retry; a successful login remains saved even if its subsequent setup is interrupted.

For an existing repository, you can also ask your coding agent to follow the bundled integration skill:

```bash theme={null}
tj docs --topic skill
```

Your agent still needs to adapt and verify the benchmark's native tools and grader.
The setup command does not automatically convert an arbitrary repository.

## Start from the cookbook

Choose **Try an open-source benchmark** in `tj init`, or select an example directly:

```bash theme={null}
tj init --cookbook gsm8k --assistant codex
```

Choose your assistant before selecting the project directory. Setup saves the example
in `trajectory.toml`, so `tj init` can resume it. Use another directory to start a
different example. The assistant locates an existing cookbook checkout or helps clone
one into a new subdirectory, then reads the example's current guide and code.

| `--cookbook` value | Example | What to expect |
| - | - | - |
| `gsm8k` | [GSM8K](https://github.com/Trajectorylabs/trajectory-cookbook/blob/main/examples/gsm8k/README.md) | Recommended first training run: math tasks with an answer-submission tool. |
| `t-factory` | [T Factory](https://github.com/Trajectorylabs/trajectory-cookbook/blob/main/examples/t_factory/README.md) | A smaller introduction to responses, rewards, and training. |
| `harvey-lab` | [Harvey LAB](https://github.com/Trajectorylabs/trajectory-cookbook/blob/main/examples/harvey_labs/README.md) | Legal research with external judge credentials and nested Podman. |
| `tau3-banking` | [Tau 3 Banking](https://github.com/Trajectorylabs/trajectory-cookbook/blob/main/examples/tau3_banking/README.md) | Multi-turn banking with an external user simulator and judge. |
| `swe-bench-pro` | [SWE-Bench Pro](https://github.com/Trajectorylabs/trajectory-cookbook/blob/main/examples/swe_bench_pro/README.md) | Code repair with large images and long-running tasks. |
| `terminal-bench-4` | [Terminal-Bench 4.0](https://github.com/Trajectorylabs/trajectory-cookbook/blob/main/examples/terminal_bench_4/README.md) | Evaluation only; this example has no official training split. |

To get the same example-specific prompt for another coding assistant:

```bash theme={null}
tj init --cookbook gsm8k --assistant copy --json --directory ./my-project
```

Read `assistant.prompt` in the JSON result. This command prepares local settings and
guidance; it does not clone a repository, access the clipboard, or submit remote work.
Use `--assistant manual --non-interactive` for a list of next steps instead.

The prompt guides you through prerequisites, a [graded diagnostic](https://docs.trajectory.ai/guides/spec.md), a
small [baseline evaluation](https://docs.trajectory.ai/guides/evaluations.md), and, for examples with training
support, a [training run and checkpoint comparison](https://docs.trajectory.ai/guides/training.md). It asks you to
review the model, task selection, run settings, and budget before submitting work.
Some cookbook scripts combine ingestion with diagnostics or evaluation with training;
the assistant checks those side effects before recommending a command.

Setup and assistant output link to the relevant Trajectory guides as you progress.
Training is opt-in, and a small smoke run does not establish that a model improved.

## Upload and validate

For cookbook examples, follow the selected example's guide to create and upload its
benchmark. If it already returned a benchmark ID, use that ID for the readiness and
diagnostic commands below.

For a customer benchmark, prepare a [BenchmarkSpec](https://docs.trajectory.ai/guides/spec.md) from your real tasks,
runtime, harness, and grader. Ask your coding assistant to save it as a JSON manifest,
or serialize your `BenchmarkSpec` with `spec.model_dump_json(indent=2)`.
`tj init` saves project settings; it does not generate this manifest for cookbook or
customer benchmarks. The upload command below assumes you saved it as `manifest.json`.

List agents and reuse a suitable one, or create an agent for this project:

```bash theme={null}
tj agents list --json
tj agents create --name my-benchmark
```

Replace `AGENT_ID` with the returned agent ID and `manifest.json` with your manifest's path:

```bash theme={null}
tj benchmarks push --manifest manifest.json --agent-id AGENT_ID --json
```

The response includes an `operation_id`. Wait for registration and image building,
then inspect readiness using the operation's `bench_id`:

```bash theme={null}
tj operations watch OPERATION_ID --timeout 300
tj benchmarks readiness BENCH_ID
tj benchmarks diagnose BENCH_ID --json
```

Use the returned `benchmark_diagnostic_id` to inspect the diagnostic:

```bash theme={null}
tj diagnostics watch DIAGNOSTIC_ID --timeout 300
tj diagnostics results DIAGNOSTIC_ID
```

Check recorded outcomes and failures before scaling up. A successful image build alone
does not verify the harness or grader. Diagnostics are explicit submissions and do not
automatically retry a lost submission response.

### Set diagnostic limits

Discover diagnostic models and options, excluding the task counts controlled by the service:

```bash theme={null}
tj diagnostics models --json
tj diagnostics options --bench-id BENCH_ID --json
```

Save `diagnostic.json` with a model from discovery and limits appropriate to your harness:

```json diagnostic.json theme={null}
{
  "base_model_slug": "MODEL_FROM_OPTIONS",
  "options": {
    "max_turns_per_trajectory": 4,
    "max_output_tokens_per_turn": 2048
  }
}
```

Validate locally, then submit when ready:

```bash theme={null}
tj benchmarks diagnose BENCH_ID --config diagnostic.json --dry-run --json
tj benchmarks diagnose BENCH_ID --config diagnostic.json --json
tj diagnostics watch DIAGNOSTIC_ID --timeout 300
tj diagnostics results DIAGNOSTIC_ID --json
```

The example limits above are a small smoke budget; preserve your source agent's budgets
for a comparable evaluation. Diagnostics run up to ten tasks with one attempt each.
Omit `val_num_tasks` and `val_trajectories_per_task`. You can set
`val_max_concurrent_trajectories` to limit concurrency; the service caps it at the number
of diagnostic tasks. This requires an API release that supports diagnostic concurrency.
A dry run validates the local schema without contacting the API; model availability,
option bounds, and benchmark readiness are checked by the service.

An HTTP 409 from `diagnostics results` means the diagnostic has not finished.
Reconnect with `watch`, then request results; do not submit another diagnostic to poll.

## Run a baseline evaluation

Discover models and the selected benchmark's supported options:

```bash theme={null}
tj evals models --json
tj evals options --bench-id BENCH_ID --json
```

Create `eval.json`. Replace `BENCH_ID` and `MODEL_FROM_OPTIONS` with values from those
responses and use options within the returned bounds:

```json eval.json theme={null}
{
  "bench_id": "BENCH_ID",
  "base_model_slug": "MODEL_FROM_OPTIONS",
  "options": {
    "val_num_tasks": 1,
    "val_trajectories_per_task": 1
  }
}
```

Validate the local schema without submitting, then launch:

```bash theme={null}
tj evals create --config eval.json --dry-run --json
tj evals create --config eval.json --json
tj evals watch EVAL_RUN_ID --timeout 300
tj evals details EVAL_RUN_ID --json
```

`--dry-run` checks the local request schema. It does not prove that a benchmark is ready,
a model is available, or your account has sufficient capacity or credits.

## Train and inspect

Start from a successful baseline and discover training-specific models and options:

```bash theme={null}
tj training options --bench-id BENCH_ID --json
tj schema training --json
```

Prepare a training JSON file using that schema and the returned option bounds. Training
is a separate explicit action:

```bash theme={null}
tj training create --config training.json --dry-run
tj training create --config training.json --json
tj training watch TRAINING_RUN_ID --timeout 300
tj training details TRAINING_RUN_ID --json
```

Run lists require a benchmark selection:

```bash theme={null}
tj evals list --bench-id BENCH_ID --json
tj training list --bench-id BENCH_ID --json
```

Use `--cursor` with the returned pagination cursor for later pages. To request cancellation:

```bash theme={null}
tj training cancel TRAINING_RUN_ID
```

Stopping a local `watch` command or closing the terminal does not cancel the remote run.

## Recover a submission

The CLI records a request ID before submitting an eval, training run, or benchmark upload.
Repeat the same command to recover that submission, including after a lost connection.
An identical run configuration reuses its receipt even after the submission succeeded.
To intentionally launch another run with the same config, choose a new request ID:

```bash theme={null}
tj evals create --config eval.json --request-id baseline-repeat-2 --json
```

Do not reuse that ID for a different configuration. Benchmark upload identity also
includes packaged file contents, so a changed harness creates a new upload. Receipts
are scoped to the credential and API destination and live outside the project.
`tj operations list` shows local submission receipts; `get` and `watch` retrieve
remote ingestion operations by ID.

## Coding agents and scripts

Use `--json` for one JSON result on stdout. Progress and login instructions go to stderr.
Ordinary commands never prompt when stdin is not a terminal; `--non-interactive` also
disables prompts explicitly. Authentication approval still requires a person.

```bash theme={null}
tj schema --json
tj schema benchmark --json
tj schema evals --json
tj docs --json
tj docs --topic index --json
```

The `directory` field from `docs --topic index` locates bundled guides in both wheel and
editable installs. In the cookbook picker, Esc returns to source selection; Ctrl+C exits
without writing setup files.

| Exit code | Meaning |
| - | - |
| `0` | Command succeeded. A submission may still be running remotely. |
| `1` | API, connection, dependency, or local error, or a watched operation finished failed, cancelled, or partially failed. |
| `2` | Invalid arguments or configuration. |
| `3` | Authentication or access error. |
| `4` | Local wait deadline expired; inspect the remote operation before retrying. |
| `130` | Interrupted locally. |

## Capture traces or generate a test fixture

To instrument an existing agent for [production trace capture](https://docs.trajectory.ai/guides/production-traces.md),
start that separate flow explicitly:

```bash theme={null}
tj init --template connect --assistant codex
```

Capture requires private-beta access. The assistant checks availability and helps verify
one captured trace; selecting this flow does not automatically connect or evaluate the agent.

For a small local test fixture, `tj init --template example --non-interactive` generates
an addition benchmark under `trajectory-example/`. Repeating setup preserves existing files.
These specialized flows are available through flags rather than the benchmark selection menu.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.