Skip to content

Development

This page covers how to set up a local development environment and contribute to XTM One.

Prerequisites

  • Docker with Docker Compose 2.24 or later, or Podman (see Podman)
  • Git
  • To run the checks and tests on the host: uv, which also provides Python 3.14, and Node.js 24 (22.18 at minimum) with Yarn

Local setup

# Clone the repository
git clone https://github.com/XTM-One-Platform/xtm-one.git && cd xtm-one

# Copy environment file
cp .env.sample .env

# Run the full stack: PostgreSQL + pgvector, Redis, Silo, platform, worker, frontend
docker compose -f docker-compose.yml -f docker-compose.dev.yml up --build

docker-compose.dev.yml is an override of the production docker-compose.yml: the same stack, with the platform and the worker running your checkout's backend sources and a Vite dev server in place of the built frontend. To drop the -f flags, set COMPOSE_FILE=docker-compose.yml:docker-compose.dev.yml in .env (; as the separator on Windows); the commands below then read docker compose ....

The root Makefile wraps these commands: make up, make start, make logs s=platform, make psql, make down (make lists them all). It runs podman compose, or docker compose when podman is not installed; force one with ENGINE=docker, add profiles with PROFILES="tools ha". make up and make start first check that the stack's host ports are free: when another checkout or process holds one, they stop, name it, and print a free set of host ports for .env.

The frontend is at http://localhost:3100 and the platform API at http://localhost:8100. Both run with hot reload: the platform restarts on a change under backend/app or backend/alembic, the worker on a change under backend/app, and Vite updates the page in place. The platform migrates the database on startup.

  • Run up --build again after a change to backend/pyproject.toml or backend/uv.lock. Frontend dependencies install on every start of the frontend service.
  • Start it in the background with up -d --build, follow it with logs -f platform worker frontend, stop it with stop.
  • Every .env setting reaches the platform and the worker. REDIS_URL and S3_ENDPOINT are the stack's own services and override .env (a Redis password set there is not used); OPENSANDBOX_DOMAIN, BASE_URL and FRONTEND_URL default to the stack's own addresses.
  • Every checkout, git worktrees included, is a compose project of its own, with its own database, Redis and bucket. To run two checkouts at once, give the second one free host ports in its .env (see Host ports).
  • Where bind mounts deliver no file change events (podman machine, or a Windows checkout under Docker Desktop), set WATCHFILES_FORCE_POLLING=true and CHOKIDAR_USEPOLLING=true in .env.
  • Open a database shell with docker compose -f docker-compose.yml -f docker-compose.dev.yml exec postgres psql -U copilot. PostgreSQL, Redis and Silo also listen on the host for the tests.
  • Every port is published on 127.0.0.1 only. To reach the stack from another device (a phone, a second laptop), set DEV_BIND_ADDRESS=0.0.0.0 in .env: PostgreSQL, Redis, Silo and the admin account then answer the network with their dev credentials.

Host ports

Each host port the stack publishes is set by a .env variable:

Variable Default Service
DEV_FRONTEND_PORT 3100 Vite dev server (FRONTEND_URL follows it)
DEV_PLATFORM_PORT 8100 Platform API (BASE_URL follows it)
DEV_POSTGRES_PORT 5432 PostgreSQL
DEV_REDIS_PORT 6380 Redis
DEV_SILO_PORT 9002 Silo S3 API
DEV_SILO_CONSOLE_PORT 9003 Silo console
DEV_REDISINSIGHT_PORT 5540 RedisInsight (tools profile)

A second checkout running beside the first, for example a git worktree, takes another set in its own .env:

DEV_FRONTEND_PORT=3110
DEV_PLATFORM_PORT=8110
DEV_POSTGRES_PORT=5442
DEV_REDIS_PORT=6390
DEV_SILO_PORT=9012
DEV_SILO_CONSOLE_PORT=9013
DEV_REDISINSIGHT_PORT=5550

Its frontend is then at http://localhost:3110. Tests run from the host against that stack take the matching DATABASE_URL, REDIS_URL and S3_ENDPOINT; scripts/ci-local.sh reads them from the running stack.

Optional profiles

Profile What it adds
tools RedisInsight on http://localhost:5540.
ha A Redis Sentinel topology for failover testing; the header of docker-compose.dev.yml lists the .env settings that switch the backend to it.

Enable one with --profile <name>, or with COMPOSE_PROFILES=<name> in .env.

OpenSandbox

The stack always runs the OpenSandbox server of docker-compose.yml (the opensandbox service), which starts the sandboxes as containers through the container engine's socket. In the dev stack it sits on a network that only the platform and the worker join, so no other container reaches it, and it requires a key: OPENSANDBOX_API_KEY from .env, else copilot-dev-opensandbox. The platform uses it with CUSTOM_TOOL_SANDBOX=opensandbox and OPENSANDBOX_API_KEY set in .env. Build the sandbox images it runs once, backend/sandbox/build-image.sh sandbox-python:0.2 and backend/sandbox/build-image.sh sandbox-nodejs:0.2 (the script needs jq and the docker command: with Podman, install podman-docker), then set OPENSANDBOX_IMAGES=python:sandbox-python:0.2,nodejs:sandbox-nodejs:0.2 in .env.

Podman

The compose files run unchanged with podman compose, whichever provider it uses (docker-compose, or podman-compose 1.6 or later):

podman compose -f docker-compose.yml -f docker-compose.dev.yml up --build
  • Under podman machine (macOS, Windows) the VM's file share delivers no change events: set WATCHFILES_FORCE_POLLING=true and CHOKIDAR_USEPOLLING=true in .env, or nothing reloads.
  • On macOS the VM cannot read a checkout under ~/Documents, ~/Desktop or ~/Downloads (bind mounts fail with operation not permitted) unless you grant it access in System Settings > Privacy & Security > Files and Folders. Keeping the checkout elsewhere also works.
  • Rootless Podman on Linux serves no /var/run/docker.sock, which the opensandbox service mounts, so the stack does not start without it: enable the user socket (systemctl --user enable --now podman.socket) and set CONTAINER_ENGINE_SOCKET=$XDG_RUNTIME_DIR/podman/podman.sock in .env.
  • Bind mounts carry the z option, so they work on SELinux hosts (Fedora, RHEL), and image names are fully qualified (docker.io/...), so Podman's short-name policy never prompts.

Python dependencies

The backend (backend/) and the documentation site (docs/) are uv projects: pyproject.toml declares the dependencies and uv.lock pins every one of them, transitive ones included. In either directory:

uv sync                         # install the locked set into .venv
uv add <package>==<version>     # add a dependency and relock; commit pyproject.toml and uv.lock
uv run <command>                # run a command in that environment

CI and the Docker images install with --locked, which fails when pyproject.toml was changed without relocking.

uv run syncs .venv to the checkout's lock before it runs the command. In a worktree whose backend/.venv is a link to another checkout's venv, that rewrites the shared environment: run uv run --no-sync, or set UV_NO_SYNC=1 in that shell.

Project structure

xtm-one/
├── backend/           → Python / FastAPI (see backend/AGENTS.md)
├── frontend/          → React 19 / TypeScript (see frontend/AGENTS.md)
├── docs/              → Product documentation (MkDocs)
├── apps/              → Platform app packages
└── docker-compose.yml → Production stack

Running checks

# Backend lint
cd backend && uv run ruff check app/

# Frontend lint and type check
cd frontend && yarn check

# Office add-in type check (an npm package of its own; its unit tests run in `yarn check` above)
cd apps/microsoft-office && npm ci && npm run check

# Backend tests
cd backend && SECRET_KEY=test uv run python -m pytest -m "not integration" -q -n auto

The Makefile runs both test suites from the host:

  • make test runs the backend unit tests as CI does, from backend/.venv, which it syncs with uv sync --locked whenever pyproject.toml or uv.lock changes (uv provides the Python of backend/.python-version). Narrow the run with ARGS, e.g. make test ARGS="tests/test_x.py -k name".
  • make e2e runs the Playwright core suite against the running dev stack (make start first), at E2E_BASE_URL from frontend/.env.e2e or http://localhost:3100. It logs in with E2E_USER_EMAIL and E2E_USER_PASSWORD from frontend/.env.e2e (copy frontend/.env.e2e.example). One spec: make e2e ARGS="tests_e2e/x.spec.ts". scripts/ci-local.sh --only e2e runs the suite as CI does, on a production build of its own.

Running tests in parallel

CI runs every backend suite under pytest-xdist (-n auto locally, -n logical in CI), and so can you. CI also splits each suite between several jobs: with PYTEST_SHARD=<index>/<total> a run keeps every <total>-th test of the sorted test ids (tests/sharding.py), so a test runs in exactly one job. The real-browser tests (-m chromium) are the exception to xdist: they time renders and sign-in redirects, so each of their jobs runs them one at a time. What that asks of a test:

  • No test depends on another one. The suite already runs in random order (pytest-randomly); xdist spreads it over processes too.
  • Every process collects the same test ids. A parametrize value built from uuid.uuid4(), time.time() or a freshly generated key turns into a different id in each worker, and the run stops with Different tests were collected between gw0 and gw1. Give such a parametrization ids= (or pytest.param(..., id=...)): the value can stay random, its name cannot.
  • A PostgreSQL test works in a database of its own, through tests/scratch_database.py. The schema is built once per process into a template database and every scratch database is a copy of it, so a new suite should use scratch_database(...) rather than a CREATE DATABASE + create_all of its own.
  • A test reading the backend's own source (a tripwire over every module under app/) reads it through tests/app_sources.py, which parses the tree once per process instead of once per suite.
  • A module that owns a fixed resource stays on one process. tests/test_bootstrap.py starts one backend on port 8099; it is marked pytest.mark.xdist_group(...) so that all its tests run in the same worker. The group only applies under --dist loadgroup, which CI and scripts/ci-local.sh pass: add it to a local -n run (uv run python -m pytest -m integration -n auto --dist loadgroup).

The integration suite needs PostgreSQL, Redis and Silo, and a database role allowed to create databases: ./scripts/ci-local.sh --only integration starts them and runs it the way CI does.

scripts/ci-local.sh runs one at a time per clone: a run started while another is in progress, from any checkout or worktree, prints Another local CI run is in progress and waits for it to finish, since runs share the machine and the database on port 5432. CI_LOCAL_NO_LOCK=1 skips the wait; CONTRIBUTING.md describes the queue.

The Playwright suite runs in CI as several jobs, each with a stack of its own: E2E_SHARD=<index>/<total> deals the spec files of a project to the jobs by the seconds each took on CI (frontend/tests_e2e/shard-timings.json). Refresh the timings after a large change to the suite from the test-results-* artifacts of several green runs (a spec's time moves by tens of percent from one run to the next, so a single run does not predict the next): cd frontend && yarn e2e:shard-timings <run 1>/*/timings.json -- <run 2>/*/timings.json -- ... takes each file's median over the runs.

Live LLM smoke suite (opt-in)

The provider unit tests run against mocked SDK clients, so they cannot tell whether a real API accepts the requests the platform builds. backend/tests/live/ sends those requests to the real Anthropic, Gemini and OpenRouter APIs with tiny prompts. It spends real money (cents per run), never runs in CI, and is skipped unless LIVE_LLM_SMOKE=1 is set and the provider's key is exported as LLM_PROVIDER_ANTHROPIC_API_KEY, LLM_PROVIDER_GEMINI_API_KEY or LLM_PROVIDER_OPENROUTER_API_KEY:

cd backend && LIVE_LLM_SMOKE=1 SECRET_KEY=test uv run python -m pytest tests/live -v -rfEs

A provider whose key is not set is skipped. backend/tests/live/README.md lists the cases, the model overrides and how the keys are kept out of the test output.

Migrations

Schema changes must use Alembic migrations. Never use inline SQL.

cd backend
uv run python -m alembic -c alembic/alembic.ini revision --autogenerate -m "description of change"

The platform auto-migrates on startup — no manual alembic upgrade needed in production.

One head, always

The migration history must have exactly one head. Two pull requests that each add a migration chained off the same parent fork the history the moment both land — and each one is green on its own, because pull request CI validates the branch against main as it was when that run started. This has happened twice on main, and both times it surfaced days later as an unrelated pull request going red.

Three guards catch it:

Guard Where Catches
tests/test_migration_integrity.py Migrations (Alembic) job, every pull request A fork you created yourself
Post-merge integrity check Migrations (Alembic) job, pull requests targeting main A fork created by another migration landing on main after your branch was cut
CI on push to main Whole CI workflow A fork that landed anyway — flagged on the merge commit that created it

If the post-merge check fails, main has moved under you:

git fetch origin main && git rebase origin/main
# re-point your earliest migration's down_revision to main's new head
cd backend && uv run python -m pytest tests/test_migration_integrity.py

When a migration lands on main, every open pull request that touches backend/alembic/versions/ also gets an automatic comment asking for that rebase.

Pull requests stacked on an integration branch

CI runs in full whatever branch you target. The post-merge check is the exception: it merges your branch into main, which only says something when that merge is what your pull request proposes. A fork you introduced is still caught by the first guard above. Rebase the integration branch itself onto main to clear what the post-merge check would have flagged.

Repository settings

The guards above detect a fork; they cannot prevent two green pull requests from being merged seconds apart. That needs a repository setting: either a merge queue on main, or required status checks with "require branches to be up to date". The merge_group triggers in .github/workflows/ci.yml and .github/workflows/e2e.yml are the prerequisite for the merge queue — enabling the queue itself is done in the repository settings, and every required check must run on merge_group events or the queue never drains.

Commit conventions

All commits follow Conventional Commits:

type(scope?): description (#issue)

Types: feat, fix, chore, docs, style, refactor, perf, test, build, ci, revert.

Next step

For product documentation guidelines, see docs/AGENTS.md. For backend and frontend specifics, see backend/AGENTS.md and frontend/AGENTS.md.