Skip to content

Troubleshooting

This page helps you diagnose common issues on a self-hosted XTM One instance. Filigran SaaS issues are handled by Filigran support.

Start every investigation with the health probes and logs described in Monitoring.

First steps

Before digging into a specific symptom, gather the basics:

docker compose ps                       # are all services healthy?
curl -s localhost:4000/api/health/ready # 200 = fully ready
docker compose logs platform --tail 100
docker compose logs worker --tail 100

Most startup and runtime problems surface in the platform or worker logs. Set LOG_LEVEL=debug temporarily for more detail, then revert.

The platform will not start

Readiness never turns green

If /api/health/ready keeps returning a non-200 status, the platform cannot reach one of its dependencies or migrations are still running.

  • Confirm PostgreSQL, Redis, and MinIO/S3 are healthy in docker compose ps.
  • Check the connection settings (DATABASE_URL, REDIS_URL, S3_ENDPOINT) match the running services.
  • Watch the logs for migration progress — a large existing database can take time on the first upgrade.

Migration errors on startup

Migrations run automatically on startup. If startup fails inside a migration, the logs name the failing revision.

  • On a multi-replica deployment, startup migrations are serialized — a replica may wait for another to finish. This is expected.
  • Never run migrations manually against production. If a migration fails repeatedly, capture the log and see the Upgrade rollback procedure.

Decryption or login failures after a restart

If integrations stop working or the admin cannot log in after a restart, SECRET_KEY most likely changed.

SECRET_KEY is permanent

SECRET_KEY encrypts every stored credential and signs sessions. If it changes, all encrypted data becomes unrecoverable and existing sessions are invalidated. Restore the original value — there is no recovery path once it is lost.

Admin cannot sign in

The admin account is reset to the .env values on every startup.

  • Confirm ADMIN_EMAIL and ADMIN_PASSWORD in .env, then restart to re-apply them.
  • If you changed the password in the UI, the next restart resets it back to the .env value — keep them in sync.

Chat or streaming does not work behind a proxy

If chat hangs, never streams, or disconnects, the reverse proxy is usually the cause.

  • Ensure the proxy forwards the WebSocket Upgrade header.
  • Raise the proxy read timeout — LLM calls can take 60 seconds or more.
  • Forward X-Forwarded-Proto and X-Forwarded-For so OAuth callbacks and audit logs work.

See the ingress requirements in Installation.

Agents give no answer or fail immediately

  • Confirm at least one AI provider is configured under Settings → AI Models with a valid API key.
  • Check outbound HTTPS access from the platform (and worker) containers to your AI provider. Air-gapped deployments must use a local provider such as Ollama.
  • Check your quota. If the agentic execution quota is exhausted, chat is refused with "You have reached your agentic execution quota for the current period" and assignment runs fail with a Quota exceeded error. Users see their usage in My Profile → Quotas; admins manage limits and per-user overrides in Settings → Quotas. Wait for the period to reset or raise the limit.
  • Inspect the run in Traces or the agent's Activity tab to see which tool or model call failed.

Assignments do not run

Assignments and their delays are driven by the worker's orchestration reconciler and the platform's scheduler.

  • Confirm the worker container is running and not crash-looping.
  • Confirm Redis is reachable — the task queue and scheduling rely on it.
  • Check the run's quota. A run whose owner is over the agentic quota is marked failed with a Quota exceeded error rather than executing — verify usage in Settings → Quotas.
  • A run stuck for a long time is swept and marked failed automatically; re-trigger it after fixing the underlying cause.

Knowledge base uploads fail

  • Large uploads need a permissive proxy body-size limit (client_max_body_size 50m; in Nginx).
  • Confirm the S3/MinIO credentials and bucket are correct and the endpoint is reachable from both the platform and worker.

PostgreSQL connection errors under load

Errors about too many connections mean the pool budget exceeds PG_MAX_CONNECTIONS. Recalculate using the formula in Configuration, accounting for every web worker and every scaled worker container.

Getting help

When opening an issue or asking the community, include:

  • the platform version (shown in the UI and container image tag)
  • relevant platform and worker log excerpts (with secrets redacted)
  • the output of docker compose ps and /api/health/ready

Community support

For questions the documentation does not answer, join the Filigran Community on Slack or open an issue on GitHub.

Next step

Review the Upgrade procedure before your next version bump, and keep Monitoring alerts in place to catch issues early.