> ## Documentation Index
> Fetch the complete documentation index at: https://docs.calado.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# LangSmith
> Forward LangSmith runs to calado with an Automation. No changes to your agent code.
## Overview
If your agent is already traced with [LangSmith](https://smith.langchain.com), calado can consume the runs LangSmith is already collecting. You do not re-instrument your app and you do not add the calado SDK. A LangSmith Automation posts matching runs to calado, and the `format: "langsmith"` adapter regroups the flat run list by `trace_id`, rebuilds the run tree, and produces the same canonical conversation and step events as the direct SDK.
Use this page if your agent is traced with **LangSmith**. If you run LangChain or LangGraph but do not use LangSmith, the [LangChain callback handler](/ingestion/langchain) is the path for you. For provider clients with no tracer, see the [Node.js SDK](/ingestion/node-sdk) or [Direct API](/ingestion/direct-api).
There are two ways in:
A LangSmith Automation rule posts matching runs to calado in real time. Zero code in your runtime. Recommended.
A small script polls LangSmith's run-query API on a cursor and POSTs each batch. Use it for backfill or where Automations are not an option.
Both paths hit `POST /api/ingest` with the same payload shape. Pick one.
## Automation setup
LangSmith Automations do not fire on every run by default. Without a rule selecting the runs you care about, the webhook stays silent. This is the step customers most often miss.
In the calado dashboard, open your agent's **Settings** page and create an API key. It is prefixed with `cl_`. You paste it into LangSmith in the next step.
In LangSmith, open the project you want to mirror and go to **Automations → + New Rule**.
Set a **filter** for the runs to forward:
* **Run type** — `chain` is the typical root for agent invocations. Add `llm` and `tool` if you also want sub-runs delivered. calado rebuilds the tree either way, as long as runs share a `trace_id`.
* **Project** — restrict to one or more projects.
* **Tags** — narrow to a subset, for example `production` or `agent`.
* **Sampling** — leave at 100% for full coverage; reduce it for high-volume projects.
Set the rule's action to **Webhook**:
* **URL** — `https://app.calado.ai/api/ingest`
* **Method** — `POST`
* **Headers** — `Authorization: Bearer cl_` and `Content-Type: application/json`
* **Body template** — wrap LangSmith's run payload in the calado envelope:
```json theme={null}
{
"format": "langsmith",
"conversations": {{ runs }}
}
```
Run your agent once. The LangSmith Automation log should show a `202`, and the conversation appears in calado within a few seconds.
## Verifying it works
In calado, open **Agents → (your agent) → Conversations**. A forwarded trace shows up as a conversation, with the LangSmith project name, tags, and any attached feedback visible in its metadata.
Raw conversations appear immediately. Analysis does not start automatically — click **Run Analysis** on the agent page when you are ready to classify.
## How the flat run list becomes a conversation
LangSmith's API returns runs as a **flat list**, not a nested tree. One webhook delivery can carry runs from several traces. The adapter:
1. Groups runs by `trace_id`. Each group becomes one conversation.
2. Finds the root of each group (the run with no parent) and rebuilds the `{root, children}` tree.
3. Maps `llm` and `chat_model` runs to messages, `tool` runs to tool calls, and `retriever` runs to retrievals.
4. Emits one conversation per `trace_id`, with each child run as a step.
A run that arrives before its parent (a webhook firing mid-trace) is passed through with a warning; calado links it once the parent arrives in a later delivery. A trace with no LLM activity is stored but marked as skipped for classification.
## What gets captured
| calado field | LangSmith source |
| ----------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------- |
| `conversation.externalId` and `sessionId` | `trace_id` |
| `conversation.messages` | `llm` and `chat_model` run inputs and outputs |
| Step name and hierarchy | `Run.name`, `id`, `parent_run_id` |
| System prompt | `serialized.kwargs.messages` |
| Tool schemas | `extra.invocation_params.tools` and `serialized.kwargs.tools` |
| Per-step inline definition | A child run's system-role input message, attached as that step's `inlineDefinition` so the sub-agent is analyzed against its own prompt |
| Tool calls | `tool` runs |
| Retrievals | `retriever` runs |
| Errors | Run `error` |
| Latency | Computed from `start_time` and `end_time` |
| Project name | `metadata.langsmithProject` |
| User feedback | `metadata.langsmithFeedback` |
| Tags | `metadata.tags` |
## Forwarder script
Use the forwarder for backfills, or where the Automation route is not viable. It polls LangSmith's `/api/v1/runs/query` endpoint on a cursor and POSTs each batch to calado.
```bash theme={null}
export LANGSMITH_API_KEY=ls__...
export LANGSMITH_PROJECT="your-project-name"
export CALADO_API_KEY=cl_...
export CALADO_INGEST_URL="https://app.calado.ai/api/ingest"
npx tsx forward.ts
```
The script writes its cursor to `.langsmith-cursor` and resumes from there on re-run. Delete the file to backfill from the beginning. The full script is in the [`examples/langsmith-forwarder/`](https://github.com/calado-ai/calado/tree/main/examples/langsmith-forwarder) directory of the calado repo, ready to copy.
## Caps
| Limit | Value |
| ------------------------- | ----- |
| Request body size | 5 MB |
| Runs per `trace_id` | 950 |
| Conversations per request | 1,000 |
| Message content | 1 MB |
The runs cap is applied per `trace_id` group. If one trace in a multi-trace delivery exceeds it, calado returns a structured `400 too_many_child_runs` naming that trace. The practical limit is usually the 5 MB body cap. If you hit either, narrow the Automation filter or split long-running work into shorter sessions.
## Security
The webhook setup puts your calado API key into the LangSmith Automations UI, as a header value. LangSmith team members with access to that project's Automation settings can read it. That is a slightly wider exposure surface than the direct-SDK path, where the key only ever lives in your runtime environment.
Treat the webhook key like any credential stored in a third-party tool. Create a dedicated calado key for the LangSmith Automation so you can rotate just that one, and rotate it whenever someone with LangSmith Automations access leaves your team. calado keys are one key per agent, so a leaked key can only POST to that single agent's stream.
Webhook-scoped sub-keys, where a leaked webhook key cannot be used for anything else, are on the calado roadmap. Until then, the rotation discipline above is the mitigation.
## Troubleshooting
**The webhook returns `400 too_many_child_runs`.** One `trace_id` group exceeded the per-trace runs cap. Narrow the Automation filter or split the trace into shorter sessions.
**The webhook returns `401`.** The bearer token was rejected. Re-paste the key into the Automation header. The usual cause is a stray space or a missing `Bearer ` prefix.
**No webhook activity at all.** The Automation rule has no matching runs, or no rule exists. LangSmith Automations do not fire by default. See [Automation setup](#automation-setup), step 2.
Strip PII before runs leave your infrastructure.