In Langfuse, a failed step is usually an observation with level ERROR and a statusMessage. Start there: filter observations by level and time, open the trace, and read the failing observation's input and output. Then group the failures by observation name and error so you fix patterns, not single runs.
Many agent failures are never marked ERROR, though: empty tool results, loops, ignored tool output. For those you need checks on the observations themselves, written as Langfuse evaluators or done by a failure-analysis tool that reads the traces.
A travel-booking agent traced with Langfuse runs about 2,000 times a day. It has tools search_flights, hold_seat and book. Support tickets mention "the bot keeps saying it booked but there's no booking". The team wants to know how many runs are affected and why.
Step by step in Langfuse
- Bound the time window. Pick the period the tickets cover. Every query, in the UI or the API, is faster and easier to reason about with a fixed window.
- Filter observations by level. In the observations table, filter level to
ERROR(andWARNINGif your code uses it). Sort or group by observation name: if one tool dominates, start there. - Open a few traces and read the failing step. Look at the failing observation's input, output and status message, then at what the agent did next. In the synthetic case,
bookfails with409 seat no longer heldand the next generation tells the user the flight is booked. - Look at the run as a whole. The timeline view shows slow steps; the agent graph view collapses repeated steps, which makes loops easy to see.
- Group repeats. Count failing runs per (failure type, feature, tool). One group with hundreds of runs is a bug; a group of one is probably noise.
- Write down the check, then automate it. Once you know what "failed" means for this agent, encode it: a Langfuse score or evaluator, a regression test, or a failure-analysis tool.
Pull failed observations via the API
For anything beyond a few traces, the Observations API v2 is quicker than clicking. It returns observation rows (not whole traces), newest first, with cursor pagination, and lets you choose field groups. This request returns failed observations from one day with their inputs and outputs:
curl -G "https://cloud.langfuse.com/api/public/v2/observations" \ -u "$LANGFUSE_PUBLIC_KEY:$LANGFUSE_SECRET_KEY" \ --data-urlencode "fromStartTime=2026-11-09T00:00:00Z" \ --data-urlencode "toStartTime=2026-11-10T00:00:00Z" \ --data-urlencode "level=ERROR" \ --data-urlencode "fields=core,basic,io" \ --data-urlencode "limit=1000"
Then group the rows. A short script is enough to see which tool and error dominate:
import refrom collections import Counterdef key(o): msg = re.sub(r"\d+", "N", (o.get("statusMessage") or "")[:80]) # 409 seat 4C -> N seat NC return (o.get("name"), msg)groups = Counter(key(o) for o in rows) # rows = all pages of "data"for (name, msg), n in groups.most_common(10): print(f"{n:5} {name:20} {msg}")
Use a key with read-only access for scripts like this: when you create a project API key, Langfuse lets you choose Viewer permissions. Observations API v2 is available on Langfuse Cloud and self-hosted Langfuse v4; self-hosted v3 uses the older v1 endpoint.
Failures Langfuse doesn't mark as errors
Langfuse records what your code tells it. A level filter finds only the steps your code or SDK marked as failed. These are usually missed:
- Empty tool results (
[],{"results": []}) that the agent answers anyway. - Malformed tool output: an HTML error page, a stack trace, or cut-off JSON returned as a normal string.
- Loops: the same tool called again and again with the same arguments, each call "successful".
- Ignored tool output: the tool returned the answer and the final message doesn't use it.
Langfuse code evaluators can check observations with your own Python or TypeScript logic (JSON parseability, schema validation, tool-call checks), and LLM-as-a-judge evaluators can score answers. Both produce scores you can filter and alert on.
Common mistakes
- Only filtering by level. It finds the failures you already instrumented and misses the silent ones.
- Tools not traced as separate observations. If tool calls are hidden inside one span, there is no failing step to find.
- Unbounded API queries. Always pass
fromStartTimeandtoStartTime. - Grouping on raw error text. IDs and numbers in messages split one bug into hundreds of groups; normalize them first.
- Using a full-access key for read-only analysis. Create a Viewer key instead.
When Rootline helps
Rootline connects to Langfuse with a project API key pair and only reads: it calls the Observations API v2, imports the last 30 days, then fetches only new observations on each sync. Every run is checked for seven failure types: errors, timeouts, agent loops, malformed tool output, empty tool output, tool output ignored, and high-cost runs. Each failed run gets the failing step highlighted and a plain-English explanation with a suggested fix, and repeats are grouped into issues by failure type, feature and tool. On the Team plan, each issue can be exported as a regression test case.
Rootline reads only through Observations API v2, so self-hosted Langfuse needs v4 (and must be reachable over https on a public address). Each import reads up to 20,000 observations. Detection is rule-based on trace contents: it won't judge subjective answer quality, and it can only see what your instrumentation records.
Official documentation
- Langfuse Public API: Observations API v2langfuse.com
- Langfuse log levels and status messageslangfuse.com
- Langfuse agent graphslangfuse.com
- Langfuse code evaluatorslangfuse.com
- Read-only (Viewer) Langfuse API keyslangfuse.com