Skip to main content
Once the server SDK tags your LLM calls, FirstFlow has the raw material to explain what your agent is actually doing: how many conversations ran, what they cost, how each turn unfolded, whether the user got what they came for, and what they were asking about. The dashboard assembles four surfaces from that data a metrics overview, a session browser, per-session quality scores, and automatic topic/intent/sentiment classification. This page describes each one and, crucially, where its numbers come from, so you can reason about why a value is missing or stale.
Everything here is built from data the server SDK sends. No instrumentation, no analytics. The single thing that joins a transcript to its traces, cost, and scores is the shared id conversationId in the browser and sessionId on the server. Keep them equal or the rows will not line up. See Conversations & sessions.

​
How it works

There are two distinct data paths, and knowing which one feeds a number tells you how fresh it is. Live traces arrive the moment you make an LLM call: when you wrap your client or call observe() or trace(), the turn model, tokens, latency, cost, errors — is recorded against the conversation and rolled into the metrics overview and the trace waterfall almost immediately. Derived analysis quality scores, topics, intent, sentiment does not run per turn. It runs in scheduled batch jobs a few minutes after a conversation goes quiet, because grading a half-finished chat is meaningless. So a transcript appears at once, but its score chips fill in on a short delay. The diagram below traces a single conversation from the first LLM call to a graded, classified session. The richer your instrumentation, the more these surfaces can show. A bare observe() call with only role and content still produces a transcript and a quality score; add model, token counts, latencyMs, and cost fields and the metrics overview and trace waterfall come alive. The batch jobs run on the AI provider you configure (Anthropic for the LLM-as-judge scoring; your chosen provider for classification), so AI features degrade gracefully if no provider key is set, transcripts and live metrics still work and only the derived chips stay empty.

​
Conversation analytics

The metrics overview answers “how is this agent doing in aggregate?” over a date range. It reads from the live trace path, so the numbers move as calls land. The core fields it reports for the selected window: On top of the raw totals it derives session-level metrics product teams care about: average session depth (messages per conversation), cost per session, resolution rate (share of sessions with outcome: "completed" among those that carry an explicit outcome), a short/medium/deep depth distribution, and a sentiment roll-up (positive/neutral/negative counts plus a net score from -1 to +1). Resolution rate only has a denominator if you actually call outcome() without outcome signals, the rate reads as no data rather than zero.

​
Sessions

The sessions browser is the per-conversation view: the full chat transcript on one side and the LLM trace waterfall on the other each generation with its model, token counts, latency, and cost, and nested spans for multi-step operations like retrieval-then-synthesis. This is where the shared id earns its keep: the transcript, the traces, the outcome and intent signals, and the quality chips all hang off the same sessionId. Add your own tags and notes for human review. Two practical consequences of the data model:
  • A transcript shows turns from every path that wrote to the conversation — wrapped client calls, manual observe(), and trace() spans. Observe user turns (role: "user"), not just assistant turns, or the transcript is one-sided and the classifier loses context.
  • Survey and form responses are analytics events, not a transactional store. A submitted survey appears as a survey_completed analytics event, not as an editable record there is no response database to query.

​
Session KPIs

Session KPIs are quality scores, one number per dimension per conversation, on a 0–100 scale. They are produced by an LLM-as-judge pass: a scheduled job picks up conversations that ended more than five minutes ago and have at least two messages, builds a transcript from the last ten user/assistant turns, and asks the model to score each active dimension against its rubric, returning strict JSON. The job runs roughly every fifteen minutes, which is why a freshly ended chat shows no scores for a few minutes. Every agent starts with five seeded dimensions. Note the direction for two of them, a lower score is the good outcome. You can edit these and add your own. A custom evaluator is a name plus a rubric the rubric text is fed directly into the judge prompt and defines what 0 and 100 mean for that dimension, so write it concretely. A custom evaluator looks like this:
Archiving a dimension stops it being scored on future conversations but keeps the history on existing ones. Scoring needs the configured AI provider; if none is set, the pass skips and the chips stay empty rather than failing the conversation.

​
Topics, intent & sentiment

A second batch pass classifies each conversation along three axes what it was about (topic), what the user was trying to do (intent), and how they felt (sentiment) against a taxonomy you curate per agent. Like the KPI pass, it runs on a fifteen-minute schedule a few minutes after a conversation ends, so the chips fill in shortly after the transcript appears. Each agent is seeded so day-one classification works without setup:
  • 13 default topics onboarding, billing, error-recovery, feature-request, how-to, troubleshooting, data-question, account-settings, feedback, navigation, integration, performance, and a reserved other fallback bucket.
  • 8 default intents get-help, complete-task, explore-feature, report-issue, request-feature, provide-feedback, get-information, and the reserved other.
  • 3 default sentiments positive, neutral, negative. Custom sentiment labels (e.g. frustrated) carry a polarity so they still roll up into the positive/neutral/negative donut and the net sentiment score.
The other bucket is reserved in every taxonomy: it cannot be deleted or renamed, and it catches anything the classifier can’t place. When classification confidence is low (below 0.6) or lands repeatedly in other, FirstFlow records the recurring theme as a suggestion a proposed new topic, intent, or sentiment label you can promote into the active list or dismiss. Suggestions only surface once a theme recurs across at least three distinct conversations, and obvious noise (greetings, thanks, test/health-check traffic) is filtered out so the list stays signal-dense. Promoting a suggestion turns it into a regular label; archiving a label is a soft-delete that preserves the classifications already attached to old conversations.
Topics and intents feed the breakdowns in the metrics overview; sentiment feeds the donut and the net score. Curate the taxonomy on the agent’s settings so the labels match your product’s vocabulary rather than the generic defaults.

​
Get the data flowing

The fastest way to make every surface on this page light up is to send richer turns. The minimum to produce a transcript and quality scores is a tagged turn with its text; adding the LLM metadata fills in the metrics overview and the trace waterfall.
If you wrap your LLM client instead of calling observe() by hand, traces and token/cost fields are captured automatically on every chat.completions.create, messages.create, responses.create, and embeddings.create call see Wrap your LLM client. Remember the one rule: a turn needs all three of firstflowAgentId, sessionId, and userId, or it passes through unobserved and never reaches any of these surfaces.

​
Next

Start by wrapping your LLM client or calling observe() to feed the transcript and metrics, then add outcome() signals to populate resolution rate; the server SDK overview ties the helpers together and conversations & sessions explains the shared id that joins everything.