Agent Analytics measures your AI agent’s conversations inside your product: who uses the agent, what they ask, how satisfied they are, and where the agent fails. You send the conversation as it happens; Userpilot automatically derives topics, frustration signals, failure signals, and slow-response flags — you never classify anything yourself.

Everything ties to the same user identity as the rest of your Userpilot analytics, so conversations show up on user profiles and in Segments, Reports, Sessions, and session replays.

Before you start

You need two things you almost certainly already have:

The Userpilot SDK snippet installed on your pages.
A call to userpilot.identify(userId, ...) — agent events attach to this user. Events fired before identify are queued locally and sent right after it.

Step 1 — Turn the module on

The agent module is off by default. Enable it in your Userpilot settings object:

window.userpilotSettings = {
  token: "YOUR_APP_TOKEN",
  agent: {
    enabled: true        // master switch for agent analytics
  }
};

Step 2 — Report messages as they happen

There is no “start conversation” call. The first message you send with a new conversationId creates the conversation automatically — just use your own chat-session id. When the user sends a prompt:

userpilot.agent.trackMessage({
  conversationId: "conv-8412",   // your chat/session id (required)
  agentId: "support-bot",        // stable agent id (required on every message)
  role: "user",                  // "user" or "agent" (required)
  text: "How do I export my data?",

  // Send these on the FIRST message of each conversation:
  agentName: "Support Bot",
  trigger: "Widget clicked",     // what opened the chat
  platform: "desktop"
});

When the agent replies:

userpilot.agent.trackMessage({
  conversationId: "conv-8412",
  agentId: "support-bot",
  role: "agent",
  text: "You can export from Settings → Data…",
  responseTimeMs: 2400,          // optional: powers slow-response detection
  model: "gpt-4o"                // optional
});

Optional per-message extras: messageId (your own message id — recommended), inputTokens, outputTokens, metadata (your own key/values), ts (epoch ms).

Page, browser, device, and session context are attached automatically — never send those. Topic, frustration, and failure classification is done by Userpilot — never send those either.

Step 3 — Report thumbs up / down

userpilot.agent.trackFeedback({
  conversationId: "conv-8412",
  value: "positive",                 // "positive" or "negative" (required)
  ratedMessageId: "msg-3",           // which reply was rated (optional)
  reason: "Solved it immediately"    // optional free text
});

That is the whole client API: two calls. No start, no end — duration and prompt counts are computed from the messages themselves.

Optional — Send from your backend instead (or as well)

Tokens, model names, and full replies often only exist where inference runs. Your backend can send the same events over HTTP to POST /v1/agent/events with these headers:

Authorization: Token YOUR_WRITE_TOKEN (your account write token — same as /v1/track)
Content-Type: application/json

Body — a single item or a batch of up to 100:

{
  "events": [
    {
      "act": "message",
      "user_id": "u-123",
      "conversation_id": "conv-8412",
      "message_id": "msg-3",
      "role": "agent",
      "agent_id": "support-bot",
      "text": "You can export from Settings…",
      "model": "gpt-4o",
      "input_tokens": 512,
      "output_tokens": 128,
      "response_time_ms": 2400
    }
  ]
}

Rules of thumb:

user_id must be the same id you pass to userpilot.identify — that is what joins conversations to profiles and segments. Anonymous users are client-transport-only.
Invalid items are rejected with a 422 and per-item errors; valid requests return 202 and are processed asynchronously.
Optionally pass "context": { "pathname", "hostname", "session_id" } — forward the SDK session id if you want session-replay links on server-sent conversations.

Recommended setup (hybrid)

your backend sends all messageevents (it owns text, tokens, model, latency); the browser SDK sends feedback(it owns identity, page, and session context). If both sides ever report the same message, use the same message_idfrom both and let only one side send text— reports merge into one message and nothing double-counts. Retries are always safe.

Privacy controls

Client-side: agent.captureText: false stops all free text from ever leaving the browser; agent.redaction: (text, field) => string lets you scrub text before sending.
Admin-side (in Userpilot, no code): an “Enable capture of conversation content” toggle per environment, plus mask patterns (matched spans stored as [REDACTED]) and excluded keywords (whole message stored as a masked placeholder). These are enforced at ingestion for every producer — SDK and backend alike.
If text is not captured, usage and feedback metrics still work, but AI-derived topics and frustration/failure signals will be unavailable.

What you’ll see in Userpilot

Where	What
Agent Analytics dashboard	Users, conversations, prompts, satisfaction, frustrations, retention, entry points
Topics / Failure signals tabs	Conversations auto-grouped by AI (you can add your own topics)
User profile → Conversations	Every conversation with full transcript
Segments / Reports / Sessions / Engagement	The Conversation started event with agent, trigger, topic, feedback, and more as filters

Launch checklist

identify () runs before or alongside agent usage
agent.enabled: true in window.userpilotSettings
trackMessage wired to both user prompts and agent replies, with conversationId + agentId on every call
trackFeedback wired to your thumbs up / down
(Hybrid) backend posts messages with the same user_id and shared message_id
Privacy: decide text capture per environment; add mask patterns for things like card numbers

Good to know

Conversations that were opened but never got a message are not tracked, and the SDK caps agent events at 5,000 per page session as a safety valve. Full field-level details live in the contract spec: docs/agent-telemetry-contract.md (v0) in the SDK.

Installation

Security

Data sync

Advanced

Agent SDK Installation

Before you start

Step 1 — Turn the module on

Step 2 — Report messages as they happen

Step 3 — Report thumbs up / down

Optional — Send from your backend instead (or as well)

Recommended setup (hybrid)

Privacy controls

What you’ll see in Userpilot

Launch checklist

Good to know

​Before you start

​Step 1 — Turn the module on

​Step 2 — Report messages as they happen

​Step 3 — Report thumbs up / down

​Optional — Send from your backend instead (or as well)

Recommended setup (hybrid)

​Privacy controls

​What you’ll see in Userpilot

​Launch checklist

Good to know

Before you start

Step 1 — Turn the module on

Step 2 — Report messages as they happen

Step 3 — Report thumbs up / down

Optional — Send from your backend instead (or as well)

Privacy controls

What you’ll see in Userpilot

Launch checklist