Why TTFT is a UX metric, not an infra metric

There's a meeting that happens on most AI product teams. The infra engineer pulls up a dashboard. Numbers. Percentiles. A latency chart with a line somewhere around 800ms. The PM looks at it, nods, and says something like "looks good." Then everyone moves on.

Nobody asked what good actually means.

TTFT — Time to First Token — is the gap between a user hitting generate and seeing the first word appear on screen. Engineers own that number. It lives in their dashboards, comes up in their retros, gets optimised in their sprints. The PM hears about it occasionally and mostly defers.

That's the problem.

TTFT is not an infrastructure metric. It's a UX metric wearing infrastructure clothes. And the difference matters because the person who should be setting the target, the person who actually understands what the user is doing in those first few seconds, keeps leaving the room before the conversation starts.

Users don't experience throughput. They don't feel p99 latency. What they feel is that moment right after they submit something. The silence. The cursor that hasn't moved yet. The uncertainty about whether anything is happening at all.

That's TTFT. And it's yours to own.

There's decades of HCI research on this. Under 100ms feels instant. Under a second feels responsive. Above a second and people start to disengage mentally. Above three seconds they start questioning whether the product is broken.

AI products live in that window. Most of the time, TTFT is what determines which side of it you're on. But here's the thing most PMs miss: the right number isn't universal. It changes completely depending on what the user is doing.

Same model. Same infra. Three completely different answers.

The writing assistant

A user is mid-sentence. They hit a shortcut and the AI continues their thought. They're not thinking about a model processing their request. They're thinking about their next sentence.

The expectation here is instant. Not fast. Instant.

Good TTFT for this feature is under 400ms. That's the threshold where the tool feels responsive rather than processing. Cross 800ms and the user's flow breaks. Cross 1.5 seconds and they've already started typing themselves. The product has failed silently.

How you measure it matters too. Not first token generated server-side. First character rendered on screen. That delta; network round trip, rendering pipeline, streaming latency; is real. Don't let your infra team measure from a different starting line than your users experience it.

The research summariser

Different scene. A user uploads a 40-page PDF. They hit submit. They know it takes a moment. They might switch tabs.

The implicit contract here isn't instant. It's "I'll wait, just don't waste my time."

Good TTFT for this feature is under 3 seconds. But honestly, TTFT is almost the wrong metric entirely. What this user cares about is total completion time and output quality. A fast first token on a mediocre summary is worse than a 2-second wait for a sharp one.

This is one of the few AI features where TTFT is the weaker signal. The metric that matters here is TTFR, time to full response. If your team is optimising TTFT on a batch feature, they're optimising for the wrong thing. That's a product call. Not theirs to make without you.

The support agent

A customer types a complaint into a chat widget. They're already frustrated. They expect a reply.

The contract here is closer to messaging a human than running a query. Every second of silence doesn't read as slowness. It reads as inattention.

Good TTFT here is under 600ms. But there's a layer below even that. You want a typing indicator firing before the first token, something that tells the user the message landed and the system is working. That perception management layer is as important as the actual latency number.

Instrument both separately. Actual TTFT and perceived response start. If your typing indicator fires at 100ms but the first token takes 2 seconds, users still feel the gap. The indicator buys you time. It doesn't close the gap.

The pattern across all three is the same.

The more synchronous the interaction, the more the user is in a flow state, mid-thought, waiting on the product to keep up, the tighter the TTFT window gets. The more asynchronous it is, the more TTFT gives way to other metrics.

The decision framework is simple once you see it. Plot your feature: how synchronous is the interaction, and how low is the user's tolerance for waiting? Features in the synchronous, low-tolerance quadrant need the tightest targets and the closest monitoring. Features in the async, high-tolerance quadrant give you room and shift the priority metric entirely.

Before you walk into the next review, know where your feature sits on that grid. Know what the threshold is. Know how you're measuring it and from where.

Because if you don't set the target, someone else will. And they'll set it based on what's achievable, not what the experience actually needs.

That's how you end up with a fast model that feels slow.

Users won't tell you. They'll just leave.