Why your agent's memory is a billing problem
There is an interesting question I tackle most times when I am thinking through agent memory. Is it a capability question, a product question, a design question, or an experience question? The more I
Leverage Points - Product & Tech
Latest articles
MoE vs. dense models: What you need to know before picking a model
Saw a comment online about Models getting dumber or Model Labs trying to just pass benchmark as a tradeoff for losing some capabilities. This brings back memory of model selection in early days where
Small model, big model: routing as a product strategy
When building AI products, the fastest and laziest way is to pick a model the way you typically pick a cloud provider. Evaluate a few options, choose one, and move on. One model, One API and everythin
Prompt design is a cost centre. Treat it like one
I remember when prompt design was one of the hottest topics in getting value out of LLM For teams building AI features, it sits in the engineering backlog, or the AI team's to-do list, or nowhere at a
Continuous batching explained for PMs who ship AI features
Engineers building AI/ML should understand continuous batching intuitively. Some PMs have never heard of it or don't understand it. That gap costs product teams real money and slower user experiences,
Why output tokens cost more than input tokens and what that means for feature design
There's a pricing detail most product teams don't look at closely enough. Input tokens and output tokens are not the same price. Output tokens cost more. Sometimes 2-3x more depending on the model. Th
Context windows are not free : What your PM never told your eng team
At first glance, you think there is no price tag for Nobody context windows. It's not listed in the model pricing. It doesn't show up in your planning. Your engineering team probably hasn't flagged it