PostHog builds product analytics, and its 10 AI products reach nearly 100,000 people a month. In October 2026 it published 38 lessons from building them.
Every lesson gets the same format and weight, and PostHog’s own last lesson says to expect half of them to be wrong in a year. We sorted all 38 so you can see which ones to build on and which ones to revisit when the next model ships.
Sponsored by HubSpot for Startups
PostHog’s lessons get an AI product shipped. The harder part comes after launch, when you need to turn AI pilots into customers who stay.
IGNITION 2026 by HubSpot for Startups: the summit for AI-native founders, Nov 10 in San Francisco
Built for AI-native teams: a curated room of 200, mostly Seed and Series A founders and GTM leaders
Practical by design: mainstage talks from founders and investors, plus 1:1 clinics with GTM experts
It’s free for approved applicants, and applying through this link gets you priority review:
The Short Answer
Lessons that last: test every AI bet like a new product, and make agents reliable and transparent before adding capability.
Lessons tied to today’s models: harness design, the ban on subagents and how context gets delivered. Expect these to change with the next model.
Lessons that depend on your users: PostHog’s agent-first bet on an MCP server fits developers. Teams serving non-technical users should build the agent inside their app.
The first question to ask: how does your AI feature beat a user opening Claude or ChatGPT?
Table of Contents
Decide Whether to Build an AI Feature at All
Choose Where the AI Lives: In Your App or Through an MCP Server
Build the Agent: Harness, Context, Cost
Earn Trust: Reliability, Showing the Work, Review Screens
Validate and Protect the Bet
FAQ
1. Decide Whether to Build an AI Feature at All
The first decision could save you from building anything at all.
Which jobs are worth handing to AI
Lesson 1’s point is you may not always need to add an AI feature in your product, especially when it could make your product worse by getting slower or unreliable.
For example, a document chat feature, which every AI tool can already do, isn’t worth adding. Not to mention, your users may already have their own AI agent or model that they trust.
If you’re not sure how it applies, lesson 3 gives 3 signs that a job is worth passing to AI (you only need 1):
Takes more than 30 seconds
Needs a language users don’t know (like SQL)
Users repeat the task at least 20 times
A good example of this is onboarding (lesson 4), like PostHog’s install wizard that they claim can do 2 hours’ worth of work in 8 minutes.
According to PostHog, users who make use of the install wizard sent their first event sooner, about 1.9 hours compared to 3.8 hours for those who don’t.
Wizard users are also more likely to pay: 14.2% of wizard users convert to paid, compared with 2.6% of others.
The catch here is that PostHog has no control test, or at least, doesn’t describe one.
Those using the wizard may already be familiar with the command line terminal. They may already be more technical and may have paid anyway, which can explain part of the gap.
How does this beat Claude?
Before you build anything, check whether users could just use a chatbot for the same job.
Lesson 10 indicates Claude as the default go-to, and so according to PostHog, you must answer the question “How does this beat Claude?”
For PostHog, the answer is context: the product’s features, users’ data, the tools, and even the skills that PostHog has written.
What to do:
Check the AI feature against the 3 signs in lesson 3.
If it hits none of the signs, don’t build it.
Note what your AI feature has that a general chatbot like Claude or ChatGPT doesn’t have (like your users’ data or your own tools).
2. Choose Where the AI Lives: In Your App or Through an MCP Server
Let’s say your feature passes. Now you have to decide where users will actually reach it.
Why PostHog now says to build for agents first
Lesson 2 says to consider whether you need to build your own AI feature inside your app. Making it easy for other agents to use your product might be more worth it.
The standard method for AI tools like Claude Code and Codex to plug into a product and use it directly is through an MCP server.
PostHog ended up building both:
PostHog AI (its own agent)
The MCP server (letting other agents use PostHog)
According to their October 2026 lessons list, there are 5x more people who use the MCP server on a daily basis compared to PostHog AI.
PostHog studied 63 million MCP tool calls (90 days to September 16, 2026) and found out the following:
About 7% of calls came from a chat window (third-party traffic)
Running SQL and reading the data schema made up 32% of all calls
In 30 days, agents created 252,000 charts using MCP, while people created 99,000 in the web app
The chart numbers might seem surprising, but they make sense since agents can make charts in seconds. The real question is how many of those 252,000 charts were actually used?
The SQL number also explains lesson 20, which says AI agents work better if you let them write their own SQL instead of forcing them to use preset tools.
When that advice won’t transfer
Going with MCP worked well for PostHog because its users are mostly software teams, people who are already working inside AI coding tools.
In PostHog’s March post, it said that a custom in-app agent is the better choice in 3 cases:
Your users aren’t engineers.
Compliance blocks outside agents.
You need full control of the experience.
For example, let’s say your users are salon owners.
They don’t use AI coding tools like Claude Code or Cursor, so building an MCP server for them would be pointless.
What to do:
Find out if your users are already using AI coding tools.
If yes, consider building an MCP server first, which is easier to build and maintain anyway (lesson 2).
If not, or compliance or control is an issue, then go with the in-app route.
Sponsored by HubSpot for Startups
Shipping the AI product is half the job. At IGNITION 2026 on Nov 10 in San Francisco, Seed and Series A founders learn how to turn it into a repeatable go-to-market motion. It’s free for approved applicants, and applying through this link gets you priority review.
3. Build the Agent: Harness, Context, Cost
PostHog admitted that half of the lessons in the list may become obsolete in a year. The lessons in this step are the ones most likely to change.
Why PostHog went through 3 harnesses
A harness is the software that connects a model to your product’s tools and data.
Lesson 11 details how PostHog AI went through 3 harnesses in a year, as described in their March 2026 post:
A coordinator that routed work to sub-agents
One agent loop with 44 tools written by hand
Anthropic’s Claude Agent SDK, plus PostHog’s own tools and skills
New models were one reason for the rebuilds (lesson 15). Cheap reasoning (OpenAI’s o4-mini) and reliable tool use (Anthropic’s Claude 4) made the old setup outdated.
PostHog’s advice is to not treat a harness as an advantage. I wrote an earlier post where I argued that the harness decides whether an agent works at all.
The harness affects quality (my point), but it won’t set you apart from the rest, which is PostHog’s point. Better to spend your effort on context.
Why more context beat better reasoning
Lesson 27 says that context failures are behind many agent failures.
A good example of this is PostHog’s install wizard. With 20+ PostHog products, 17+ SDKs, and 25+ frameworks, the possible setups are in the thousands, and the agent kept choosing wrong.
PostHog then added docs, source code, SDK reference, and example apps. This helped more than the reasoning and planning changes it had tried.
Lesson 28 says some context has to come from people. PostHog AI’s /init command researches your product and asks you to confirm the summary before saving it.
But even if the idea of giving the agent more context is correct, PostHog changed how it delivers that context. PostHog AI’s system prompt included every event and property in a user’s data.
PostHog replaced that list with a tool that the agent can call when it needs to look up an event or property (lesson 29).
Lesson 30 says that when the product changes faster than the docs, your agents will start working from outdated information.
PostHog now has agents that look at merged pull requests and open a doc pull request when needed to keep the docs current.
What a run costs, and why PostHog banned subagents
Lesson 25 says to learn your unit costs.
One wizard run cost PostHog an average of about $6.67 in February 2026. For PostHog, the better activation and retention of its own users are worth more than that.
PostHog’s February 2026 post has a section called “The subagent ban”.
A subagent is a helper agent to the main agent. It doesn’t share the main agent’s cache, so the subagent has to read everything again, adding to the cost.
They didn’t always finish either. PostHog shared an extreme case where the wizard spawned 5 subagents, then idled for 5 minutes waiting for the subagents to return.
2 of the subagents didn’t return, leading the wizard to run a kill command. That whole process took about 15 minutes, and it was all a waste. The real work then took about 3 minutes.
Lesson 26 can’t put it more simply: subagents cost more than you think. Across wizard runs, cost generally rises with the number of turns, from about $2 to about $11.
The same February 2026 post noted that the newer Sonnet 4.6 at that time seemed to handle subagents better. This lesson may not survive the next model.
I’m not telling you to be skeptical, but it helps if you remember what PostHog is selling.
For example, lessons 22 (AI observability) and 24 (tracking agent usage) match PostHog’s AI Observability and MCP Analytics products.
What to do:
Expect to rebuild your harness, and don’t count on it as your advantage.
When the agent fails, check its context first before working on the reasoning.
Calculate your cost per run, and test the subagents again on current models.
4. Earn Trust: Reliability, Showing the Work, Review Screens
A working agent doesn’t mean people will rely on it.
Lesson 23 answers the question: capability or reliability? PostHog listed these early complaints about its AI:
Inconsistent performance
Unexpected failures
Generic error messages
Unclear limits
No sign of uncertainty
Users weren’t asking for more features. As PostHog puts it, “capabilities are worthless if they aren’t reliable.”
Lesson 36 contains an example of when PostHog reversed its decision to hide the agent’s reasoning and failed tool calls.
PostHog’s reason was that it looked messy. But according to the users, that only made it hard for them to trust the AI. PostHog now streams everything to users.
Lessons 19 and 37 are about UI and how its role has changed now that AI agents do the work. Basically, more of the screens now are for approving, reviewing, and undoing the agent’s work.
What to do:
Go through the complaint list first, starting with generic errors and unclear limits.
Show users the agent’s reasoning and its failed tool calls.
Add a place to approve, review, and undo what the agents did.
5. Validate and Protect the Bet
Validating an AI feature starts before launch and continues after it.
How to judge the bet, and who could beat it
Lesson 8 talks about how to validate AI bets by treating them like a new product.
Before you build, decide what it’ll be judged on and how long it gets (its runway).
Once it exists, ask your team how disappointed they’d be to lose it.
PostHog asked its own team right before launching a product. Only 23% said they’d be very disappointed, which was way lower than the 40% benchmark Sean Ellis set for product-market fit.
Lesson 9 is about writing down how an AI-native competitor would beat you, then funding that yourself.
PostHog’s example is Intercom, which it says “probably” pivoted so hard into Fin, its AI support agent, for this reason.
In Intercom’s case, an AI that answers the tickets means fewer human agents are needed. Fewer human agents mean fewer Intercom seats, and Intercom charges per seat.
How to protect it from prompt injection
Lesson 33 talks about prompt injection. It’s when someone hides instructions in text that an AI agent will read in hopes that the agent will obey as if the prompt came from you.
PostHog has a feature called self-driving which opens pull requests from product signals. Some of those signals might come from the public.
PostHog’s defense is an LLM classifier at the top of the pipeline, a separate model that drops signals that look malicious.
OWASP ranks prompt injection first in its 2025 and 2026 LLM Top 10 lists. It also says that mitigation doesn’t remove prompt injection entirely.
PostHog’s filter alone can’t solve this, and self-driving can open PRs without a human approving them.
What to do:
Write the criteria and runway down before building anything, then run the survey on your team.
Write down how an AI-native competitor would beat you, then fund your own version.
Always treat public inputs as untrusted, especially if an agent can act without approval.
Anyway, if I had to pick 1 lesson to start with, I’d implement lesson 10 and ask how the AI feature could beat Claude. If it can’t, then you’re just building a worse version.
FAQ
What are PostHog’s 38 AI product lessons?
According to PostHog, they are lessons from building 10 AI products used by nearly 100,000 people a month, published in October 2026. They cover when to build AI features, agent design, cost, context, MCP servers and security.
Which AI product lessons will still hold next year?
The product lessons hold: validate each AI bet with written criteria and a runway, and favor reliability and visible reasoning before new capabilities.
Why did PostHog stop using subagents?
Subagents cost more than expected, since each one keeps a separate cache and rereads everything, and some went silent, which triggered retries. PostHog’s install wizard cost about $6.67 per run.
Should you build an MCP server or an in-app AI agent?
Build an MCP server when your users are developers who work in tools like Claude Code. Build an in-app agent when your users are non-engineers, when compliance blocks outside agents or when you need full control of the experience.
How do you make an AI agent reliable?
Give it rich, current context and show users its reasoning and tool calls. PostHog also treats the harness as replaceable as models improve.
Keep reading:
Your AI Agent Might Be Paying $11,000 a Month to Answer Yes or No
Google’s agent broke a 56-year math record. Yours forgets yesterday
If this helped, send it to the founder or PM on your team who is shipping AI features.
Been thinking about that half of this expires in a year thing all morning. Like we rebuild stuff, then a new model drops and half the setup is just... pointless now? And yeah clients don't care how clever it is, they care if it works Tuesday and then again Wednesday. Reliability, reliability, reliability.