Top 10 Learnings From Building Our Own AI VP of Customer Success “Qbee”

We’ve now rolled out 21+ AI agents and 12+ vibe coded apps at SaaStr AI, used 1,100,000+ times. AI SDRs. An AI VP of Marketing (10K) that in many ways now manages us, not the other way around. AI-powered VC tools that have run over a million startup valuations. A LinkedIn attendee card generator I shipped from the back of a Waymo, and an AI Parking Pass app that distributed 5,000 custom parking passes and saved weeks of human time.

And QBee is the best thing we’ve built. By a lot.

QBee is our AI VP of Customer Success. Qbee managed all 100+ sponsors for SaaStr AI Annual 2026, all our media sponsors, and everyone coming back for 2027. She sends every one of them hyper-personalized check-ins. She tracks 13 core tasks with dozens of subtasks per customer. She follows up without drama. She identifies gaps in real time. She pushes a Slack and email update to our team every single day.

She was built on Replit, by Amelia (our Chief AI Officer), with zero engineers. Our AI token costs across all of our vibe coded apps combined haven’t hit $200/month yet.

The result: 70% fewer human hours on customer management. 10x more customer logins and on-time submissions versus our old off-the-shelf tool. Sponsors didn’t even realize most of the communication was coming from an AI until we told them on a webinar.

Here are the top 10 things we learned building her. If you’re thinking about doing this yourself, steal all of it.

1. It Didn’t Start as an AI VP of Customer Success. The Agent Emerged.

QBee was not designed to be an AI VP of Customer Success. That was never the plan.

The original goal in January was modest: replace the off-the-shelf sponsor portal we’d been paying for. That tool had zero AI. No SSO. No analytics. We couldn’t even tell if a sponsor had logged in unless they submitted something. Amelia’s initial spec was basic project management. Assign tasks, add single sign-on, build in light reminder automations.

That’s all QBee did for the first couple of weeks. And she was already better than what we had.

Then real customer data started flowing. Logins, task completions, usage patterns, gaps. Once the data existed, the agentic possibilities became obvious. Daily personalized check-ins. Triggered follow-ups. Real-time internal reporting. Proactive outreach.

The learning: don’t try to build the AI VP of Customer Success on day one. Build the dashboard or portal first. Ship it. Let the data tell you what to automate next. The agentic layer emerges from the operational layer, not the other way around.

2. Daily Beats Quarterly. The QBR Was Always a Broken Concept.

QBee is partially a riff on QBR, the Quarterly Business Review. Except she doesn’t run quarterly. She runs every day. Sometimes twice.

Think about the dated (but still way too common) QBR for a second. A customer has deliverables due in 60 days. A trigger event worth celebrating happened yesterday. A contract gap shows up in the data this morning. And the plan is to wait until the next scheduled quarterly call to surface it?

Quarterly is retrospective by definition, and in many ways, just a stealth upsell motion. Everything in a QBR has already happened. Great CS isn’t something you do in a one-hour call every 90 days. It’s something you do continuously, in the background, across every account, every day.

That’s what QBee actually does. It watches who’s engaged, who’s gone quiet, who’s about to miss a deadline, who just had a trigger event worth acting on. It runs whether anyone asks her to or not. It acts.

If you’re building AI for customer success and it still waits to be asked a question, you’re building last year’s product.

3. Hyper-Personalized Means 4 to 6 Unique Data Points Per Message. Not a Merge Tag.

The biggest mistake people make with “AI-powered” customer emails is thinking a first-name merge tag and some light context injected into the body of a email qualifies as “personalization”. It doesn’t. Customers see through it immediately. It adds no value and triggers the AI Slop meter.

Every QBee message to a sponsor reflects their specific situation and needs right now.

Which tasks are done. Which are overdue. What’s coming up next. Their unique registration links. Their booth number. Their badge allotment. Their content deadlines. Their last login date. Their speaker slot.

Four to six unique data points per message, minimum. Usually more.

No human CSM at scale was ever doing this. It’s impossible to maintain that level of detail across 100 accounts every week. But it’s trivial for an agent with clean data pipes.

The bar for “personalized” is way higher than most teams realize. And that’s why the sponsors didn’t know or at least case it was AI. They assumed it was a human because no AI tool had ever written them something that contextual before. The tell of AI used to be that it sounded generic. The tell of good AI now is that it knows more about your account than your old human CSM did. And this isn’t that high of a bar in many cases.

4. Agent Hop for Security. Minimize Sensitive Data Stored in the Agent.

QBee does not store sensitive customer data directly. We call this “agent hopping.” Sensitive data lives in the system built to secure it. The agent calls the API to assemble the picture.

QBee’s customer database lives in Salesforce. Contracts, contacts, deal details, sponsorship tier, booth assignments. All in Salesforce. User authentication lives in Clerk. Registration links come from the Bizzabo API. Email delivery runs through Resend.

QBee has no contracts sitting in her knowledge base. She hops between systems to assemble the full picture per sponsor, per message.

Amelia built a custom Salesforce Connected App to make this work. Had she done that before? No. She asked Claude and the Replit agent how to do it, got it working in about 20 minutes, and the integration has been rock solid since.

The more sensitive data you store directly in your agent, the more you have to become a security expert whether you wanted to or not. Constant audits, pen testing, break-it exercises. Most of us don’t want that job. Keep sensitive data in established systems. Make the agent hop.

5. Write the Spec in / with Claude Before You Open Replit / Lovable / etc.

Before opening Replit, Amelia wrote a spec. User flows, a dashboard, checklists, an asset library, upload functionality, single sign-on. It was maybe 60% of what QBee does today. That was fine.

If writing a spec from scratch seems intimidating, start in Claude. Just say “I need help writing a spec for a customer success portal” and iterate. You don’t need to be a prompt engineer in 2026. Those days are over. Talk to Claude, describe what you want, iterate until the spec makes sense, then give it to your vibe coding platform.

The more granular your spec, the less you’ll iterate later and the lower your token costs will be. But don’t let perfectionism stop you from shipping. Amelia’s first spec wasn’t that detailed. QBee turned out fine. The goal is to get to production, not to have a perfect spec.

You can literally give saastrsponsors.com to your vibe coding agent and say “I want something like this for my business.” That works.

6. Deploy to One Customer Per Tier First. Not All 100+ at Once.

We did not roll QBee out to all 100+ sponsors on day one. Amelia picked one customer at each sponsorship tier: Diamond, Platinum, Gold, Silver. Four test accounts.

Things broke. The Salesforce integration disconnected twice the first week. Edge cases around pending users in Clerk caused some emails to fail silently. A session timeout issue meant one sponsor stayed logged in for 5 days straight and then couldn’t upload anything.

We learned, fixed, expanded. One tier at a time.

Rolling out to all 100 sponsors on day one would have been a disaster. Rolling out to 4 meant we had real production signal with contained blast radius. We fixed what broke before most of the custom