Introducing “The Agents”: A New Weekly Show Where We Share Everything Happening With Our 20+ AI Agents in Production. The Good, The Bad, and The Broken.

We get asked about our agents probably 50 times a week. CEOs of public companies. Founders just deploying their first AI SDR. RevOps leaders trying to figure out if they should build or buy. Everyone wants to know what’s actually happening behind the scenes when you run 20+ AI agents in production with a team of 3 humans. We can’t do 50 consulting calls a week. But we can do something better. Welcome to The Agents, Episode #001. This is a new weekly show with me and Amelia Lerutte, SaaStr’s Chief AI Officer, where we pull back the curtain on everything happening across our live agentic stack. Every week. All the bumps, breakthroughs, and real talk. No sugarcoating. Our goal is simple: accelerate your success on the agentic journey by sharing ours, including all the parts that don’t make it into the LinkedIn posts. Here’s what we covered in the debut episode: You Can Build It. But Who Maintains It? This is the meta question nobody talks about after you vibe code your first app. And it’s the question that explains why “I’m going to kill Salesforce with my vibe coded CRM” is still mostly a meme. Getting an app into production is like closing a sale. It’s the start of a journey, not the end. We walked through three live examples from just this week: 1. Preview environment outage. Several of our apps lost database connectivity in preview. Production was fine, but we couldn’t iterate on anything for hours. Amelia’s initial diagnosis was wrong. The agent tried to help but then blamed Qualified (our inbound tool), which wasn’t the issue. Then it blamed other third-party integrations. It just kept pointing fingers at the most complex integration it could find rather than identifying the actual problem. The real question: if you don’t have someone checking your agents 24/7, how long before you even notice the backend is broken while the frontend looks fine? Days, maybe. 2. Micro hallucinations in 10K, our AI VP of Marketing. 10K has 5 years of revenue data, hundreds of millions worth of attendee and sponsor data points, beautiful graphs, proactive daily check-ins. It’s very good. But it keeps getting confused about what year it is. Yesterday it told us we were 44% ahead of plan. This morning, 11%. Same agent, same data, same day. When I asked what happened, it said: “Oh yeah, I was comparing to the wrong year. And because I didn’t have the right year, I made up the data.” I now spend about 15 minutes a day maintaining 10K. Two weeks ago I wasn’t doing that at all. Without it, the agent drifts. Slowly, quietly, further from reality. 3. Model-based regressions in our pitch deck analyzer. We’ve graded over 4,000 startup pitch decks. The analyzer runs two passes through Claude with complex data extraction. It was stable for months. Then around January, without any code changes on our end, it started telling every startup they had $100K in revenue growing 500%. Again and again. What happened? A subtle model update (probably a dot release) introduced hallucinations into a complex multi-step workflow. I kept fixing it. It kept breaking. The code didn’t change. The model did. Three examples. One conclusion: set and forget does not work with agents. Clay’s Agent Tried to Charge Us 5x. And Then Told Us to Upgrade. We’re big fans of Clay. We use it heavily for enrichment and lookalike targeting. But this story is worth telling because it’s going to happen to every company that puts an AI agent in front of customers. Amelia was building a VIP list late on a Sunday night. Same workflow she’d run the week before. Clay’s Sculptor agent quoted her roughly 11,000 credits for what had cost about 2,500 the previous week. 5x. When she pushed back, she caught two things: First, the agent had defaulted to the most expensive enrichment model when a cheaper one would produce the same result. She called it out and got the cost down by half. Most customers wouldn’t have known to do that. Second, the agent wasn’t properly trained on Clay’s own new pricing. Clay had just rolled out more complex pricing. The agent didn’t understand how the new pricing actually worked, so it steered Amelia toward upgrading her plan when she didn’t need to. She ended up clicking the upgrade button at 11pm on a Sunday because she was tired and needed to get the work done. That shouldn’t be on the customer. When she flagged it to Clay’s team, they acknowledged the Sculptor wasn’t fully trained on the new pricing scenarios. It’s resolved now. But the lesson is universal: if you don’t constantly train your customer-facing agents on every product change, every pricing update, every new workflow, they will give your customers wrong answers. Read a segment of your agent’s customer interactions every single day. Forever. No Lead Left Behind: The Simplest Unlock of the Entire Agentic Journey. The reason all our agents work is not because they’re smarter than humans. It’s because there is no lead left behind. Every person who hits our website can talk to an agent and then a human. Every prospect who wants a discount finds out in real time. Every sponsor, even one that raised $2M at a demo day and a human rep might dismiss, still gets a conversation. The agent doesn’t judge. The agent doesn’t get busy. The agent doesn’t decide that a B lead isn’t worth the time. Even at our scale, Amelia admitted this week that she had leads left behind because she was sprinting on production deadlines. Her agents were sitting idle, ready to work. All our agents are idle 90% of the time. We have an order of magnitude more capacity than we had pre-agents. The next frontier is figuring out how to fully use that capacity. But step one is just making sure every single lead, prospect, and customer gets touched the way they want to be touched, in real time. Salesforce Put Qualified on Their Homepage the Day the Deal Closed. We noticed this week that if you go to salesforce.com without logging in, you’ll see a Qualified agent instead of the old support agent. The day the acquisition deal was done, they flipped it. Smart move. AgentForce works, but it’s broad and extensible. Qualified is a focused GTM agentic tool. Historically it was just about qualifying inbound. It’s expanding into outbound now. But it’s narrower, faster to deploy, and easier for GTM teams to use. The interesting detail: Qualified’s original avatar was modeled after a real person (Blake). Salesforce swapped it to a 3D cartoon version. Probably a combination of IP/likeness considerations and wanting to signal that this is now a Salesforce product.