Originally published on LinkedIn, John’s Corner is where Fortimize CEO John Hamon shares the perspectives, challenges, and hard truths he sees from the front lines. View original publication.
No, you’re not being overly cautious.
Recent research confirms what everyone already feels: customer-facing agents are still scary and high-risk in regulated industries like banking, wealth, and insurance:
- A 2025 Microsoft + Salesforce paper (200k+ simulated conversations across 15 top LLMs) showed ~39% average performance drop in multi-turn vs. single-turn tasks. Models make early assumptions, lock them in, and rarely recover — even reasoning models at low temperature struggle.
- The follow-on paper (Feb 2026, “Intent Mismatch Causes LLMs to Get Lost in Multi-Turn Conversation”) doubles down: the issue persists in newer models (e.g., GPT-5.2 dropping from 92.7% to 48.5%). Root cause? Structural ambiguity in conversational context and intent alignment gaps — scaling and training alone won’t fix it; new approaches are needed.
Internal agents win today because of the problem space itself:
Internal queries (e.g., “Summarize this household/account/case”) start with rich CRM context: full history, relationships, and prior notes. The scope is well-bounded, so the intent of the query is clearer from the start, and fewer conversation turns are needed to reach a useful result. And higher error rates are acceptable anyway since it’s internal (therefore, easy to verify/escalate).
External agents? Customers enter cold: low initial context, open-ended intent, multi-turn back-and-forth to clarify. A misinterpreted turn easily cascades into incoherence, frustration, compliance risks, or financial exposure.
This isn’t just a governance/InfoSec issue. It’s an inherent probability of suboptimal outcomes, as evidenced by hard data.
2026 strategy for Salesforce customers:
Prioritize internal Agentforce use cases now — table stakes, tap into core capabilities. Immediate ROI, no surprise consumption bills.
Delay broad external agent rollout until multi-turn reliability matures (12–18+ months likely).
That is why we’re deeply focused on internal use cases: our clients generate measurable ROI in <90 days by focusing on supporting their employees, while building data/governance for safe external use in the future.
Internal agents: deploying aggressively?
External: still too risky in your view?
