Agentic Commerce Demo – Voice-first LLM Checkout

This is a third rewrite of a demo I’ve been experimenting with that explores agent-driven commerce using real tools. It started as a Python and HTML project, then moved to JavaScript and React once I hit limitations around orchestration, latency and modularity.

The core idea was to see if an LLM-based agent could guide a user through onboarding, product selection, and payment – all through voice – without relying on a traditional frontend or support flow.

Stack

• OpenAI GPTs for both orchestrator and specialist agents
• ElevenLabs for speech-to-text and text-to-speech
• Pinecone as the vector store for product recall and metadata
• Stripe Agent Toolkit for product catalogue and checkout links
• Salesforce as the customer record system
• Twilio for OTP
• Node.js as the backend layer
• ngrok to expose webhooks

What it does

• New user: Onboards via voice, sets up identity, verifies via OTP, browses products, and receives a checkout link
• Returning user: Recognised by phone number, verified again, and taken straight to purchase using a stored payment method

Everything is voice-driven and agent-orchestrated, from start to finish.

What went wrong

• Prompt structure was too fragile – especially around product handoff
• Agents would hallucinate product names or miss lookup steps
• Too much logic in prompt instructions made behaviour hard to debug
• Race conditions between OTP verification, Salesforce creation, and Stripe checkout
• Voice UX was inconsistent – speech-to-text accuracy was unreliable and latency was too high for fluid use

Why I built it this way

I avoided using vertical stacks on purpose. I wanted something composable, using real APIs that reflect what startups or teams might reach for. I also don’t write code the traditional way – most of this was done through iterative prompt design, test scaffolding, and pairing with LLMs via Cursor or Claude Code.

This isn’t meant to be polished. It’s functional, testable, and closer to what you might build internally for validation or customer education.

Link

Watch the demo: https://youtu.be/VQpAc2ye6BE