The challenge
Friendsy needed a production platform where non-technical users can deploy AI agents that answer and place real phone calls, and hold a natural conversation. The hard part of voice AI is latency: every extra half-second between the caller finishing a sentence and the AI replying makes the agent feel robotic. The system had to run speech-to-text, an LLM, and text-to-speech in a real-time streaming loop, stay fast under load, and remain flexible enough to swap providers at every stage.
What we built
We took the product from concept to production. At the core is a Pipecat-based real-time audio pipeline over WebRTC, connected to phone networks through Twilio and other VOIP providers for both inbound and outbound calling. An STT → LLM → TTS routing layer lets each agent mix and match providers (Deepgram or OpenAI Whisper for transcription; GPT-4.1, Claude, or Gemini for reasoning; Cartesia or ElevenLabs for voice) with live cost estimation for any combination.
Around the pipeline sits the SaaS layer: a Next.js dashboard with a step-by-step agent builder, phone number management, call logs and recordings, real-time analytics over WebSockets, and a flexible API for programmatic control. The backend runs on FastAPI with PostgreSQL, Redis, and S3 storage, shipped in Docker containers to AWS/GCP through CI/CD pipelines, engineered to scale from the first customer without a rewrite.
The result
A production SaaS platform live on real phone lines: 0.8-second average AI response latency, a 94.2% call success rate, and thousands of calls handled, with an architecture the Friendsy team can confidently extend, from new AI providers to new telephony integrations.
"Working with your team was a strong experience from start to finish. You delivered a production ready SaaS solution with solid architecture, clean code, and a clear focus on scalability and long term maintainability."