diff --git a/fern/calls/voicemail-detection.mdx b/fern/calls/voicemail-detection.mdx index f120e7614..ed39b8549 100644 --- a/fern/calls/voicemail-detection.mdx +++ b/fern/calls/voicemail-detection.mdx @@ -5,7 +5,7 @@ slug: calls/voicemail-detection When you're running outbound voice agents, voicemails are a reality — but wasting time or missing opportunities because of them shouldn't be. -**Vapi's updated voicemail detection system** gives you faster, smarter, and more flexible handling of voicemail events, so you can keep your calls efficient, responsive, and professional. +**Vapi's voicemail detection** gives you faster, smarter, and more flexible handling of voicemail events, so you can keep your calls efficient, responsive, and professional. ## **Why Voicemail Detection Matters** @@ -16,9 +16,9 @@ When you're running outbound voice agents, voicemails are a reality — but wast --- -## **Today's Detection Options** +## **Detection Options** -You can now choose between several detection methods — but not all are created equal: +You can choose between several detection methods — but not all are created equal: | Detection Method | Strengths | Weaknesses | Recommendation | | :--------------- | :-------- | :--------- | :------------- | @@ -26,11 +26,11 @@ You can now choose between several detection methods — but not all are created | **Google** | Very good accuracy, reliable | Slightly longer detection time than Vapi | ✅ Recommended | | **OpenAI** | High accuracy, flexible phrasing | Higher cost | ✅ Good option if budget allows | | **Twilio** (legacy) | Very fast machine beep detection | Prone to false positives | ⚠️ Use only in special cases | -| **Vapi Voicemail Tool** (legacy) | Keyword-based detection in transcription | Slow, context-dependent | ⚠️ Use only if needed | +| **[Vapi Voicemail Tool](/tools/voicemail-tool)** (beta) | Assistant-driven voicemail decisions | Most cost effective, requires good prompting | ✅ Best for customization and cost efficiency | --- -## **New Default Behavior: Vapi Voicemail Detection** +## **Vapi Voicemail Detection** With **Vapi Voicemail Detection**, your assistant will: @@ -39,13 +39,13 @@ With **Vapi Voicemail Detection**, your assistant will: - **Interrupt the bot's first message** appropriately if voicemail is detected mid-sentence. - **Minimize false positives** by combining audio analysis (beeps) and transcription intelligence. -All three providers — **Vapi, Google, and OpenAI** — now support **interruption handling** and **false positive protection**. +All three providers — **Vapi, Google, and OpenAI** — support **interruption handling** and **false positive protection**. --- ## **How to Configure It** -On the **Assistants tab**, you'll find an updated Voicemail Detection section: +On the **Assistants tab**, you'll find the Voicemail Detection section: ![Vapi Voicemail Detection Configuration](/static/images/advanced-tab/vapi-voicemail-detection.png) @@ -54,24 +54,47 @@ You can choose your preferred detection provider: - **Google** - **OpenAI** - **Twilio** -- **Tool-based (legacy)** +- **Tool-based (beta)** ## **Advanced Configuration Options** For each detection method, you can fine-tune the following parameters: + +**Important:** `frequencySeconds` has a minimum allowed value of 2.5 seconds. + + | Parameter | Description | | :-------- | :---------- | -| **Initial Detection Delay** | How long to wait (in seconds) before starting voicemail detection. | -| **Detection Retry Interval** | How frequently to check for voicemail after the initial delay. | -| **Max Detection Retries** | Maximum number of detection attempts before stopping. | -| **Max Voicemail Message Wait** | Maximum time to wait before leaving a voicemail if no beep is detected. | +| **type** | Detection method: `audio` (default, best for Google/Vapi) or `transcript` (ASR-based, best for OpenAI). Only `transcript` is supported for Google and OpenAI providers. | +| **backoffPlan.startAtSeconds** | How long to wait (in seconds) before starting voicemail detection. | +| **backoffPlan.frequencySeconds** | How frequently to check for voicemail after the initial delay. | +| **backoffPlan.maxRetries** | Maximum number of detection attempts before stopping. | +| **beepMaxAwaitSeconds** | Maximum duration from call start to wait for a voicemail beep before speaking the message. If set too low, the bot may start speaking before the actual beep and get cut off. Default: 30 seconds (0-60 range). | These settings allow you to balance: - **Speed** (how quickly voicemail is detected) - **Accuracy** (reducing false positives) - **Cost** (fewer detection attempts = lower API costs) +### **Important: beepMaxAwaitSeconds Configuration** + +The `beepMaxAwaitSeconds` parameter is critical for voicemail message timing: + +- **What it does**: Sets the maximum time from call start to wait for a voicemail beep before the bot starts speaking its message +- **If beep detected early**: Bot speaks immediately after the beep (optimal) +- **If no beep by timeout**: Bot starts speaking the voicemail message anyway +- **Risk of low values**: Bot may start speaking before the actual beep and get cut off by the voicemail system + + +**Setting too low a value** (under 15-20 seconds) may cause your voicemail message to be cut off. Most voicemail systems play 10-20 seconds of greeting before the beep. + + +**Recommended values:** +- **Conservative**: 25-30 seconds (default: 30) +- **Aggressive**: 15-20 seconds (requires testing with your specific voicemail patterns) +- **Range**: 0-60 seconds + --- ## **How Vapi Detection Works** @@ -86,4 +109,493 @@ This hybrid approach means **less call delay, fewer mistakes, and a much more na --- -By switching to Vapi's new detection system, you'll avoid the common pitfalls of older voicemail detection, while creating a **faster, smarter, and more professional experience** for your users. +## **Complete Configuration Examples** + +Here are complete assistant configurations for different real-world scenarios: + +### **Sales Outreach Assistant** + +This configuration optimizes for fast detection and professional voicemail delivery in sales scenarios: + + +```json title="API Configuration" +{ + "name": "Sales Outreach Assistant", + "voice": { + "provider": "vapi", + "voiceId": "Paige" + }, + "model": { + "model": "gpt-4o", + "messages": [ + { + "role": "system", + "content": "You are a professional sales representative calling prospects about our software solutions. Be friendly, concise, and respect their time. If you reach voicemail, keep your message under 30 seconds." + } + ], + "provider": "openai", + "temperature": 0.3 + }, + "firstMessage": "Hi! This is Sarah from TechSolutions. I hope I'm catching you at a good time.", + "voicemailMessage": "Hi, this is Sarah from TechSolutions. I'm calling about the software demo you requested. I'd love to show you how we can help streamline your operations and save you time. Please call me back at 555-0123, or I'll try you again tomorrow. Thanks!", + "endCallMessage": "Thanks for your time. Have a great day!", + "transcriber": { + "model": "nova-2", + "language": "en", + "provider": "deepgram", + "smartFormat": true + }, + "firstMessageMode": "assistant-waits-for-user", + "voicemailDetection": { + "provider": "vapi", + "backoffPlan": { + "maxRetries": 5, + "startAtSeconds": 2, + "frequencySeconds": 2.5 + }, + "beepMaxAwaitSeconds": 12 + }, + "messagePlan": { + "idleMessages": ["Hello? Are you still there?"], + "idleTimeoutSeconds": 8 + }, + "startSpeakingPlan": { + "waitSeconds": 0.7, + "smartEndpointingPlan": { + "provider": "livekit", + "waitFunction": "2000 / (1 + exp(-10 * (x - 0.5)))" + } + }, + "stopSpeakingPlan": { + "numWords": 2, + "backoffSeconds": 0.8 + }, + "backgroundDenoisingEnabled": true +} +``` +```typescript title="TypeScript SDK" +import { VapiClient } from "@vapi-ai/server-sdk"; + +const vapi = new VapiClient({ token: process.env.VAPI_API_KEY }); + +const salesAssistant = await vapi.assistants.create({ + name: "Sales Outreach Assistant", + voice: { + speed: 0.9, + provider: "vapi", + voiceId: "Paige" + }, + model: { + model: "gpt-4o", + messages: [{ + role: "system", + content: "You are a professional sales representative calling prospects about our software solutions. Be friendly, concise, and respect their time. If you reach voicemail, keep your message under 30 seconds." + }], + provider: "openai", + temperature: 0.3 + }, + firstMessage: "Hi! This is Sarah from TechSolutions. I hope I'm catching you at a good time.", + voicemailMessage: "Hi, this is Sarah from TechSolutions. I'm calling about the software demo you requested. I'd love to show you how we can help streamline your operations and save you time. Please call me back at 555-0123, or I'll try you again tomorrow. Thanks!", + voicemailDetection: { + provider: "vapi", + backoffPlan: { + maxRetries: 5, + startAtSeconds: 2, + frequencySeconds: 2.5 + }, + beepMaxAwaitSeconds: 12 + }, + transcriber: { + model: "nova-2", + language: "en", + provider: "deepgram", + endpointing: 8, + smartFormat: true + }, + backgroundDenoisingEnabled: true +}); +``` +```python title="Python SDK" +from vapi import Vapi + +client = Vapi(token=os.getenv("VAPI_API_KEY")) + +sales_assistant = client.assistants.create( + name="Sales Outreach Assistant", + voice={ + "provider": "vapi", + "voiceId": "Paige" + }, + model={ + "model": "gpt-4o", + "messages": [{ + "role": "system", + "content": "You are a professional sales representative calling prospects about our software solutions. Be friendly, concise, and respect their time. If you reach voicemail, keep your message under 30 seconds." + }], + "provider": "openai", + "temperature": 0.3 + }, + first_message="Hi! This is Sarah from TechSolutions. I hope I'm catching you at a good time.", + voicemail_message="Hi, this is Sarah from TechSolutions. I'm calling about the software demo you requested. I'd love to show you how we can help streamline your operations and save you time. Please call me back at 555-0123, or I'll try you again tomorrow. Thanks!", + voicemail_detection={ + "provider": "vapi", + "backoff_plan": { + "max_retries": 5, + "start_at_seconds": 2, + "frequency_seconds": 2 + }, + "beep_max_await_seconds": 12 + }, + transcriber={ + "model": "nova-2", + "language": "en", + "provider": "deepgram", + "smart_format": True + }, + background_denoising_enabled=True +) +``` + + +### **Customer Support Callback Assistant** + +Optimized for high-accuracy detection and detailed voicemail messages: + + +```json title="API Configuration" +{ + "name": "Customer Support Assistant", + "voice": { + "provider": "vapi", + "voiceId": "Elliot" + }, + "model": { + "model": "gpt-4o", + "messages": [ + { + "role": "system", + "content": "You are a customer support representative calling customers back about their support tickets. Be empathetic, helpful, and provide clear next steps. Always reference their ticket number." + } + ], + "provider": "openai", + "temperature": 0.2 + }, + "firstMessage": "Hello! This is Maya from Customer Support. I'm calling back about your recent support request.", + "voicemailMessage": "Hi, this is Maya from Customer Support calling about ticket #{{ticketNumber}}. I have some updates on your issue and want to help resolve this quickly. Please call me back at 1-800-SUPPORT, or reply to your support email and I'll get back to you within 2 hours. Thanks!", + "voicemailDetection": { + "provider": "google", + "backoffPlan": { + "maxRetries": 8, + "startAtSeconds": 3, + "frequencySeconds": 3 + }, + "beepMaxAwaitSeconds": 20 + }, + "transcriber": { + "model": "nova-2", + "language": "en", + "provider": "deepgram", + "endpointing": 12, + "smartFormat": true + }, + "backgroundDenoisingEnabled": true +} +``` +```typescript title="TypeScript SDK" +const supportAssistant = await vapi.assistants.create({ + name: "Customer Support Assistant", + voice: { + provider: "vapi", + voiceId: "Elliot" + }, + model: { + model: "gpt-4o", + messages: [{ + role: "system", + content: "You are a customer support representative calling customers back about their support tickets. Be empathetic, helpful, and provide clear next steps. Always reference their ticket number." + }], + provider: "openai", + temperature: 0.2 + }, + voicemailDetection: { + provider: "google", // Using Google for maximum accuracy with audio detection + backoffPlan: { + maxRetries: 8, + startAtSeconds: 3, + frequencySeconds: 3 + }, + beepMaxAwaitSeconds: 20 + }, + firstMessage: "Hello! This is Maya from Customer Support. I'm calling back about your recent support request.", + voicemailMessage: "Hi, this is Maya from Customer Support calling about ticket #{{ticketNumber}}. I have some updates on your issue and want to help resolve this quickly. Please call me back at 1-800-SUPPORT, or reply to your support email and I'll get back to you within 2 hours. Thanks!" +}); +``` + + +### **Appointment Reminder Assistant** + +Balanced configuration for appointment confirmations with fallback voicemail: + + +```json title="API Configuration" +{ + "name": "Appointment Reminder Assistant", + "voice": { + "provider": "vapi", + "voiceId": "Rohan" + }, + "model": { + "model": "gpt-4o", + "messages": [ + { + "role": "system", + "content": "You are calling to remind patients about their upcoming appointments. Be warm, professional, and provide all necessary details including date, time, and preparation instructions." + } + ], + "provider": "openai", + "temperature": 0.1 + }, + "firstMessage": "Hi! This is Ryan calling from Dr. Smith's office about your upcoming appointment.", + "voicemailMessage": "Hi, this is Ryan from Dr. Smith's office calling to remind you about your appointment on {{appointmentDate}} at {{appointmentTime}}. Please arrive 15 minutes early and bring your insurance card. If you need to reschedule, please call us at 555-CLINIC. Thanks!", + "voicemailDetection": { + "provider": "vapi", + "backoffPlan": { + "maxRetries": 6, + "startAtSeconds": 2.5, + "frequencySeconds": 2.5 + }, + "beepMaxAwaitSeconds": 15 + }, + "transcriber": { + "model": "nova-2", + "language": "en", + "provider": "deepgram", + "endpointing": 10, + "smartFormat": true + } +} +``` + + +--- + +## **Provider-Specific Configurations** + +### **Vapi Provider (Recommended)** +Best balance of speed and accuracy: + +```json +{ + "voicemailDetection": { + "provider": "vapi", + "backoffPlan": { + "maxRetries": 5, + "startAtSeconds": 2, + "frequencySeconds": 2.5 + }, + "beepMaxAwaitSeconds": 12 + } +} +``` + +### **Google Provider** +Maximum accuracy for critical calls (best with audio detection): + +```json +{ + "voicemailDetection": { + "provider": "google", + "backoffPlan": { + "maxRetries": 8, + "startAtSeconds": 3, + "frequencySeconds": 3 + }, + "beepMaxAwaitSeconds": 20 + } +} +``` + +### **OpenAI Provider** +High accuracy with cost consideration (best with transcript detection): + +```json +{ + "voicemailDetection": { + "provider": "openai", + "type": "transcript", + "backoffPlan": { + "maxRetries": 6, + "startAtSeconds": 2.5, + "frequencySeconds": 3 + }, + "beepMaxAwaitSeconds": 15 + } +} +``` + +--- + +## **Detection Types** + +Choose between two detection methods based on provider performance: + +| Detection Type | Performance | Providers | Recommended For | +|---------------|-------------|-----------|-----------------| +| **`audio`** (default) | Best for Google, Vapi | All providers | Google: Maximum accuracy
Vapi: General use cases | +| **`transcript`** | Best for OpenAI | Google, OpenAI only | OpenAI: Optimal performance
High-volume campaigns with OpenAI | + + +**Provider-specific recommendations:** +- **Google**: Use default `audio` detection for best accuracy +- **OpenAI**: Use `type: "transcript"` for optimal performance +- **Vapi**: Use default `audio` detection + + +--- + +## **Disabling Voicemail Detection** + +To completely disable voicemail detection for your assistant, set the `voicemailDetection` property to `"off"`: + + +```json title="API Configuration" +{ + "name": "Assistant Without Voicemail Detection", + "voicemailDetection": "off", + "model": { + "provider": "openai", + "model": "gpt-4o" + } +} +``` +```typescript title="TypeScript SDK" +const assistant = await vapi.assistants.create({ + name: "Assistant Without Voicemail Detection", + voicemailDetection: "off", + model: { + provider: "openai", + model: "gpt-4o" + } +}); +``` +```python title="Python SDK" +assistant = client.assistants.create( + name="Assistant Without Voicemail Detection", + voicemail_detection="off", + model={ + "provider": "openai", + "model": "gpt-4o" + } +) +``` + + + +When voicemail detection is disabled, your assistant will continue the conversation normally regardless of whether it reaches a voicemail system. + + +--- + +## **Configuration Best Practices** + +### **Tuning for Different Scenarios** + +| Use Case | Recommended Provider | startAtSeconds | frequencySeconds | maxRetries | beepMaxAwaitSeconds | +|----------|---------------------|----------------|------------------|------------|-------------------| +| **Sales Outreach** | Vapi | 2 | 2.5 | 5 | 25 | +| **Customer Support** | Google | 3 | 3 | 8 | 20 | +| **Appointment Reminders** | Vapi | 2.5 | 2.5 | 6 | 15 | +| **Lead Qualification** | Vapi | 1.5 | 2.5 | 4 | 20 | +| **Follow-up Calls** | Google | 2.5 | 3 | 7 | 18 | + +### **Cost Optimization Tips** + +1. **Lower maxRetries** for high-volume campaigns +2. **Increase startAtSeconds** to reduce false positives +3. **Use Vapi provider** for best cost-to-accuracy ratio +4. **Tune beepMaxAwaitSeconds** carefully - too low causes cut-off messages, too high delays voicemail delivery + +### **Accuracy Optimization Tips** + +1. **Use Google provider** for maximum accuracy +2. **Increase maxRetries** for important calls +3. **Lower startAtSeconds** for faster detection +4. **Tune frequencySeconds** based on your voicemail patterns (minimum 2.5 seconds) + +--- + +## **Troubleshooting Common Issues** + +### **False Positives (Detecting voicemail when human answers)** + +**Symptoms:** Assistant leaves voicemail message when human picks up + +**Solutions:** +- Increase `startAtSeconds` to 3-4 seconds +- Switch to Google or OpenAI provider +- Increase `frequencySeconds` to 3-4 seconds + +```json +{ + "voicemailDetection": { + "provider": "google", + "backoffPlan": { + "startAtSeconds": 3.5, + "frequencySeconds": 3.5, + "maxRetries": 6 + } + } +} +``` + +### **Missed Voicemails (Not detecting actual voicemail)** + +**Symptoms:** Assistant continues talking to voicemail recording + +**Solutions:** +- Decrease `startAtSeconds` to 1-2 seconds +- Increase `maxRetries` to 8-10 +- Keep `frequencySeconds` at minimum value (2.5 seconds) + +```json +{ + "voicemailDetection": { + "provider": "vapi", + "backoffPlan": { + "startAtSeconds": 1.5, + "frequencySeconds": 2.5, + "maxRetries": 8 + } + } +} +``` + +### **Slow Detection** + +**Symptoms:** Takes too long to detect voicemail + +**Solutions:** +- Use Vapi provider for fastest detection +- Decrease `startAtSeconds` (frequencySeconds minimum is 2.5) +- Ensure good audio quality + +```json +{ + "voicemailDetection": { + "provider": "vapi", + "backoffPlan": { + "startAtSeconds": 1, + "frequencySeconds": 2.5, + "maxRetries": 6 + } + } +} +``` + +--- + +By using Vapi's detection system, you'll avoid the common pitfalls of voicemail detection, while creating a **faster, smarter, and more professional experience** for your users. + +## **Related Documentation** + +- **[Voicemail Tool](/tools/voicemail-tool)** - Alternative assistant-controlled voicemail approach for maximum flexibility