Voice activity detection (VAD)
Voice activity detection (VAD) is the process of identifying which parts of an audio stream contain human speech and which contain silence or non-speech sound.
How VAD works
A VAD component evaluates short segments of incoming audio and estimates whether speech is present. Basic approaches use properties such as signal energy and frequency patterns. More advanced approaches can better separate speech from background sounds, but every method still has to make decisions under uncertain conditions.
VAD does not determine what the caller said. Speech-to-text performs transcription, while VAD supplies timing information about when speech appears to start or stop. That signal can help an AI phone agent begin capturing an utterance, avoid processing long periods of silence, and notice when a caller speaks while the agent's audio is playing.
VAD is also not the complete endpointing decision. A pause may indicate the end of a turn, or it may occur in the middle of a phone number, address, or thoughtful response. Endpointing uses the available speech timing, and sometimes additional context, to decide when the system should answer.
Why it matters for AI phone calls
If VAD is too sensitive, keyboard clicks, road noise, hold music, or another person in the room may be treated as caller speech. That can interrupt playback or send meaningless audio for transcription. If it is not sensitive enough, quiet words and short confirmations may be missed.
The practical tradeoff is responsiveness versus stability. Fast detection can make interruption and turn-taking feel immediate, but false triggers make a conversation erratic. Conservative detection avoids some noise while risking clipped speech or delayed responses.
Teams should test VAD behavior with the audio their callers actually produce: speakerphone echo, mobile connections, background conversation, varied speaking volume, short answers, and pauses inside longer details. The right behavior depends on the call flow. A rapid qualification call may favor quick response, while collecting an address or account number may require more tolerance for pauses.