Skip to main content

​
Speaker diarization

Automatically detect each speaker in multi-speaker audio recordings.
Example diarization output
Key input parameters:
  • url: Audio file to process, as a public URL, a signed URL, or a media:// reference
  • model: Choose the diarization model
  • numSpeakers: Expected number of speakers, leave empty for automatic detection
  • minSpeakers/maxSpeakers: Range for speaker detection
  • exclusive: Enable exclusive diarization mode, equivalent to diarization but without overlapping speech. Useful for easier reconciliation with STT/ASR results.
  • vadSensitivity: Tune voice activity detection between precision and recall
  • crosstalkSensitivity: Tune overlapping speech detection between precision and recall
  • speechProbability: Include a frame-level speech probability curve
  • crosstalkProbability: Include a frame-level overlapping speech probability curve
  • speakerProbability: Include per-speaker, frame-level probability curves
  • turnLevelConfidence: Include per-segment confidence scores
  • transcription/transcriptionConfig: Enable and configure speaker-attributed transcription
  • webhook/webhookStatusOnly: Receive results by webhook when the job completes
Learn how to diarize an audio file

​
Streaming / real-time diarization

Track who is speaking in live audio. The streaming API receives 100 ms audio chunks over WebSocket and emits speaker start/end events as the conversation happens. Learn how to diarize live audio

​
Speaker Identification vs. Diarization

Diarization answers “who spoke when?” with generic labels (SPEAKER_00, SPEAKER_01, etc.). Identification answers “who is speaking?” by recognizing specific known voices using voiceprints.

​
Voiceprint

Captures a speaker’s voice to identify that person in other audio recordings. Best practices:
  • Use clear, high-quality audio (max 30 seconds)
  • One voiceprint per speaker
Learn how to identify speakers with voiceprints

​
Output scores

Every score the API returns runs from 0 to 100, and comes in two families. Confidence scores measure how certain the model is about a speaker assignment — per segment with turnLevelConfidence, and per voiceprint match automatically on identification jobs. Frame-level probability scores are well-calibrated probabilities sampled every 20ms on a shared time axis:
  • speechProbability — is anyone speaking?
  • crosstalkProbability — is more than one person speaking?
  • speakerProbability — is this particular speaker speaking?
Use them to assess reliability, gate quality automatically, and drive human-in-the-loop correction. Learn more about output scores

​
Overlapped speech detection

Detect when multiple speakers talk over each other and attribute overlapping speech to the correct speakers. Find overlapping speech by comparing timestamps of segments from different speakers. For example:
Example diarization output
In this example, both SPEAKER_00 and SPEAKER_01 are talking between 12.5-14.0 seconds. You can also use the segment timestamps to calculate statistics such as total speaker time per speaker, total overlap duration, and percentage of overlapped speech, etc.

​
Voice activity and cross-talk sensitivity

Adjust how sensitively the API detects speech and overlapping speech using the vadSensitivity and crosstalkSensitivity parameters. Available with the precision-3 model. Learn more about voice activity and cross-talk sensitivity

​
STT Orchestration: Speaker-attributed transcripts

We host open-source transcription models like Nvidia Parakeet-tdt-0.6b-v3 and OpenAI whisper-large-v3-turbo with specialized STT + diarization reconciliation logic for speaker-attributed transcripts. To use this feature, make a request to the diarize API endpoint with the transcription: true flag. If you want to configure other parameters, like the transcription model to use, refer to the API reference. Learn more about speech to text with diarization Already have your own transcript? Merge it with our diarization results using this tutorial.