# Account
Source: https://docs.pyannote.ai/administration/account
Create your pyannoteAI account and manage account and team settings.
Create a pyannoteAI account to access the dashboard, use the API playground, generate API keys, monitor usage, and manage billing.
Create an account or sign in at [dashboard.pyannote.ai](https://dashboard.pyannote.ai).
## Create an account
1. Go to [dashboard.pyannote.ai](https://dashboard.pyannote.ai)
2. Sign up with your work email or GitHub account
3. Complete your account setup
## Dashboard and API playground
After signing in, you can use the dashboard to:
* Use the API playground to test the latest models without writing code
* Create and manage API keys
* View usage information
* Access billing details and current pricing
## Account and team management
In team settings, you can invite members to your team.
You can also delete a team from the same page if you are the team owner.
To delete your account, you must first leave or delete every team you belong to.
# Billing
Source: https://docs.pyannote.ai/administration/billing
Understand how pyannoteAI billing works for diarization, identification, voiceprint jobs, and streaming.
Use the billing and usage page of the [dashboard](https://dashboard.pyannote.ai) to check your current pricing, available credits or subscription status, recent usage, and monthly budget.
## Included usage
The Developer and Starter plans have included monthly usage that can be spent on any endpoint. Once your included usage is consumed, additional usage is billed at the same per-unit rate (pay-as-you-go).
## How billing works
Only jobs that reach the `succeeded` status are billed. Rejected job submissions, failed jobs, and canceled jobs are not billed.
`/diarize` and `/identify` jobs are billed based on the audio duration in seconds that is sent for processing.
`/voiceprint` jobs are billed per voiceprint created rather than by audio duration.
Streaming sessions created via `POST /live` are billed based on the audio duration in seconds sent over the WebSocket connection and processed successfully. A stream is billed for the duration processed, regardless of how it ends.
If transcription is enabled on `/diarize`, the job is billed at the transcription price shown as `STT Orchestration` in our plans. You still receive diarization segments as part of the output at no additional cost.
### Minimum charge
There is a 20-second minimum charge for all successful jobs to `/diarize`, `/identify`, and `/live`. If a job or stream processes an audio duration that is less than 20 seconds, it is billed as 20 seconds.
Example charges:
* An 8-second `/diarize` job is billed as 20 seconds
* A 20-second `/identify` job is billed as 20 seconds
* A 182-second `/diarize` job is billed as 182 seconds
* A `/voiceprint` job is billed per created voiceprint
* A 5-second stream is billed as 20 seconds
* A 3-minute stream is billed as 180 seconds
## What can affect cost
* The endpoint you use
* The model that was selected
* The audio duration in seconds for `/diarize`, `/identify`, and `/live` streams
* Whether you requested transcription with a `/diarize` job
For your current per-unit rates (diarization, STT Orchestration, identification, voiceprint), use the plan page in the [dashboard](https://dashboard.pyannote.ai) as the source of truth.
## Setting a monthly budget
Use the **Monthly budget** setting on the billing page to manage your spend by configuring a monthly spend limit. You can also set thresholds to receive email alerts when your team reaches specific usage levels. You can update or remove these limits and alerts at any time.
There may be a delay in enforcing this limit, and you remain responsible for any overage incurred.
If your team exceeds the monthly budget during a billing period, subsequent API requests will be rejected.
# Create Stream
Source: https://docs.pyannote.ai/api-reference/create-stream
POST /v1/live
# Submit Diarization Job
Source: https://docs.pyannote.ai/api-reference/diarize
POST /v1/diarize
This endpoint allows you to create a new diarization from a remote audio URL.
For comprehensive guides on diarization, see these tutorials:
* [How to Diarize Audio](/tutorials/how-to-diarize-audio) - Complete diarization workflow
* [Diarization and Speech-to-Text Merge](/tutorials/diarization-asr-merge) - Combining diarization with transcription
* [How to Upload Files](/tutorials/how-to-upload-files) - Working with local files
* [Confidence Scores](/tutorials/confidence-scores) - Understanding and using confidence scores
* [Speaker Configuration](/tutorials/speaker-configuration) - Configuring speaker counts and overlapping speech
# Get Job
Source: https://docs.pyannote.ai/api-reference/get-job
GET /v1/jobs/{jobId}
This endpoint allows you to retrieve the details of a job that you have created, including the results if the job is completed.
The `output` field contains the job results and is only available when the job
status is `succeeded`. Job results are automatically deleted after 24 hours of
job completion.
# Get Stream Status
Source: https://docs.pyannote.ai/api-reference/get-stream
GET /v1/live/{id}
# Submit Identification Job
Source: https://docs.pyannote.ai/api-reference/identify
POST /v1/identify
This endpoint allows you to create a new diarization with speaker identification from a remote audio URL.
For a complete guide on speaker identification using voiceprints, see the [Identification with Voiceprints](/tutorials/identification-with-voiceprints) tutorial.
# List Jobs
Source: https://docs.pyannote.ai/api-reference/list-jobs
GET /v2/jobs
This endpoint allows you to list all the jobs that you have created. They are
sorted in descending order by the time they were created (latest job first).
For performance reasons, job results are not included in the response. Retrieve
the results by using the [get job](/api-reference/get-job) endpoint.
By default the API will return the first 10 jobs. You can use the `limit` and
`cursor` query parameters to paginate through the list of jobs.
# Diarization
Source: https://docs.pyannote.ai/api-reference/schemas/diarizationschema
A diarization object represents the results of a diarization job.
# Identify
Source: https://docs.pyannote.ai/api-reference/schemas/identifyschema
An identify object represents the results of an identify job.
# Voiceprint
Source: https://docs.pyannote.ai/api-reference/schemas/voiceprintschema
A voiceprint object represents the results of a voiceprint job.
# Stream audio
Source: https://docs.pyannote.ai/api-reference/streaming
# Test API Key
Source: https://docs.pyannote.ai/api-reference/test
GET /v1/test
This is a test endpoint. Use it to test your API key and ensure that it is working correctly.
For a complete guide on getting started with the API, see the [Quick Start](/quickstart) page.
# Upload Media File
Source: https://docs.pyannote.ai/api-reference/upload-media
POST /v1/media/input
To use the provided temporary storage is a two step process.
You start by declaring a media:// url that you can reference in any other API calls. The response will provide a url where you can put your media. This allows you to use the media:// url as a short-cut for a temporary storage location.
You'll be returned a pre-signed url you can use to PUT and upload your media file. The temporary storage should allow you to read and write to the media:// locations for a period of at least 24 hours before it is removed.
For cases when your file is not public accessible, you can upload your file to the pyannoteAI servers using this endpoint.
For a complete guide on uploading files, see the [How to Upload Files](/tutorials/how-to-upload-files) tutorial.
# Submit Voiceprint Job
Source: https://docs.pyannote.ai/api-reference/voiceprint
POST /v1/voiceprint
This endpoint allows you to create a new voiceprint from a remote audio URL.
For a complete guide on creating and using voiceprints for speaker identification, see the [Identification with Voiceprints](/tutorials/identification-with-voiceprints) tutorial.
# Authentication
Source: https://docs.pyannote.ai/authentication
All API endpoints are authenticated using your API Key as a Bearer token.
Generate an API key on the dashboard at [https://dashboard.pyannote.ai](https://dashboard.pyannote.ai).
```bash Request Example theme={null}
curl --request GET \
--url https://api.pyannote.ai/v1/test \
--header: 'Authorization: Bearer ' \
--header: 'Content-Type: application/json'
```
# Data retention
Source: https://docs.pyannote.ai/data-retention
Understanding our data lifecycle and deletion policies.
Learn how long we keep audio files, job results, and stream metadata. This page also covers our data residency practices and our commitment to privacy.
For more details, please also refer to our [Terms of use](https://pyannote.ai/terms-of-use) (which includes our Data Processing Agreement), [Privacy policy](https://pyannote.ai/privacy-policy), and [Trust Center](https://trust.pyannote.ai).
## AI model training
We never use your audio data, outputs produced by jobs or streams, or any other customer data to train our AI models.
If you report an issue with model quality, accuracy, or unexpected results, you may choose to share example data with us, including the audio file, the produced output, and your expected output. We may use that data for validation and testing purposes if you give us explicit consent to do so.
## How audio files are processed
You can submit jobs with two types of audio input:
1. An audio URL (e.g., a presigned S3 URL)
2. A file uploaded to our servers via the [Media API](/api-reference/upload-media)
In both cases, your audio is temporarily loaded on ephemeral workers (processing servers) that perform the requested operations (e.g., diarization, identification, voiceprinting, transcription) using our AI models. The local copy on the processing server is always deleted immediately after processing completes.
### When using audio URLs
When you provide a URL as input for a job (e.g., a presigned S3 URL), we temporarily download the audio file to our processing servers solely for the purpose of processing. No copy of the audio file is retained after processing completes.
We recommend creating presigned URLs with limited validity (e.g. 1 hour) when using private storage services
(e.g., AWS S3, Google Cloud Storage, Azure Blob Storage).
### When using uploaded files with [Media API](/api-reference/upload-media)
When you upload an audio file via the [Media API](/api-reference/upload-media), it is stored in our temporary storage (`media://`) and automatically deleted within **48 hours**, regardless of whether it has been used in a job.
When an uploaded file is used as input for a job, it is temporarily downloaded from our storage to the processing server. As with audio URLs, this local copy on the processing server is deleted immediately after processing completes. However, the uploaded file itself remains in temporary storage until the 48-hour retention period expires. You can reuse the same uploaded file across multiple jobs without re-uploading it, as long as it is within the 48-hour retention window.
## Streams
Streams process audio chunks on ephemeral workers as they are received. Audio chunks are never stored and are only processed on the ephemeral worker running the stream.
Stream outputs are sent over the WebSocket and are not stored. We retain stream metadata for billing and debugging purposes, such as the model used and stream duration in seconds.
## Job outputs
Job results are retained for **24 hours** after the job completes. All data in the `output` field of the job response is deleted after this period. This applies to all output types, including:
* **Diarization** segments
* **Identification** segments
* Generated **voiceprints**
* **Transcription** segments
We strongly recommend that you configure your systems to retrieve and store these results in your own database
immediately upon job completion.
## Data residency & infrastructure
Our core services are hosted on infrastructure provided by a subprocessor in the EEA. This includes the Dashboard, Playground, API servers, databases, and storage.
Audio processing with AI models is performed on ephemeral workers that temporarily load the data needed to run the job or stream. This includes diarization, identification, voiceprinting, transcription, and streaming diarization. The loaded data is deleted immediately after processing completes.
Streaming sessions (e.g. with the Live-1 model) are currently always processed on servers in the EEA. They are currently not affected by your processing region setting.
For jobs, these ephemeral workers may run in any region by default, including outside the EEA. If you select **EU (European Economic Area)** as your processing region, this audio processing is performed within the EEA only.
You can choose your processing region in the [processing region settings](https://dashboard.pyannote.ai/settings/_/processing-region):
* **Any region** (default): audio processing for jobs may be performed outside the EEA.
* **EU (European Economic Area)**: audio processing for jobs is performed within the EEA only. Because our core services and storage are also hosted in the EEA, your input and output data remain within the EEA.
Changes to this setting apply only to subsequent jobs submitted through the API or Playground. Existing and in-progress jobs are not affected.
For a list of our subprocessors, visit our [Trust Center](https://trust.pyannote.ai).
### Enterprise & self-hosted
We offer on-premise and self-hosted versions as part of our Enterprise plans. With the self-hosted solution, audio files and job results remain entirely within your own environment, server, or cloud infrastructure.
Custom DPA and BAA documents are available for Enterprise customers.
For all other plans, a Data Processing Agreement (DPA) is included in our [Terms of use](https://pyannote.ai/terms-of-use).
[Contact us](mailto:support@pyannote.ai) if you are interested in our Enterprise plans.
# Feature overview
Source: https://docs.pyannote.ai/features
Explore the key features of the pyannoteAI API
## Speaker diarization
Automatically detect each speaker in multi-speaker audio recordings.
```json Example diarization output theme={null}
[
{
"speaker": "SPEAKER_00",
"start": 10.0,
"end": 15.0
},
{
"speaker": "SPEAKER_01",
"start": 12.5,
"end": 14.0
}
]
```
**Key input parameters:**
* `num_speakers`: Expected number of speakers, leave empty for automatic detection
* `min_speakers`/`max_speakers`: Range for speaker detection
* `exclusive`: Enable exclusive diarization mode, equivalent to diarization but without overlapping speech. Useful for easier reconciliation with STT/ASR results.
* `model`: Choose diarization model
* `confidence`: Include confidence scores
**[Learn how to diarize an audio file ](/tutorials/how-to-diarize-audio)**
***
## Streaming / real-time diarization
Track who is speaking in live audio. The streaming API receives 100 ms audio chunks over WebSocket and emits speaker start/end events as the conversation happens.
**[Learn how to diarize live audio ](/tutorials/streaming-real-time)**
***
## Speaker Identification vs. Diarization
**Diarization** answers "who spoke when?" with generic labels (`SPEAKER_00`, `SPEAKER_01`, etc.).
**Identification** answers "who is speaking?" by recognizing specific known voices using voiceprints.
***
## Voiceprint
Captures a speaker's voice to identify that person in other audio recordings.
**Best practices:**
* Use clear, high-quality audio (max 30 seconds)
* One voiceprint per speaker
**[Learn how to identify speakers with voiceprints ](/tutorials/identification-with-voiceprints)**
***
## Confidence scores
Receive confidence scores for each speaker segment to assess reliability and perform human in the loop correction. Set the `confidence` parameter to `true` in your diarization or identification request.
**[Understanding confidence scores ](/tutorials/confidence-scores)**
***
## Overlapped speech detection
Detect when multiple speakers talk over each other and attribute overlapping speech to the correct speakers.
Find overlapping speech by comparing timestamps of segments from different speakers. For example:
```json Example diarization output theme={null}
[
{
"speaker": "SPEAKER_00",
"start": 10.0,
"end": 15.0
},
{
"speaker": "SPEAKER_01",
"start": 12.5,
"end": 14.0
}
]
```
In this example, both `SPEAKER_00` and `SPEAKER_01` are talking between `12.5`-`14.0` seconds.
You can also use the segment timestamps to calculate statistics such as total
speaker time per speaker, total overlap duration, and percentage of overlapped
speech, etc.
***
## STT Orchestration: Speaker-attributed transcripts
We host open-source transcription models like [Nvidia Parakeet-tdt-0.6b-v3](https://huggingface.co/nvidia/parakeet-tdt-0.6b-v3) and [OpenAI whisper-large-v3-turbo](https://huggingface.co/dropbox-dash/faster-whisper-large-v3-turbo) with specialized STT + diarization reconciliation logic for speaker-attributed transcripts.
To use this feature, make a request to the diarize API endpoint with the `transcription: true` flag. If you want to configure other parameters, like the transcription model to use, refer to the [API reference](/api-reference/diarize).
**[Learn more about speech to text with diarization ](/tutorials/speech-to-text-diarization)**
**Already have your own transcript?** Merge it with our diarization results using [this tutorial](/tutorials/diarization-asr-merge).
# Welcome to pyannoteAI
Source: https://docs.pyannote.ai/introduction
pyannoteAI is a state-of-the-art AI platform for speaker diarization, designed for developers and companies who rely on accurate voice analysis in complex audio environments.
Whether your audio is multi-speaker, noisy, or affected by external elements, pyannoteAI delivers precision and reliability where it matters most.
Want to test the latest models without writing code? Use the [API playground](/tutorials/api-playground)
Learn how to get started making API requests
Submit a diarization job to identify who spoke when in your audio files.
Explore our state-of-the-art diarization models to find the right fit for
your needs.
### With pyannoteAI you can:
* Use state-of-the-art speaker diarization and identification via voiceprints to determine **who spoke when** and **who is speaking** in any audio.
* Embed accurate diarization into your own products, from meeting tools to voice AI systems, using our simple API.
* Reliable diarization is the foundation of conversational AI. It improves transcription accuracy, enables speaker-specific logic, and powers downstream analytics or personalized models built on top of your data.
## Resources
Read articles and insights about speaker diarization and voice AI.
See what's new in pyannoteAI with our latest updates and improvements.
Join our community to ask questions and connect with other developers.
# Models
Source: https://docs.pyannote.ai/models
Choose the right speaker diarization model for your audio processing needs
Highest-accuracy diarization for recordings, with support for voiceprints and speaker identification.
Streaming diarization over WebSocket for live audio.
Open-source diarization for research, prototyping, and self-hosting.
## Choosing the right model
### Precision-2
**Best for:** Startups, SMEs, and enterprises who need the state-of-the-art in speaker diarization accuracy and advanced features like voiceprints and speaker identification.
Precision-2 is 28% more accurate, on average, than Community-1.
Self-hosted options for Precision-2 are available on Enterprise plans.
**Typical use cases:** phone call analytics, meeting transcription with speaker attribution, video dubbing and timestamp-critical workflows, building training data for voice assistants, and more.
**Advanced features:**
* **Speaker identification with voiceprints**: Identify known speakers in your audio using pre-enrolled voiceprints
* **Exclusive diarization mode:** Returns speaker diarization where only one single speaker (the most likely to be transcribed) is active at a time, making STT reconciliation easier
* **Flexible speaker count control:** Set `minSpeakers`, `maxSpeakers` and `numSpeakers` parameters for any number of speakers
* **Human-in-the-loop correction:** Use confidence scores to help streamline manual correction processes
[Learn more about Precision-2 ](https://www.pyannote.ai/blog/precision-2)
***
### Live-1
**Best for:** Teams building live voice products that need speaker labels before a recording finishes: contact centers, meeting tools, broadcast workflows, and real-time voice agents.
**Typical use cases:** live meeting assistants, contact center agent assist, live captioning with speaker attribution, broadcast speaker attribution, and multi-party voice agents that need to track who is speaking.
**Advanced features:**
* **Sub-300ms latency:** Speaker labels arrive fast enough for live captioning and real-time agent assist, tested against noisy, real-world audio rather than clean studio recordings.
* **Native streaming architecture:** Processes audio in 100ms chunks over WebSocket, with a speaker tracking layer that holds consistency across the stream without needing the full conversation.
* **Built for real conditions:** Trained and validated on overlapping speech, background noise, and conversations with more than two participants.
**Technical specs:**
* Up to 8 speakers, up to 5 hours duration per stream
* Input: 16 kHz mono audio, 100ms chunks over WebSocket
* Output: `diarization_speaker_start` and `diarization_speaker_end` events, each with a start or end timestamp and speaker label
[Learn more about Live-1](https://www.pyannote.ai/blog/how-we-built-streaming-diarization)
***
### Community-1 (hosted)
**Best for:** Teams who want the open-source model without managing infrastructure
**Typical use cases:** Prototyping, low-volume production workloads, testing and validation
**Key Benefits:**
* **Cost efficiency:** hosted at cost, ideal for experimentation and low-volume workloads
* **No infrastructure management:** Focus on your application while we handle the deployment
* **Easy migration:** Start with hosted Community-1 and upgrade to Precision-2 when needed
* **Same powerful model:** Access the same Community-1 model through our API without setup complexity
[Learn more about Community-1 ](https://www.pyannote.ai/blog/community-1)
***
### Community-1 (self-hosted with pyannote.audio 4.0)
**Best for:** Researchers, developers, and personal hobby projects who want full control over their diarization models and workflows.
**Typical use cases:** Academic work, product-iteration, prototyping, and custom diarization deployment (e.g., dataset-specific fine-tuning or custom reconciliation with STT).
**Key Benefits:**
* **Best open-source speaker diarization model available** - outperforms pyannote.audio 3.1 across all key metrics
* **Open-source flexibility:** Full transparency into model weights and code allowing local and offline training and inference.
**Trade-offs:**
* Lower accuracy compared to Precision-2
* No support for advanced features like speaker identification and voiceprints
* Requires deploying the model on your own infrastructure
[Learn more about pyannote.audio 4.0 ](https://github.com/pyannote/pyannote-audio)
***
## How to specify a model in diarization requests
When making a diarization request, you can specify which model to use using the `model` parameter:
```bash theme={null}
curl -X POST "https://api.pyannote.ai/v1/diarize" \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"url": "https://files.pyannote.ai/marklex1min.wav",
"model": "precision-2"
}'
```
By default, if you do not specify a model, the API will use the Precision-2 model.
### Switch between models
You can easily switch between models by changing the `model` parameter:
* `"model": "community-1"` for Community-1
* `"model": "precision-2"` for Precision-2
**Note:** Speaker identification and voiceprint features are not available for Community-1 models. These advanced features are exclusive to Precision-2
### Compare results between models
To compare performance between models on your specific data:
1. Process the same audio file with both models
2. Compare the diarization results
3. Evaluate which model provides better accuracy for your use case
## Pricing
For detailed pricing information, visit our [pricing page](https://www.pyannote.ai/pricing).
# OpenAPI specification
Source: https://docs.pyannote.ai/openapi
Download the latest OpenAPI specification for pyannote.ai at [https://docs.pyannote.ai/openapi.json](https://docs.pyannote.ai/openapi.json).
Use this specification to generate API clients or import it into tools such as [Bruno](https://usebruno.com/) or [Postman](https://www.postman.com/).
# Quickstart guide
Source: https://docs.pyannote.ai/quickstart
Get started with the pyannoteAI API in 5 minutes
Learn how to create an account, generate an API key, and make your first API call.
Want to test the latest models without writing code? Try the [API playground](/tutorials/api-playground)
Sign up for a free account at [dashboard.pyannote.ai](https://dashboard.pyannote.ai) to access the pyannoteAI platform.
1. Go to [dashboard.pyannote.ai](https://dashboard.pyannote.ai)
2. Signup with your work email or use your GitHub account to register
3. Complete your account setup
Once registered, you'll have access to your team dashboard where you can manage API keys, view usage statistics, and access billing details.
Need help with setup or billing? See [Account](/administration/account) and [Billing](/administration/billing).
Generate an API key to authenticate your requests to the pyannoteAI API.
1. In your dashboard, navigate to the **API Keys** section
2. Click **"Create new key"**
3. Give your key a descriptive name (e.g., "Development key")
4. Copy the key and store it securely (you won't be able to see it again after you leave the page)
![API Keys Page]()
Keep your API key secure and never expose it in client-side code or public repositories.
Verify your API key is working by making a test request to our test endpoint.
```bash Test API Call theme={null}
# Set your API key as an environment variable in production
# export PYANNOTE_API_KEY="your_api_key_here"
curl --request GET \
--url https://api.pyannote.ai/v1/test \
--header 'Authorization: Bearer YOUR_API_KEY_HERE' \
--header 'Content-Type: application/json'
```
```python test.py theme={null}
import requests
api_key = "YOUR_API_KEY_HERE" # In production, use environment variables: os.getenv("PYANNOTE_API_KEY")
url = "https://api.pyannote.ai/v1/test"
response = requests.get(url, headers={"Authorization": f"Bearer {api_key}"})
print(response.json())
```
```typescript test.ts theme={null}
const apiKey = "YOUR_API_KEY_HERE"; // In production, use environment variables: process.env.PYANNOTE_API_KEY
const url = "https://api.pyannote.ai/v1/test";
const response = await fetch(url, {
headers: {
Authorization: `Bearer ${apiKey}`,
},
});
console.log(await response.json());
```
If successful, you'll receive a response like:
```json theme={null}
{
"message": "API key is valid",
"status": "success"
}
```
## What's next?
Now that you have a working API key, you can start building with pyannoteAI.
Learn how to submit audio files for speaker diarization and process the
results.
Explore the complete API documentation for all available endpoints.
### Need help?
* 📖 Browse our [tutorials](/tutorials) for step-by-step guides
* ❓ Check our [FAQs](/support/faqs) for common questions
* 💬 Join our [Discord community](https://discord.gg/vux8UH9QmV) for support
* 📧 Email us at **[support@pyannote.ai](mailto:support@pyannote.ai)**
# Rate Limits
Source: https://docs.pyannote.ai/ratelimits
Rate limits are enforced per endpoint and scoped per team. Each endpoint has its own independent limit and a fixed window of 60 seconds.
The default rate limit is 100 requests per minute for submitting jobs, creating streams and media endpoints, and 300 requests per minute for listing jobs or getting a job by ID.
If you exceed a rate limit, you will receive a `429 Too Many Requests` response.
## Rate Limit Headers
Successful responses from endpoints include:
* `X-RateLimit-Limit`: Maximum requests allowed in current window.
* `X-RateLimit-Remaining`: Requests remaining in current window.
* `X-RateLimit-Reset`: Seconds until current window resets
When rate limited (`429`), response includes:
* `Retry-After`: Seconds to wait before retrying.
# FAQs
Source: https://docs.pyannote.ai/support/faqs
Common questions and answers.
On this page you'll find answers to common questions about our API and AI models.
This error typically occurs when the provided audio file URL is not a direct link or when there are issues with accessing the file. Make sure the URL is direct and publicly accessible, without requiring additional confirmation steps.
Please refer to the [troubleshooting guide](/support/troubleshooting) for more information on how to resolve this issue.
Identification jobs support up to 50 voiceprints.
The API supports files of up to 1GiB for diarization and identification jobs, and up to 100MiB for voiceprint jobs.
The maximum duration of an audio file for diarization and speaker identification is 24 hours.
For voiceprints, the maximum is 30 seconds.
Please check the status of your job using the [get job](/api-reference/get-job) API endpoint.
If the job is still in progress, please wait for it to complete.
If the job is completed, but you haven't received the webhook request, please check your webhook configuration and make sure that the webhook URL is correct. For more information, see [webhooks](/webhooks).
pyannoteAI supports webhook signatures, which can be used to verify the authenticity of webhook requests. See [verifying webhooks](/webhooks/verifying-webhooks) for more information.
Most major audio formats are supported, including mp3, wav, m4a, ogg, flac, among others.
We recommend the use of compressed audio to make sure the file size is under the 1GiB limit.
No, the speaker diarization model used in the API is an improved version of the pyannote open-source model. It is faster and more accurate.
Visit our [Trust Center](https://trust.pyannote.ai) for details on security practices, compliance certifications, and data handling. For data retention policies, see [Data retention](/data-retention).
See [rate limits](/ratelimits).
Please first check the documentation and the [troubleshooting section](/support/troubleshooting) for common issues.
If you still have questions, please contact us at [support@pyannote.ai](mailto:support@pyannote.ai).
# Status
Source: https://docs.pyannote.ai/support/status
Visit our status page to view the current status of our services, any ongoing issues, and upcoming scheduled maintenance.
[Status page](https://status.pyannote.ai)
# Troubleshooting
Source: https://docs.pyannote.ai/support/troubleshooting
Common issues and how to solve them.
This troubleshooting guide is designed to help you resolve the most common issues users face when working with pyannoteAI.
## 'Could not load audio' Error
**Description**: This error occurs when pyannoteAI is unable to access an audio file. This issue usually stems from an incorrect or inaccessible file URL.
### Possible Causes and Solutions
* **Invalid or Indirect URL**: Ensure that the audio file URL is a direct link and publicly accessible. Some cloud storage services provide indirect URLs that require authentication or a confirmation page before downloading. Make sure the URL points directly to the file itself without requiring any additional steps.
* **URL Permissions**: The URL should not require login credentials or any kind of permissions that prevent pyannoteAI from accessing the audio file. Double-check that the file is publicly available, and test the link in an incognito browser window to confirm accessibility.
* **URL Expiry**: If you're using temporary URLs (e.g., generated by cloud storage providers), ensure that the URL is valid for the entire duration of the processing job. Expired links will cause this error.
* **File Size Limit**: While the "Could not load audio" error is usually related to accessibility, it can also occur if the file size is too large. Ensure the file size is within the recommended limits to avoid any loading issues.
## Rate limited (429 Too Many Requests)
See [rate limits](/ratelimits).
## Errors creating the job
* **Invalid request**: Check the body for missing or incorrect parameters. Ensure the request body is correctly formatted and all required fields are included.
* **file\_not\_audio**: The provided file is not in an accepted audio format. Ensure the URL points to an audio file and not a cloud storage provider like Dropbox, Box, Google Drive, or similar services. Direct link is required. For files stored in object storage (e.g., S3), make sure the URL is signed with an expiration to grant temporary access.
* **file\_too\_large**: The file size exceeds the allowed limit. The maximum size is 1GiB for **identify** and **diarization** jobs, and 100MiB for **voiceprint** jobs. Please reduce the file size before uploading.
* **http\_error**: The URL returned a non-2xx status code. Verify the URL is correct and accessible.
* **ssl\_error**: The SSL certificate for the URL is invalid. Check the certificate validity or use an HTTPS-compliant URL.
* **too\_many\_redirects**: The URL is redirecting more than 2 times. Simplify the URL or reduce the number of redirects.
* **connection\_failed**: The server hosting the file is not accessible. Confirm that the URL is correct, and the server is online and reachable.
* **http\_timeout**: The URL took more than 10 seconds to respond. Please ensure the server hosting the file is properly configured and responds within the allowed time.
* **empty\_response**: The URL returned an empty response. Ensure the file is properly uploaded and accessible via the URL.
* **unknown\_error**: An unknown error occurred downloading the audio. Please contact support for assistance.
## Possible errors running the job
Once a job is created, there are a few possible errors that can occur during processing:
* **no\_speech\_detected**: No speech was detected in the audio file.
* **invalid\_voiceprint**: The provided voiceprint is invalid (only applicable to `identify` jobs).
Please first check the documentation and the [faq section](/support/faqs) for answers to common questions.
If you still have questions, please contact us at [support@pyannote.ai](mailto:support@pyannote.ai).
# How to use the pyannoteAI API playground
Source: https://docs.pyannote.ai/tutorials/api-playground
Learn how to use the pyannoteAI API playground to test speaker diarization, speaker identification, and speech-to-text orchestration without writing code.
The API playground lets you test pyannoteAI directly from the dashboard before writing code.
You can run diarization, speaker identification, and speech-to-text (STT) orchestration workflows in one place, then move to API calls to integrate pyannoteAI into your application.
![API playground empty state]()
## Run a job in the playground
In the dashboard, open [**Playground**](https://dashboard.pyannote.ai/playground) and select the mode you want to test:
* **Diarization**: identify who spoke when
* **Identification**: match speakers against voiceprints
Configure your job in the right panel:
* **Audio file**: upload a file or add a URL to a file you want to process
* **Model**: choose the diarization model
* **Transcription model**: select a transcription model when you want STT orchestration (or keep it off for diarization only)
* **Speaker controls**: optionally set `numSpeakers`, or use `minSpeakers` and `maxSpeakers`
* **Additional settings**: enable confidence outputs or exclusive diarization when needed
Click **Submit job** and review the output directly in the playground UI.
Once the job is complete, you can view the result in the interactive UI.
To download the output, click **Download** and choose the format you want.
![API playground results view]()
## Credit usage and limitations
* You receive free credits at signup, which can be used to run playground jobs.
* Playground jobs consume your credits exactly like normal API requests, based on your plan.
* Usage from the playground appears in your normal **Usage** and **Billing** pages.
* The playground has stricter rate limits than direct API usage.
* The playground is best for interactive testing; use direct API integration for higher-volume automation.
## Next steps
Once you're ready to integrate pyannoteAI into your application, you can follow these tutorials to get started with the API:
* [How to diarize an audio file](/tutorials/how-to-diarize-audio)
* [Identification with voiceprints](/tutorials/identification-with-voiceprints)
* [Speech-to-text with diarization](/tutorials/speech-to-text-diarization)
# Confidence scores
Source: https://docs.pyannote.ai/tutorials/confidence-scores
Understanding and using confidence scores for diarization quality assessment and human-in-the-loop correction.
Confidence scores provide a measure of the certainty of the model in its predictions. These scores range from 0 to 100, with higher values indicating greater confidence.
## Types of confidence scores
There are three types of confidence scores available:
### 1. Sample-level confidence scores
Sample-level confidence scores provide granular confidence values at regular intervals throughout the audio.
To include sample-level confidence scores in your diarization or identification results, add `"confidence": true` to your request body. When enabled, the job output will include:
* The `confidence` object containing:
* `score`, an array with the confidence score for each sample
* `resolution`, indicating the time interval in seconds between confidence score samples
For example, if the `resolution` is `0.02`, it means each confidence score represents a 20-millisecond interval in the audio.
### 2. Turn-level confidence scores
Turn-level confidence scores provide confidence values for each diarization segment (turn), making it easier to assess the quality of specific speaker assignments.
To include turn-level confidence scores in your results, add `"turnLevelConfidence": true` to your request body. When enabled, the job output will include a `"confidence"` object for each diarization segment. The object contains each speaker as the key and a number from 0-100 indicating the confidence score for each speaker assignment.
### 3. Identification confidence scores
Identification confidence scores are specific to speaker identification tasks and show how well each voiceprint matches each speaker segment. These scores are included automatically when using the [identify endpoint](/api-reference/identify).
The identification output includes a `voiceprints` array where each speaker has a `confidence` object containing the confidence scores for each voiceprint label:
```json theme={null}
{
"voiceprints": [
{
"speaker": "SPEAKER_00",
"match": "John Doe",
"confidence": {
"John Doe": 86,
"Jane Smith": 12
}
}
]
}
```
Identification confidence scores are different from diarization confidence scores. They measure how well a voiceprint matches a speaker, not how certain the diarization model is about the speaker assignment.
## Enabling confidence scores
By default, confidence scores are not included in the output. You must explicitly enable them in your request:
```json theme={null}
{
"url": "https://example.com/audio.wav",
"confidence": true,
"turnLevelConfidence": true
}
```
For identification jobs, confidence scores are included automatically in the `voiceprints` section of the output.
## Using confidence scores
### Quality assessment
Confidence scores can be particularly useful for:
* Identifying segments where the model is less certain, which may benefit from manual review
* Filtering results based on a confidence threshold for higher accuracy
* Analyzing the overall reliability of the diarization or identification for a given audio file
### Human-in-the-loop correction
Confidence scores enable efficient human-in-the-loop workflows by highlighting segments that need attention:
Identify segments with confidence scores below your threshold (e.g., \< 70) for manual review
Focus human review time on the most uncertain segments rather than reviewing entire transcripts
Use confidence scores as a quality metric to ensure diarization meets your accuracy requirements
### Example: Human review workflow with turn-level confidence
Here's how you might implement a human-in-the-loop correction workflow:
```python confidence_review.py theme={null}
def prioritize_for_review(diarization_result, confidence_threshold=70):
low_confidence_segments = []
for index, segment in enumerate(diarization_result["segments"]):
confidence = segment["confidence"]
speaker_confidence = confidence[segment["speaker"]]
if speaker_confidence < confidence_threshold:
low_confidence_segments.append({
"segment_index": index,
"start_time": segment["start"],
"end_time": segment["end"],
"speaker": segment["speaker"],
"confidence": speaker_confidence,
"audio_url": generate_segment_audio_url(segment)
})
# Sort by lowest confidence first
return sorted(low_confidence_segments, key=lambda x: x["confidence"])
# Generate review queue for human correction
review_queue = prioritize_for_review(diarization_result, 75)
print(f"Found {len(review_queue)} segments needing review")
```
```typescript confidence_review.ts theme={null}
function prioritizeForReview(diarizationResult: any, confidenceThreshold = 70) {
const lowConfidenceSegments = [];
diarizationResult.segments.forEach((segment: any, index: number) => {
const confidence = segment.confidence;
const speakerConfidence = confidence[segment.speaker];
if (speakerConfidence < confidenceThreshold) {
lowConfidenceSegments.push({
segmentIndex: index,
startTime: segment.start,
endTime: segment.end,
speaker: segment.speaker,
confidence: speakerConfidence,
audioUrl: generateSegmentAudioUrl(segment)
});
}
});
// Sort by lowest confidence first
return lowConfidenceSegments.sort((a, b) => a.confidence - b.confidence);
}
// Generate review queue for human correction
const reviewQueue = prioritizeForReview(diarizationResult, 75);
console.log(`Found ${reviewQueue.length} segments needing review`);
```
### Example: Identification confidence filtering
For identification tasks, you can filter results based on identification confidence:
```python identification_filter.py theme={null}
def filter_by_identification_confidence(identification_result, threshold=50):
filtered_voiceprints = []
for vp in identification_result["voiceprints"]:
# Find the highest confidence match
top_match = max(vp["confidence"].items(), key=lambda x: x[1])
if top_match[1] >= threshold:
filtered_voiceprints.append(vp)
return filtered_voiceprints
high_confidence_matches = filter_by_identification_confidence(identification_result, 60)
print(f"Found {len(high_confidence_matches)} high confidence matches")
```
```typescript identification_filter.ts theme={null}
function filterByIdentificationConfidence(identificationResult: any, threshold = 50) {
return identificationResult.voiceprints.filter((vp: any) => {
const topMatch = Object.entries(vp.confidence)
.sort(([,a], [,b]) => (b as number) - (a as number))[0];
return topMatch[1] >= threshold;
});
}
const highConfidenceMatches = filterByIdentificationConfidence(identificationResult, 60);
console.log(`Found ${highConfidenceMatches.length} high confidence matches`);
```
## Best practices
* **Threshold selection**: The optimal confidence threshold depends on your use case. For instance, in critical applications sensitive to false positives, you might want to use a higher threshold.
* **Combine multiple types**: Using different confidence score types together gives you comprehensive insight - turn-level for segment assessment, sample-level for detailed analysis, and identification confidence for voiceprint matching.
* **Performance impact**: Enabling confidence scores may slightly increase processing time and output size. Only enable them when you actually need the confidence information.
## Interpreting confidence scores
Confidence scores range from 0 to 100, with higher values indicating greater confidence in the model's predictions. Use these scores to identify segments that may need human review or verification based on your specific accuracy requirements.
For identification confidence scores, consider using the `matching.threshold` parameter in your identification request to automatically filter out low-confidence matches.
By leveraging confidence scores effectively, you can build robust diarization and identification workflows that balance automation with human oversight, ensuring high-quality results while minimizing manual effort.
# How to merge Diarization and STT results
Source: https://docs.pyannote.ai/tutorials/diarization-asr-merge
Learn how to combine diarization results with automatic speech recognition to get transcribed speaker segments.
Use our hosted open-source STT models with specialized reconciliation to obtain speaker-attributed transcripts.
## Prerequisites
Use this tutorial only if you have your own transcripts from another STT service (like OpenAI Whisper, Google Speech-to-Text, etc.) that you want to combine with diarization results.
* Diarization results from pyannoteAI
* Transcript segments from your chosen ASR service
## Step 1: Get diarization segments
First, get diarization segments from a diarization job (see [how to diarize](/tutorials/how-to-diarize-audio) and [Get job](/api-reference/get-job)).
Set the `exclusive` parameter to `true` when requesting diarization for speaker-attributed transcripts.
* This removes overlapping speech, ensuring each segment contains exactly one speaker, which makes it easier to align with STT/ASR results that don't normally work well with overlapping speech.
* **Note**: Exclusive diarization results are provided in the `exclusiveDiarization` field of the job output, alongside the regular diarization results.
Here is an example of some diarization segments:
```json Example diarization segments theme={null}
[
{
"start": 0.5,
"end": 5.2,
"speaker": "SPEAKER_00"
},
{
"start": 5.2,
"end": 7.8,
"speaker": "SPEAKER_01"
},
{
"start": 8.1,
"end": 12.4,
"speaker": "SPEAKER_01"
}
]
```
Note that the segments contain `start` and `end` timestamps in seconds along with speaker labels.
## Step 2: Get transcript segments with timestamps
Get the transcript segments with timestamps based on the same audio with your chosen ASR service. Here is an example of OpenAI `gpt-4o-transcribe` and `whisper-1` API transcript output, with segment timestamps:
```json Example OpenAI transcript segments theme={null}
{
"task": "transcribe",
"duration": 42.7,
"text": "Agent: Thanks for calling OpenAI support.\nCustomer: Hi, I need help with diarization.",
"segments": [
{
"type": "transcript.text.segment",
"id": "seg_001",
"start": 0.0,
"end": 5.2,
"text": "Thanks for calling OpenAI support."
},
{
"type": "transcript.text.segment",
"id": "seg_002",
"start": 5.2,
"end": 12.8,
"text": "Hi, I need help with diarization."
}
],
"usage": {
"type": "duration",
"seconds": 43
}
}
```
Here, the `segments` array contains `start` and `end` timestamps in seconds along with the transcribed text.
## Step 3: Merge results
Combine the diarization segments with the ASR transcript segments by aligning them based on their timestamps. You can use the following segment-level adaptation of WhisperX's current [`assign_word_speakers` logic in diarize.py](https://github.com/m-bain/whisperX/blob/main/whisperx/diarize.py):
```python merge_diarization_asr.py theme={null}
# Assuming diarization_segments is a list of dictionaries from Step 1
# Example: diarization_segments = [{"start": 0.5, "end": 3.2, "speaker": "SPEAKER_00"}, ...]
diarization_segments = sorted(diarization_segments, key=lambda x: x["start"])
# Assuming transcript_result is a dictionary from Step 2
# Example: transcript_result = {"segments": [{"start": 0.0, "end": 5.2, "text": "..."}, ...]}
transcript_segments = transcript_result.get("segments", [])
# Set to True to assign the nearest speaker when there is no overlap.
fill_nearest = False
for seg in transcript_segments:
seg_start = seg.get("start", 0.0)
seg_end = seg.get("end", 0.0)
speaker_overlap: dict[str, float] = {}
for dia in diarization_segments:
intersection = min(dia["end"], seg_end) - max(dia["start"], seg_start)
if intersection <= 0:
continue
speaker = dia["speaker"]
speaker_overlap[speaker] = speaker_overlap.get(speaker, 0.0) + intersection
if speaker_overlap:
seg["speaker"] = max(speaker_overlap.items(), key=lambda x: x[1])[0]
continue
if fill_nearest and diarization_segments:
midpoint = (seg_start + seg_end) / 2
nearest = min(
diarization_segments,
key=lambda x: abs(((x["start"] + x["end"]) / 2) - midpoint),
)
seg["speaker"] = nearest["speaker"]
continue
seg["speaker"] = "UNKNOWN"
```
Resulting merged segments will look something like this:
```json Merged diarization + ASR segments theme={null}
[
{
"start": 0.0,
"end": 5.2,
"text": "Thanks for calling OpenAI support.",
"speaker": "SPEAKER_00"
},
{
"start": 5.2,
"end": 12.8,
"text": "Hi, I need help with diarization.",
"speaker": "SPEAKER_01"
}
]
```
Learn more about [WhisperX on GitHub](https://github.com/m-bain/whisperX/) and [OpenAI Whisper](https://github.com/openai/whisper)
# How to diarize an audio file
Source: https://docs.pyannote.ai/tutorials/how-to-diarize-audio
This tutorial shows how to diarize an audio file using the pyannoteAI API
### Prerequisites
Before you start, you'll need:
* pyannoteAI account with credit or active subscription
* An API key from your dashboard
* A publicly accessible audio file URL
For help creating an account and getting your API key, see the [quickstart guide](/quickstart). For pricing and charging details, see [Billing](/administration/billing).
## 1. Diarize API request
Send a POST request to the diarize endpoint with your audio file URL.
In our example we use a sample audio file hosted on pyannoteAI servers. Its a 79 second recording with two speakers. You may use this url to test the API: `https://files.pyannote.ai/marklex1min.wav`
**The URL must be a direct link to a publicly accessible audio file.** Make sure the URL points directly to the file (e.g., ends with `.wav`, `.mp3`, etc.) and is accessible without authentication.
Typically, you'll use a signed URL from cloud storage such as AWS S3 buckets or other cloud storage services. **We also offer our own upload file solution.** For details on uploading audio files to our servers, see:
* [How to upload an audio file](/tutorials/how-to-upload-files)
```python diarize.py theme={null}
import requests
url = "https://api.pyannote.ai/v1/diarize"
api_key = "YOUR_API_KEY" # In production, use environment variables: os.getenv("PYANNOTE_API_KEY")
headers = {"Authorization": f"Bearer {api_key}", "Content-Type": "application/json"}
data = {"url": "https://files.pyannote.ai/marklex1min.wav"}
response = requests.post(url, headers=headers, json=data)
if response.status_code != 200:
print(f"Error: {response.status_code} - {response.text}")
else:
print(response.json())
```
```bash theme={null}
# Set your API key as an environment variable in production
# export PYANNOTE_API_KEY="your_api_key_here"
curl -X POST "https://api.pyannote.ai/v1/diarize" \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{"url": "https://files.pyannote.ai/marklex1min.wav"}'
```
```typescript diarize.ts theme={null}
const url = "https://api.pyannote.ai/v1/diarize";
const apiKey = "YOUR_API_KEY"; // In production, use environment variables: process.env.PYANNOTE_API_KEY
const headers = {
Authorization: `Bearer ${apiKey}`,
"Content-Type": "application/json",
};
const data = {
url: "https://files.pyannote.ai/marklex1min.wav",
};
const response = await fetch(url, {
method: "POST",
headers,
body: JSON.stringify(data),
});
if (!response.ok) {
console.error(`Error: ${response.status} - ${await response.text()}`);
} else {
console.log(await response.json());
}
```
The response will include a `jobId` that you can use to track the diarization job progress:
```json Example response theme={null}
{
"jobId": "3c8a89a5-dcc6-4edb-a75d-ffd64739674d",
"status": "created"
}
```
***
## 2. Get diarization result
Once you have a `jobId`, you can retrieve the results using either polling or using webhooks:
**Job results are automatically deleted after 24 hours**, for all endpoints.
Make sure to save your results in your own database.
### Polling
Poll the [get job](/api-reference/get-job) endpoint to check job status and retrieve results when complete.
Be cautious of [rate limits](/ratelimits) when polling. Excessive requests can lead to rate
limiting. **In production, we strongly recommend using webhooks instead.**
```python polling.py theme={null}
import time
api_key = "YOUR_API_KEY" # In production, use environment variables: os.getenv("PYANNOTE_API_KEY")
headers = {"Authorization": f"Bearer {api_key}"}
while True:
response = requests.get(
f"https://api.pyannote.ai/v1/jobs/{job_id}", headers=headers
)
if response.status_code != 200:
print(f"Error: {response.status_code} - {response.text}")
break
data = response.json()
status = data["status"]
if status in ["succeeded", "failed", "canceled"]:
if status == "succeeded":
print("Job completed successfully!")
print(data["output"])
else:
print(f"Job {status}")
break
print(f"Job status: {status}, waiting...")
time.sleep(10) # Wait 10 seconds before polling again
```
```bash theme={null}
# Set your API key as an environment variable in production
# export PYANNOTE_API_KEY="your_api_key_here"
# Poll for job status
curl -H "Authorization: Bearer YOUR_API_KEY" \
"https://api.pyannote.ai/v1/jobs/YOUR_JOB_ID"
# Keep polling until status is "succeeded", "failed", or "canceled"
```
```typescript polling.ts theme={null}
const jobId = "YOUR_JOB_ID"; // From the diarize response
const apiKey = "YOUR_API_KEY"; // In production, use environment variables: process.env.PYANNOTE_API_KEY
const headers = {
Authorization: `Bearer ${apiKey}`,
};
async function pollJob() {
while (true) {
const response = await fetch(`https://api.pyannote.ai/v1/jobs/${jobId}`, {
headers,
});
if (!response.ok) {
console.error(`Error: ${response.status} - ${await response.text()}`);
break;
}
const data = await response.json();
const status = data.status;
if (["succeeded", "failed", "canceled"].includes(status)) {
if (status === "succeeded") {
console.log("Job completed successfully!");
console.log(data.output);
} else {
console.log(`Job ${status}`);
}
break;
}
console.log(`Job status: ${status}, waiting...`);
await new Promise((resolve) => setTimeout(resolve, 10000)); // Wait 10 seconds
}
}
pollJob();
```
### Webhook
Specify a webhook URL when creating the diarization job to receive updates automatically when the job reaches a terminal status.
Webhooks are sent for terminal statuses only: `succeeded`, `failed`, and
`canceled`. They are not sent for `pending`, `created`, or `running`.
For `failed` and `canceled` jobs, payloads include `jobId` and `status`
(without `output`).
#### 1. Specify your webhook URL
Add the `webhook` parameter to your diarization request payload.
If you only need status updates (useful for smaller payloads), set
`webhookStatusOnly` to `true` (default is `false`):
```python diarize_with_webhook.py theme={null}
data = {
"url": "https://files.pyannote.ai/marklex1min.wav",
"webhook": "https://your-server.com/webhook"
}
```
```bash theme={null}
-d '{
"url": "https://files.pyannote.ai/marklex1min.wav",
"webhook": "https://your-server.com/webhook"
}'
```
```typescript diarize_with_webhook.ts theme={null}
const data = {
url: "https://files.pyannote.ai/marklex1min.wav",
webhook: "https://your-server.com/webhook"
};
```
#### 2. Create server exposing webhook endpoint
Here we show a simple example of how to expose a server that accepts the webhook POST requests. You can use any web framework of your choice.
```python webhook.py theme={null}
from flask import Flask, request, jsonify
app = Flask(__name__)
@app.route('/webhook', methods=['POST'])
def handle_webhook():
data = request.json
status = data.get('status')
if status == 'succeeded':
print("Diarization completed!")
print("Job ID:", data['jobId'])
if 'output' in data:
print("Results:", data['output'])
if status == 'failed':
print("Job failed.")
if status == 'canceled':
print("Job canceled.")
return jsonify({'status': 'received'}), 200
if __name__ == '__main__':
app.run(port=5000)
```
```typescript webhook.ts theme={null}
import express from 'express';
const app = express();
app.use(express.json());
app.post('/webhook', (req, res) => {
const data = req.body;
const status = data.status;
if (status === 'succeeded') {
console.log('Diarization completed!');
console.log('Job ID:', data.jobId);
if (data.output) {
console.log('Results:', data.output);
}
}
if (status === 'failed') {
console.log('Job failed.');
}
if (status === 'canceled') {
console.log('Job canceled.');
}
res.json({ status: 'received' });
});
app.listen(5000, () => {
console.log('Server running on port 5000');
});
```
You can also use a tool like [ngrok](https://ngrok.com/) to expose your local server to the internet
for testing webhooks, or use [webhook.site](https://webhook.site/) for quick
testing.
**Learn more about webhooks:**
* [Receiving webhooks](/webhooks/receiving-webhooks) - Learn about webhook payloads, retries, and failure codes
* [Verifying webhooks](/webhooks/verifying-webhooks) - Learn how to verify webhook signatures to ensure requests are from pyannoteAI
## Pricing
Diarization jobs are billed based on the audio duration sent for processing, with a 20-second minimum per successful job. See [Billing](/administration/billing) for details.
# Upload audio files
Source: https://docs.pyannote.ai/tutorials/how-to-upload-files
This tutorial shows how to upload your files to a temporary location to run jobs like diarization
For cases when your audio file is not publicly accessible, you can upload it to the pyannoteAI servers using the Media APIs.
This tutorial shows you how to upload your audio files to a temporary storage location and use it to diarize your audio.
Some notes on using the Media APIs:
* **Temporary**: The audio files you upload are only temporarily stored and will be automatically removed within **48 hours**.
* **Secure**: The media is stored in an isolated location that only you can access with your API token. It is encrypted while in transit with HTTPS. Access to the production system is controlled to prevent unauthorized use.
## 1. Create a storage location and get a pre-signed PUT URL
First, you need to create a temporary storage location by sending a value for the `url` parameter. This endpoint returns a pre-signed PUT URL that you can use to upload your audio file.
The url should be in the form `media://object-key` where the `object-key` can be any alpha-numeric string you choose to identify your file. The object-key is unique to your team, meaning no other team can access files stored under your object keys.
For example:
* `media://my-audio-file-123`
* `media://meetings/2024-01-15/call.wav`
* `media://customer_conversation_20240115`
You'll use this key to refer to your uploaded file in subsequent API requests (e.g. diarization).
```typescript uploadAudio.ts theme={null}
import axios from 'axios';
import fs from 'fs';
async function uploadAudioFile(inputPath: string, objectKey: string, apiKey: string): Promise {
try {
// Create the pre-signed PUT URL
const response = await axios.post(
'https://api.pyannote.ai/v1/media/input',
{ url: `media://${objectKey}` }, // Replace with your desired object-key
{
headers: {
'Authorization': `Bearer ${apiKey}`, // In production, use environment variables: process.env.PYANNOTE_API_KEY
'Content-Type': 'application/json'
}
}
);
const presignedUrl = response.data.url;
// Upload local file to the pre-signed URL
console.log(`Uploading ${inputPath} to ${presignedUrl}`);
const fileData = fs.readFileSync(inputPath);
await axios.put(presignedUrl, fileData, {
headers: {
'Content-Type': 'application/octet-stream'
}
});
console.log('File uploaded successfully!');
} catch (error) {
console.error('Error uploading file:', error);
throw error;
}
}
// Usage
// uploadAudioFile('./path/to/your/audio.wav', 'my-meeting-recording', 'YOUR_API_KEY');
```
```bash theme={null}
# Set your API key as an environment variable in production
# export PYANNOTE_API_KEY="your_api_key_here"
# Define your media object key
OBJECT_KEY="my-meeting-recording"
# Create pre-signed PUT URL
curl -X POST "https://api.pyannote.ai/v1/media/input" \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d "{\"url\": \"media://${OBJECT_KEY}\"}"
# Upload file to the returned pre-signed URL
curl -X PUT "PRESIGNED_URL_FROM_RESPONSE" \
-H "Content-Type: application/octet-stream" \
--data-binary "@/path/to/your/audio.wav"
```
```python upload_audio.py theme={null}
import requests
# Define your media object key
object_key = "my-meeting-recording" # Replace with your desired object-key
# Create the pre-signed PUT URL.
api_key = "YOUR_API_KEY" # In production, use environment variables: os.getenv("PYANNOTE_API_KEY")
response = requests.post(
"https://api.pyannote.ai/v1/media/input",
json={"url": f"media://{object_key}"},
headers={
"Authorization": f"Bearer {api_key}",
"Content-Type": "application/json"
}
)
response.raise_for_status()
data = response.json()
presigned_url = data["url"]
# Upload local file to the pre-signed URL.
print("Uploading {0} to {1}".format(input_path, presigned_url))
with open(input_path, "rb") as input_file:
# Upload your local audio file.
requests.put(presigned_url, data=input_file)
```
## 2. Diarize your uploaded audio file
After you have uploaded your audio file, you can use it in e.g. a diarization job by providing the media key you created earlier as `URL` in the input.
```typescript createJob.ts theme={null}
import axios from 'axios';
async function createDiarizationJob(objectKey: string, apiKey: string) {
try {
const body = {
url: `media://${objectKey}` // Use the same object-key you used when creating the pre-signed URL
};
const response = await axios.post(
'https://api.pyannote.ai/v1/diarize',
body,
{
headers: {
'Authorization': `Bearer ${apiKey}`, // In production, use environment variables: process.env.PYANNOTE_API_KEY
'Content-Type': 'application/json'
}
}
);
console.log(response.data);
return response.data;
} catch (error) {
console.error('Error creating diarization job:', error);
throw error;
}
}
// Usage
// createDiarizationJob('my-meeting-recording', 'YOUR_API_KEY');
```
```bash theme={null}
# Set your API key as an environment variable in production
# export PYANNOTE_API_KEY="your_api_key_here"
# Define your media object key (same as used in upload)
OBJECT_KEY="my-meeting-recording"
curl -X POST "https://api.pyannote.ai/v1/diarize" \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d "{\"url\": \"media://${OBJECT_KEY}\"}"
```
```python create_job.py theme={null}
import requests
# Define your media object key (same as used in upload)
object_key = "my-meeting-recording" # Use the same object-key you used when creating the pre-signed URL
# Replace the input value with your temporary storage location.
body = {
"url" : f"media://{object_key}",
}
url = "https://api.pyannote.ai/v1/diarize"
api_key = "YOUR_API_KEY" # In production, use environment variables: os.getenv("PYANNOTE_API_KEY")
headers = {
"Authorization": f"Bearer {api_key}",
"Content-Type": "application/json"
}
response = requests.post(url, json=body, headers=headers)
response.raise_for_status()
print(response.json())
```
Great! You have successfully uploaded your audio file and used it to create a diarization job.
See the [diarization tutorial](/tutorials/how-to-diarize-audio) for more details on how to process the diarization results.
## API Reference
* [Upload files](/api-reference/upload-media)
* [Get a job](/api-reference/get-job)
* [Diarize an audio file](/api-reference/diarize)
# How to identify speakers with voiceprints
Source: https://docs.pyannote.ai/tutorials/identification-with-voiceprints
This tutorial shows how to create voiceprints and identify speakers using the pyannoteAI API
Speaker identification is the process of determining who is speaking in an audio file by comparing their voice characteristics against known voiceprints. Unlike diarization which only separates speakers into generic labels (`SPEAKER_00`, `SPEAKER_01`, etc.), identification assigns specific identities to speakers.
### What are voiceprints?
A voiceprint is a unique digital representation of a person's voice characteristics, similar to a fingerprint but for voice. It captures the distinctive features of how someone speaks, allowing the system to recognize that person in future audio recordings.
**Voiceprints are for identification only** - they do not improve the accuracy of diarization. Diarization separates speakers, while identification assigns names/labels to those speakers.
### Voiceprint requirements
* **One voiceprint per speaker**: Create only one voiceprint for each person.
* **Single speaker only**: The recording must contain only the target speaker's voice with no overlapping speakers.
* **Maximum duration**: Audio samples must be at most 30 seconds long for creating voiceprints.
* **Consistent speaking style**: The voiceprint should capture the person's normal speaking voice.
* **Language**: Our models are language agnostic, so voiceprints can be created in any spoken language.
### Prerequisites
Before you start, you'll need:
* pyannoteAI account with credit or active subscription
* An API key
* An audio recording of a single speaker for the voiceprint creation
* An audio recording with multiple speakers for diarization + identification
For help creating an account and getting your API key, see the [quickstart guide](/quickstart). For pricing and charging details, see [Billing](/administration/billing).
## 1. Create a voiceprint
First, create a voiceprint for each speaker you want to identify. This is a one-time process for each person.
Send a POST request to the [voiceprint endpoint](/api-reference/voiceprint) with an audio file containing the speaker's voice.
```python create_voiceprint.py theme={null}
import requests
url = "https://api.pyannote.ai/v1/voiceprint"
api_key = "YOUR_API_KEY" # In production, use environment variables: os.getenv("PYANNOTE_API_KEY")
headers = {"Authorization": f"Bearer {api_key}", "Content-Type": "application/json"}
data = {"url": "https://example.com/speaker-voice-sample.wav"}
response = requests.post(url, headers=headers, json=data)
if response.status_code != 200:
print(f"Error: {response.status_code} - {response.text}")
else:
print(response.json())
```
```bash theme={null}
# Set your API key as an environment variable in production
# export PYANNOTE_API_KEY="your_api_key_here"
curl -X POST "https://api.pyannote.ai/v1/voiceprint" \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{"url": "https://example.com/speaker-voice-sample.wav"}'
```
```typescript create_voiceprint.ts theme={null}
const url = "https://api.pyannote.ai/v1/voiceprint";
const apiKey = "YOUR_API_KEY"; // In production, use environment variables: process.env.PYANNOTE_API_KEY
const headers = {
Authorization: `Bearer ${apiKey}`,
"Content-Type": "application/json",
};
const data = {
url: "https://example.com/speaker-voice-sample.wav",
};
const response = await fetch(url, {
method: "POST",
headers,
body: JSON.stringify(data),
});
if (!response.ok) {
console.error(`Error: ${response.status} - ${await response.text()}`);
} else {
console.log(await response.json());
}
```
The response will include a `jobId` to track the voiceprint creation:
```json Example response theme={null}
{
"jobId": "3c8a89a5-dcc6-4edb-a75d-ffd64739674d",
"status": "created"
}
```
### Get voiceprint results
To retrieve the voiceprint results, use the same polling or webhook approach described in the [How to diarize an audio file](/tutorials/how-to-diarize-audio) tutorial. The process works identically for voiceprint jobs.
**Save voiceprints to your own data storage**
* Job outputs (including voiceprints) are automatically **deleted after 24 hours**.
* Voiceprints are reusable, so store them securely for future identification requests.
```json example job voiceprint output theme={null}
{
"jobId": "3c8a89a5-dcc6-4edb-a75d-ffd64739674d",
"status": "succeeded",
"createdAt": "2024-02-20T12:00:00Z",
"updatedAt": "2024-02-20T12:00:00Z",
"output": {
"voiceprint": "U29tZVZvaWNlUHJpbnREYXRhMQ=="
}
}
```
***
## 2. Identify speakers in audio
Now that you have a voiceprint, you can identify a speaker in new audio recordings.
Send a POST request to the [identify endpoint](/api-reference/identify) with the audio file URL and the voiceprints you want to match against.
```python identify_speakers.py theme={null}
import requests
url = "https://api.pyannote.ai/v1/identify"
api_key = "YOUR_API_KEY"
headers = {"Authorization": f"Bearer {api_key}", "Content-Type": "application/json"}
data = {
"url": "https://example.com/meeting-audio.wav",
"voiceprints": [
{
"label": "John Doe", # The speaker label you want to assign
"voiceprint": "U29tZVZvaWNlUHJpbnREYXRhMQ==" # Replace with actual voiceprint
},
# Add more voiceprints as needed
],
# Optional matching parameters
"matching": {
"threshold": 50, # Only match if confidence is 50% or higher
"exclusive": True # Prevent multiple speakers matching same voiceprint
}
}
response = requests.post(url, headers=headers, json=data)
if response.status_code != 200:
print(f"Error: {response.status_code} - {response.text}")
else:
print(response.json())
```
```bash theme={null}
curl -X POST "https://api.pyannote.ai/v1/identify" \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"url": "https://example.com/meeting-audio.wav",
"voiceprints": [
{
"label": "John Doe",
"voiceprint": "U29tZVZvaWNlUHJpbnREYXRhMQ=="
}
],
"matching": {
"threshold": 50,
"exclusive": true
}
}'
```
```typescript identify_speakers.ts theme={null}
const url = "https://api.pyannote.ai/v1/identify";
const apiKey = "YOUR_API_KEY";
const headers = {
Authorization: `Bearer ${apiKey}`,
"Content-Type": "application/json",
};
const data = {
url: "https://example.com/meeting-audio.wav",
voiceprints: [
{
label: "John Doe", // The speaker label you want to assign
voiceprint: "U29tZVZvaWNlUHJpbnREYXRhMQ==", // Replace with actual voiceprint
},
// Add more voiceprints as needed
],
// Optional matching parameters
matching: {
threshold: 50, // Only match if confidence is 50% or higher
exclusive: true, // Prevent multiple speakers matching same voiceprint
},
};
const response = await fetch(url, {
method: "POST",
headers,
body: JSON.stringify(data),
});
if (!response.ok) {
console.error(`Error: ${response.status} - ${await response.text()}`);
} else {
console.log(await response.json());
}
```
The response will include a `jobId` for tracking the identification job:
```json Example response theme={null}
{
"jobId": "4d9b9ab6-edd7-5feca-b86e-gee75840775e",
"status": "created"
}
```
**Multiple voiceprints**: You can add multiple voiceprints for different people in the same request. Each voiceprint must have a unique label. The system will attempt to match all provided voiceprints against the audio.
**Voiceprint selection**: Voiceprints may match to speakers even when the person isn't actually in the audio. Be cautious about including voiceprints of people who may not be present. Review confidence scores carefully and set appropriate thresholds when unsure about speaker presence.
***
## 3. Get identification results
To retrieve the identification results, use the same polling or webhook approach described in the [How to diarize an audio file](/tutorials/how-to-diarize-audio) tutorial. The process works identically for identification jobs.
```json Example identification output theme={null}
{
"jobId": "4d9b9ab6-edd7-5feca-b86e-gee75840775e",
"status": "succeeded",
"createdAt": "2025-11-06T09:07:49.932Z",
"updatedAt": "2025-11-06T09:07:53.229Z",
"output": {
"diarization": [
{
"speaker": "SPEAKER_00",
"start": 3.005,
"end": 5.945
},
{
"speaker": "SPEAKER_01",
"start": 6.345,
"end": 9.565
},
...
],
"identification": [
{
"speaker": "John Doe",
"start": 3.005,
"end": 5.945,
"diarizationSpeaker": "SPEAKER_00",
"match": "John Doe"
},
{
"speaker": "SPEAKER_01",
"start": 6.345,
"end": 9.565,
"diarizationSpeaker": "SPEAKER_01",
"match": null
},
...
],
"voiceprints": [
{
"speaker": "SPEAKER_00",
"match": "John Doe",
"confidence": {
"John Doe": 86
}
},
{
"speaker": "SPEAKER_01",
"match": null,
"confidence": {
"John Doe": 16
}
}
]
}
}
```
Learn more details about each parameter of the identification output in the [identification schema reference](/api-reference/schemas/identifyschema).
***
## Understanding the Results
### Diarization vs Identification
* **Diarization**: Separates audio into speaker segments with generic labels (SPEAKER\_00, SPEAKER\_01, etc.)
* **Identification**: Matches those segments to known voiceprints with specific labels (John Doe, Jane Smith, etc.)
### Confidence scores
The confidence scores show how well each voiceprint matches each speaker segment:
* Higher scores indicate better matches
* Use the `threshold` parameter to filter out low-confidence matches
* Consider the context when interpreting confidence scores
### Matching options
* **`matching.threshold`**: Minimum confidence score required for a match (0-100, default: `0`). Set higher values (50-70) for more strict matching, lower values for more lenient matching.
* **`matching.exclusive`**: Prevent multiple speakers from matching the same voiceprint (default: `true`). Set to `false` if you want multiple speakers to potentially match the same voiceprint.
## Pricing
`/identify` jobs are billed based on the audio duration sent for processing, with a 20-second minimum per successful job, and `/voiceprint` jobs are billed per voiceprint created. See [Billing](/administration/billing) for details.
# Configuring the number of speakers
Source: https://docs.pyannote.ai/tutorials/speaker-configuration
Learn how to configure speaker detection and handle overlapping speech in diarization tasks.
When working with audio diarization, you often have prior knowledge about the expected number of speakers or specific requirements for how overlapping speech should be handled. This tutorial covers the key configuration options available in pyannoteAI for speaker detection and exclusive diarization.
## Number of speakers
By default, pyannoteAI automatically detects the number of speakers in your audio with no upper limit. However, you can improve accuracy and performance by providing speaker count constraints when you have this information.
### Exact speaker count
When you know the exact number of speakers, use `numSpeakers` for better results. This is common for:
* Phone conversations (2 speakers)
* Interviews (2 speakers)
* Panel discussions with known participants
* Meeting recordings with known attendees
Setting `numSpeakers` typically results in better overall diarization performance since the model can optimize for a specific speaker count.
### Speaker count ranges
When the exact number is unknown but you have reasonable bounds, use `minSpeakers` and `maxSpeakers`:
* `minSpeakers`: Minimum number of speakers to detect
* `maxSpeakers`: Maximum number of speakers to detect
This is useful when there are optional participants in your recordings, such as:
* Conference calls with variable attendance
* Classroom recordings where some students may be absent
* Broadcast content with variable guest counts
### Parameter rules and constraints
* `numSpeakers` cannot be used together with `minSpeakers` or `maxSpeakers`
* If both `minSpeakers` and `maxSpeakers` are set, `minSpeakers` must be ≤ `maxSpeakers`
* Setting `numSpeakers=2` is equivalent to `minSpeakers=2` and `maxSpeakers=2`
## Exclusive diarization
By default, diarization results may include overlapping speech segments where multiple speakers are talking simultaneously. While this provides true accurate diarization, some applications require non-overlapping speaker turns.
Enable exclusive diarization by setting `"exclusive": true`. This provides:
* **Non-overlapping segments**: Each time period is assigned to exactly one speaker
* **Easier integration**: Simpler to combine with speech-to-text or other processing
Exclusive diarization results are provided in the `exclusiveDiarization` field of the job output, alongside the regular diarization results.
### When to use exclusive diarization
Exclusive diarization is particularly useful for:
* **Transcription workflows**: Easier to align with ASR output
* **Meeting minutes**: Cleaner, more readable summaries
* **Content analysis**: Simpler speaker turn analysis
* **Legal proceedings**: Clear attribution of speech segments
## Best practices
### Start with automatic detection
When unsure about speaker count, begin with automatic detection (no parameters) to understand your audio content, then refine with constraints in subsequent processing.
### Use exact counts when possible
If you have reliable information about speaker count, always use `numSpeakers` for optimal performance.
### Consider your use case
* **Analysis and research**: Use regular diarization to capture natural speech patterns
* **Transcription and documentation**: Consider exclusive diarization for cleaner output
* **Unknown number of speakers**: Use speaker count ranges to handle variability
### Test with your data
Different audio quality and recording conditions may affect how well the constraints work. Test with representative samples from your specific use case.
# STT Orchestration: Speech-to-text with speaker diarization
Source: https://docs.pyannote.ai/tutorials/speech-to-text-diarization
Learn how to get transcribed speaker segments with automatic speech recognition and diarization in a single API call.
New
Enable speaker-attributed transcription in your diarization jobs. Our API runs the `precision-2` diarization model and an STT model (Nvidia's Parakeet-tdt-0.6b-v3 or OpenAI's whisper-large-v3-turbo), then applies specialized STT reconciliation logic to match transcript segments and speakers with highly accurate results.
If you already have transcripts from another service and want to combine them with our diarization results, see [how to merge diarization and STT results](/tutorials/diarization-asr-merge).
### Prerequisites
Before you start, you'll need a pyannoteAI account with credits or an active subscription, and a pyannoteAI API key. For help creating an account and getting your API key, see the [quickstart guide](/quickstart). For pricing and charging details, see [Billing](/administration/billing).
Transcription is available as an add-on feature for diarization jobs. Please note the following constraints:
* **Diarization model**: Only available with the `precision-2` model
* **No identification**: Cannot be used with speaker identification jobs
* **Supported languages**: A total of 100 languages are supported, although this number may vary depending on the chosen transcription model; for a complete list, refer to the [API Reference](/api-reference/diarize#body-transcription-config)
* **STT models**: [Nvidia Parakeet-tdt-0.6b-v3](https://huggingface.co/nvidia/parakeet-tdt-0.6b-v3) or [OpenAI whisper-large-v3-turbo](https://huggingface.co/dropbox-dash/faster-whisper-large-v3-turbo) models for transcription
## 1. Create diarization job with transcription
Send a POST request to the diarize endpoint with `transcription: true`.
Learn more about the diarize endpoint in the [how to diarize tutorial](/tutorials/how-to-diarize-audio).
```python diarize_with_transcription.py theme={null}
import requests
url = "https://api.pyannote.ai/v1/diarize"
api_key = "YOUR_API_KEY" # In production, use environment variables: os.getenv("PYANNOTE_API_KEY")
headers = {"Authorization": f"Bearer {api_key}", "Content-Type": "application/json"}
data = {
"url": "https://files.pyannote.ai/marklex1min.wav",
"transcription": True
}
response = requests.post(url, headers=headers, json=data)
if response.status_code != 200:
print(f"Error: {response.status_code} - {response.text}")
else:
print(response.json())
```
```bash theme={null}
# Set your API key as an environment variable in production
# export PYANNOTE_API_KEY="your_api_key_here"
curl -X POST "https://api.pyannote.ai/v1/diarize" \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"url": "https://files.pyannote.ai/marklex1min.wav",
"transcription": true
}'
```
```typescript diarize_with_transcription.ts theme={null}
const url = "https://api.pyannote.ai/v1/diarize";
const apiKey = "YOUR_API_KEY"; // In production, use environment variables: process.env.PYANNOTE_API_KEY
const headers = {
Authorization: `Bearer ${apiKey}`,
"Content-Type": "application/json",
};
const data = {
url: "https://files.pyannote.ai/marklex1min.wav",
transcription: true,
};
const response = await fetch(url, {
method: "POST",
headers,
body: JSON.stringify(data),
});
if (!response.ok) {
console.error(`Error: ${response.status} - ${await response.text()}`);
} else {
console.log(await response.json());
}
```
The response will include a `jobId` that you can use to track the job progress:
```json Example response theme={null}
{
"jobId": "3c8a89a5-dcc6-4edb-a75d-ffd64739674d",
"status": "created"
}
```
To further configure transcriptions see [Additional options](/tutorials/speech-to-text-diarization#5-configure-additional-transcription-options).
***
## 2. Get the speaker attributed transcription results
Once you have a `jobId`, retrieve the results using either polling or webhooks. See [how to get results](/tutorials/how-to-diarize-audio#2-get-diarization-result) for detailed examples.
**Job results are automatically deleted after 24 hours**, for all endpoints. Make sure to save your results in your own database.
When transcription is enabled, the completed job output will include both your standard diarization results (`diarization` object, and `exclusiveDiarization` if enabled) **and** two additional transcription fields in the `output` object:
### Word-level transcription
Individual words with precise timestamps and speaker attribution:
```json wordLevelTranscription theme={null}
{
...,
"wordLevelTranscription": [
{
"start": 0.5,
"end": 0.8,
"text": "Hello",
"speaker": "SPEAKER_00"
},
{
"start": 0.9,
"end": 1.2,
"text": "everyone",
"speaker": "SPEAKER_00"
}
]
}
```
### Turn-level transcription
Complete speaker turns with full text, ideal for creating readable transcripts:
```json turnLevelTranscription theme={null}
{
...,
"turnLevelTranscription": [
{
"start": 0.5,
"end": 3.2,
"text": "Hello everyone, welcome to the meeting.",
"speaker": "SPEAKER_00"
},
{
"start": 3.5,
"end": 6.8,
"text": "Hi, thanks for having me.",
"speaker": "SPEAKER_01"
}
]
}
```
***
## 3. Practical use cases
### Word-level transcription
Word-level transcription is ideal for applications requiring precise timing:
* **Subtitles and captions**: Generate accurate timestamps for each word to create synchronized subtitles
* **Video editing**: Enable precise word-level navigation for editing tools
* **Detailed analysis**: Analyze speaking patterns, word timing, and more
* **Search and indexing**: Create searchable transcripts with exact word positions
### Turn-level transcription
Turn-level transcription provides complete speaker utterances, making it more suitable for:
* **Meeting notes**: Generate readable transcripts of conversations and meetings
* **Interview transcripts**: Create clean, easy-to-read interview documentation
* **Customer service logs**: Document support calls with speaker-attributed dialogue
* **Content summarization**: Feed into AI summarization tools for generating meeting summaries
***
## 4. Format transcript example
Here's an example of formatting the turn-level transcription into a readable transcript:
```python format_transcript.py theme={null}
def format_transcript(turn_level_transcription):
"""Format turn-level transcription as a readable transcript"""
transcript = []
for turn in turn_level_transcription:
speaker = turn["speaker"]
text = turn["text"]
timestamp = f"{int(turn['start'] // 60)}:{int(turn['start'] % 60):02d}"
transcript.append(f"{speaker} ({timestamp}): {text}")
return "\n\n".join(transcript)
# Example usage
output = response.json()["output"]
print(format_transcript(output["turnLevelTranscription"]))
```
```typescript format_transcript.ts theme={null}
function formatTranscript(turnLevelTranscription: any[]) {
const transcript = turnLevelTranscription.map((turn) => {
const speaker = turn.speaker;
const text = turn.text;
const minutes = Math.floor(turn.start / 60);
const seconds = Math.floor(turn.start % 60).toString().padStart(2, '0');
const timestamp = `${minutes}:${seconds}`;
return `${speaker} (${timestamp}): ${text}`;
});
return transcript.join('\n\n');
}
// Example usage
const output = response.output;
console.log(formatTranscript(output.turnLevelTranscription));
```
This will produce a transcript like:
```
SPEAKER_00 (0:00): Hello everyone, welcome to the meeting.
SPEAKER_01 (0:03): Hi, thanks for having me.
```
***
## 5. Configure additional transcription options
[Nvidia Parakeet-tdt-0.6b-v3](https://huggingface.co/nvidia/parakeet-tdt-0.6b-v3) is the default transcription model, to choose a different one like [OpenAI whisper-large-v3-turbo](https://huggingface.co/dropbox-dash/faster-whisper-large-v3-turbo), explicitly set it using the `transcriptionConfig` object:
```python diarize_with_transcription_whisper.py theme={null}
import requests
url = "https://api.pyannote.ai/v1/diarize"
api_key = "YOUR_API_KEY" # In production, use environment variables: os.getenv("PYANNOTE_API_KEY")
headers = {"Authorization": f"Bearer {api_key}", "Content-Type": "application/json"}
data = {
"url": "https://files.pyannote.ai/marklex1min.wav",
"transcription": True,
"transcriptionConfig": {
"model": "faster-whisper-large-v3-turbo"
}
}
response = requests.post(url, headers=headers, json=data)
if response.status_code != 200:
print(f"Error: {response.status_code} - {response.text}")
else:
print(response.json())
```
```bash theme={null}
# Set your API key as an environment variable in production
# export PYANNOTE_API_KEY="your_api_key_here"
curl -X POST "https://api.pyannote.ai/v1/diarize" \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"url": "https://files.pyannote.ai/marklex1min.wav",
"transcription": true,
"transcriptionConfig": {
"model": "faster-whisper-large-v3-turbo"
}
}'
```
```typescript diarize_with_transcription_whisper.ts theme={null}
const url = "https://api.pyannote.ai/v1/diarize";
const apiKey = "YOUR_API_KEY"; // In production, use environment variables: process.env.PYANNOTE_API_KEY
const headers = {
Authorization: `Bearer ${apiKey}`,
"Content-Type": "application/json",
};
const data = {
url: "https://files.pyannote.ai/marklex1min.wav",
transcription: true,
transcriptionConfig: {
model: "faster-whisper-large-v3-turbo"
}
};
const response = await fetch(url, {
method: "POST",
headers,
body: JSON.stringify(data),
});
if (!response.ok) {
console.error(`Error: ${response.status} - ${await response.text()}`);
} else {
console.log(await response.json());
}
```
For the full list of configuration options, refer to the [API reference](/api-reference/diarize).
***
## Limitations and considerations
**Current limitations and future development:**
* **Diarization model**: Only available with the `precision-2` model. Additional diarization model support is coming later.
* **STT model**: Currently supports [Nvidia Parakeet-tdt-0.6b-v3](https://huggingface.co/nvidia/parakeet-tdt-0.6b-v3) and [OpenAI whisper-large-v3-turbo](https://huggingface.co/dropbox-dash/faster-whisper-large-v3-turbo). We are working on adding support for additional transcription models.
* **Not compatible with identification**: Transcription cannot be used with speaker identification jobs yet.
* **Language support**: Refer to the [API Reference](/api-reference/diarize#body-transcription-config) for the complete list of supported languages.
**Processing time:**
Transcription jobs take longer to process than diarization-only jobs, as they run both diarization and speech recognition models.
## Pricing
When transcription is enabled, `/diarize` jobs are billed at the transcription price (`STT Orchestration`) in addition to diarization pricing, per successful job. See [Billing](/administration/billing) for details.
# Streaming diarized transcription with OpenAI
Source: https://docs.pyannote.ai/tutorials/streaming-diarized-transcription
Learn how to combine pyannoteAI streaming diarization with OpenAI realtime transcription to print speaker-attributed transcripts live.
This tutorial shows how to combine two live streams:
* pyannoteAI streaming diarization tells you **who** is speaking and **when** speaker turns end.
* OpenAI realtime transcription tells you **what** was said.
The key step is deciding which speaker label belongs to each transcript segment as both APIs stream events independently.
## Prerequisites
* A pyannoteAI API key from the [dashboard](https://dashboard.pyannote.ai)
* An OpenAI API key
* Python 3.10+
* Microphone access
Install dependencies:
```bash theme={null}
pip install sounddevice numpy "websockets>=13" "openai[realtime]" python-dotenv requests
```
Create a `.env` file:
```bash theme={null}
PYANNOTEAI_API_KEY=sk_xxx
OPENAI_API_KEY=sk-xxx
```
## How the merge works
pyannote emits `diarization_speaker_start` and `diarization_speaker_end` events. OpenAI emits transcription deltas and completed transcript segments.
The script keeps two pieces of shared state:
* `active`: speakers pyannote currently hears. Live transcript deltas are displayed with latest active speaker.
* `pending`: speaker labels waiting for completed OpenAI transcript segments. When pyannote emits a speaker end event, script commits OpenAI's audio buffer and queues that speaker label.
This line is the handoff point between diarization and transcription:
```python theme={null}
await oai.conn.input_audio_buffer.commit()
```
It tells OpenAI to finalize audio collected during the pyannote speaker turn. When OpenAI later emits `transcription.completed`, the script pops from `pending` and prints completed text with that speaker.
```python theme={null}
sp = convo.pending.popleft() if convo.pending else convo.speaker
```
## Complete script
Save this as `live_diarized_transcription.py`:
```python theme={null}
#!/usr/bin/env python3
import asyncio
import base64
import json
import os
import signal
from collections import deque
import numpy as np
import requests
import sounddevice as sd
import websockets
from dotenv import load_dotenv
from openai import AsyncOpenAI
load_dotenv()
PYANNOTEAI_API_KEY = os.environ["PYANNOTEAI_API_KEY"]
OPENAI_API_KEY = os.environ["OPENAI_API_KEY"]
SAMPLE_RATE = 16_000
OPENAI_RATE = 24_000
CHUNK = 1600
_COLORS = ["\033[32m", "\033[33m", "\033[34m", "\033[35m", "\033[36m", "\033[31m", "\033[37m", "\033[93m"]
_RESET = "\033[0m"
_colors: dict[str, str] = {}
def color(speaker: str) -> str:
_colors.setdefault(speaker, _COLORS[len(_colors) % len(_COLORS)])
return _colors[speaker]
class Conversation:
def __init__(self) -> None:
self.active: list[str] = []
self.pending: deque[str] = deque()
self.uncommitted = False
def started(self, speaker: str) -> None:
if speaker not in self.active:
self.active.append(speaker)
def ended(self, speaker: str) -> None:
if speaker in self.active:
self.active.remove(speaker)
@property
def speaker(self) -> str | None:
return self.active[-1] if self.active else None
def upsample(x: np.ndarray) -> np.ndarray:
n = len(x) * OPENAI_RATE // SAMPLE_RATE
return np.interp(np.linspace(0, len(x) - 1, n), np.arange(len(x)), x).astype(np.float32)
def create_stream() -> tuple[str, str]:
r = requests.post(
"https://api.pyannote.ai/v1/live",
headers={"Authorization": f"Bearer {PYANNOTEAI_API_KEY}"},
)
r.raise_for_status()
data = r.json()
return data["id"], data["url"]
async def pyannote_task(ws, convo: Conversation, oai) -> None:
async for raw in ws:
msg = json.loads(raw)
data = msg.get("data", {})
if msg.get("type") == "diarization_speaker_start":
convo.started(data["speaker"])
elif msg.get("type") == "diarization_speaker_end":
convo.ended(data["speaker"])
if oai.conn is not None and convo.uncommitted:
convo.pending.append(data["speaker"])
convo.uncommitted = False
await oai.conn.input_audio_buffer.commit()
elif msg.get("type") == "error":
print(f"\npyannote error: {msg.get('message')}")
class OpenAI:
conn = None
async def openai_task(client: AsyncOpenAI, convo: Conversation, oai: OpenAI) -> None:
async with client.realtime.connect(extra_query={"intent": "transcription"}) as conn:
await conn.session.update(session={
"type": "transcription",
"audio": {"input": {
"format": {"type": "audio/pcm", "rate": OPENAI_RATE},
"transcription": {"model": "gpt-realtime-whisper", "language": "en"},
"turn_detection": None,
}},
})
oai.conn = conn
partial = ""
async for event in conn:
et = event.type
if et.endswith("transcription_session.created"):
print(f"OpenAI session: {event.session.id}")
elif et.endswith("transcription.delta"):
partial += event.delta
sp = convo.speaker
tag = f"{color(sp)}[{sp}]{_RESET} " if sp else ""
print(f"\r{tag}{partial}\033[K", end="", flush=True)
elif et.endswith("transcription.completed"):
sp = convo.pending.popleft() if convo.pending else convo.speaker
text = event.transcript.strip()
if text:
tag = f"{color(sp)}[{sp}]{_RESET} " if sp else ""
print(f"\r{tag}{text}\033[K")
partial = ""
def microphone(loop, queue: "asyncio.Queue[np.ndarray]") -> sd.InputStream:
def callback(indata, frames, time, status):
try:
loop.call_soon_threadsafe(queue.put_nowait, indata[:, 0].copy())
except asyncio.QueueFull:
pass
stream = sd.InputStream(samplerate=SAMPLE_RATE, channels=1, dtype="float32", blocksize=CHUNK, callback=callback)
stream.start()
return stream
async def pump(queue: "asyncio.Queue[np.ndarray]", ws, convo: Conversation, oai: OpenAI) -> None:
while True:
frame = await queue.get()
await ws.send(frame.astype(" None:
loop = asyncio.get_running_loop()
convo, oai = Conversation(), OpenAI()
queue: "asyncio.Queue[np.ndarray]" = asyncio.Queue(maxsize=50)
stream_id, url = create_stream()
print(f"pyannote stream: {stream_id}")
client = AsyncOpenAI(api_key=OPENAI_API_KEY)
async with websockets.connect(url) as ws:
mic = microphone(loop, queue)
print("\nListening — speak now (Ctrl+C to stop)\n")
stop = asyncio.Event()
loop.add_signal_handler(signal.SIGINT, stop.set)
tasks = [
asyncio.create_task(pyannote_task(ws, convo, oai)),
asyncio.create_task(openai_task(client, convo, oai)),
asyncio.create_task(pump(queue, ws, convo, oai)),
]
await asyncio.wait({*tasks, asyncio.create_task(stop.wait())}, return_when=asyncio.FIRST_COMPLETED)
print("\nStopping...")
mic.stop()
mic.close()
for task in tasks:
task.cancel()
await asyncio.gather(*tasks, return_exceptions=True)
await ws.send(json.dumps({"type": "end_of_stream"}))
if __name__ == "__main__":
asyncio.run(main())
```
Run it:
```bash theme={null}
python live_diarized_transcription.py
```
Speak into your microphone. Partial text updates in place while OpenAI streams deltas. Completed turns print as separate lines with speaker labels like `[SPEAKER_00]`.
Example output:
```bash theme={null}
python live_diarized_transcription.py
pyannote stream: 7d3f4a21-9b6c-4f8a-8f12-3c9d8b6e2a44
Listening — speak now (Ctrl+C to stop)
[SPEAKER_00] Hello
[SPEAKER_00] This is transcribing while diarization is running
[SPEAKER_00] The speaker label stays attached to this turn
[SPEAKER_01] Now another person is speaking, and the live transcript switches speakers.
[SPEAKER_01] It keeps printing completed turns as they arrive.
[SPEAKER_00] And now the first speaker is back.
^C
Stopping...
```
## Important details
### Create a pyannote stream
The script creates a streaming session with one simple request:
```python theme={null}
r = requests.post(
"https://api.pyannote.ai/v1/live",
headers={"Authorization": f"Bearer {PYANNOTEAI_API_KEY}"},
)
```
The response contains a WebSocket URL. Connect to it and send 16 kHz mono float32 PCM chunks every 100 ms. See [Streaming Diarization](/tutorials/streaming-real-time) for stream format details.
### Use pyannote as turn detector
OpenAI realtime transcription can do its own turn detection, but this script disables it:
```python theme={null}
"turn_detection": None
```
pyannote owns speaker turns, so speaker boundaries and transcript segment boundaries stay aligned.
### Send one audio stream to both APIs
The microphone captures 16 kHz float32 audio. That exact frame goes to pyannote:
```python theme={null}
await ws.send(frame.astype("
Content-Type: application/json
{}
```
Response:
```json theme={null}
{
"id": "123e4567-e89b-12d3-a456-426614174000",
"url": ""
}
```
The response contains a single-use `url`. You can hand this URL directly to your end-user's client, it only grants access to this one stream and carries no team credentials or API key.
**2. Connect to the WebSocket URL**
Open a WebSocket connection to the `url` returned above. The connection authenticates automatically via the token embedded in the URL, no additional headers needed.
Cold-starts may delay the WebSocket connection by a few seconds. Wait for the connection to be fully open before sending audio, the `open` event (or equivalent in your WebSocket client) is your signal that it is safe to start streaming.
**3. Stream audio and receive diarization events**
Send raw audio binary frames over the WebSocket at real-time pace every 100 ms. The server enforces a maximum 5-second buffer; rushing audio ahead of real-time will cause the connection to be closed. The server will emit JSON diarization events as speakers are detected.
```text theme={null}
→
→
← {"type":"diarization_speaker_start","data":{"timestamp":0.42,"speaker":"SPEAKER_00"}}
→
← {"type":"diarization_speaker_end","data":{"timestamp":1.86,"speaker":"SPEAKER_00"}}
```
## Input events
### audio\_chunk
Send audio as raw binary WebSocket frames. The audio must meet these requirements:
| Property | Value |
| -------------- | -------------------------------------------- |
| Format | PCM float 32-bit little-endian (`pcm_f32le`) |
| Sample rate | 16 kHz |
| Channels | Mono |
| Chunk duration | 100 ms |
Send **raw PCM bytes only** — do not include any file headers (e.g. WAV/RIFF headers). The server expects a continuous stream of audio samples with no container or metadata.
The API tracks up to **8 speakers** simultaneously. In case the stream involves more speakers, multiple speakers will end up being merged into one.
### end\_of\_stream
When you have no more audio to send, signal the end of the stream by sending a JSON text frame:
```json theme={null}
{"type": "end_of_stream"}
```
Sending this message is optional, but recommended. It tells the server that no more audio frames will be sent, allowing it to finalize diarization and emit any remaining events without waiting for a timeout. The server will then close the connection with close code 1000: normal closure. **Do not send further audio frames after `end_of_stream`.** Using this message is recommended over abruptly closing the socket, which may cause final outputs to be lost.
## Output events
The server emits JSON text frames with the following event types:
### `diarization_speaker_start`
Emitted when a speaker begins a turn.
```json theme={null}
{
"type": "diarization_speaker_start",
"data": {
"timestamp": 1.24,
"speaker": "SPEAKER_00"
}
}
```
### `diarization_speaker_end`
Emitted when a speaker's turn ends.
```json theme={null}
{
"type": "diarization_speaker_end",
"data": {
"timestamp": 3.86,
"speaker": "SPEAKER_00"
}
}
```
`timestamp` is in seconds, relative to the start of the stream. `speaker` is a stable string label for the duration of the session.
### `error`
Emitted when the server encounters a problem processing a frame (e.g. wrong chunk size).
```json theme={null}
{
"type": "error",
"message": "Invalid chunk size"
}
```
## Pricing
Streams are billed based on the audio duration sent over the WebSocket connection, with a 20-second minimum per stream. See [Billing](/administration/billing) for details.
## Limits
| Limit | Value |
| ----------------------------------- | ---------- |
| Concurrent running streams per team | 10 streams |
| Idle timeout (no audio received) | 5 seconds |
| Maximum stream duration per stream | 5 hours |
## Example: Streaming microphone
```python Python theme={null}
# /// script
# requires-python = ">=3.10"
# dependencies = [
# "pyaudio",
# "requests",
# "websocket-client",
# ]
# ///
"""
pyannote.ai streaming diarization from microphone
Usage:
API_KEY=sk_xxx uv run main.py
"""
import json
import os
import signal
import struct
import threading
import pyaudio
import requests
import websocket
API_KEY = os.environ.get("API_KEY", "sk_xxx")
SAMPLE_RATE = 16_000
CHUNK_DURATION_MS = 100
CHUNK_SIZE = (SAMPLE_RATE * CHUNK_DURATION_MS) // 1000 # 1600 samples
# ANSI colors assigned to speakers in order of first appearance
_ANSI_COLORS = [
"\033[32m",
"\033[33m",
"\033[34m",
"\033[35m",
"\033[36m",
"\033[31m",
"\033[37m",
"\033[93m",
]
_RESET = "\033[0m"
_speaker_colors: dict[str, str] = {}
def speaker_color(speaker: str) -> str:
if speaker not in _speaker_colors:
_speaker_colors[speaker] = _ANSI_COLORS[
len(_speaker_colors) % len(_ANSI_COLORS)
]
return _speaker_colors[speaker]
def on_message(ws_app, message):
msg = json.loads(message)
t = msg.get("type")
if t in ("diarization_speaker_start", "diarization_speaker_end"):
speaker = msg["data"]["speaker"]
ts = msg["data"]["timestamp"]
color = speaker_color(speaker)
label = "start" if t == "diarization_speaker_start" else "end "
print(f"{color}[{label}] {speaker} @ {ts:.2f}s{_RESET}")
def on_open(ws_app):
print("Connected. Streaming... (press Ctrl+C to stop)")
pa = pyaudio.PyAudio()
stream = pa.open(
rate=SAMPLE_RATE,
channels=1,
format=pyaudio.paInt32,
input=True,
frames_per_buffer=CHUNK_SIZE,
)
def audio_thread():
try:
while ws_app.keep_running:
pcm_i32 = stream.read(CHUNK_SIZE, exception_on_overflow=False)
# convert int32 → float32 (pcm_f32le) for the API
samples_i32 = struct.unpack(f"{CHUNK_SIZE}i", pcm_i32)
pcm_f32 = struct.pack(
f"{CHUNK_SIZE}f",
*(s / 2147483648.0 for s in samples_i32),
)
ws_app.send_bytes(pcm_f32)
except Exception as exc:
print(f"Audio error: {exc}")
finally:
stream.stop_stream()
stream.close()
pa.terminate()
t = threading.Thread(target=audio_thread, daemon=True)
t.start()
def main():
print("Creating stream session...")
response = requests.post(
"https://api.pyannote.ai/v1/live",
headers={
"Authorization": f"Bearer {API_KEY}",
"Content-Type": "application/json",
},
)
response.raise_for_status()
url = response.json()["url"]
print("Connecting WebSocket...")
ws_app = websocket.WebSocketApp(
url,
on_open=on_open,
on_message=on_message,
)
def handle_sigint(sig, frame):
print("\nSending end_of_stream...")
if ws_app.sock and ws_app.sock.connected:
ws_app.send(json.dumps({"type": "end_of_stream"}))
else:
ws_app.close()
signal.signal(signal.SIGINT, handle_sigint)
ws_app.run_forever()
if __name__ == "__main__":
main()
```
Combine streaming diarization with OpenAI realtime transcription to print speaker-attributed transcripts live.
# Use AWS S3 private objects
Source: https://docs.pyannote.ai/tutorials/use-s3-private-files
This tutorial shows how to use signed urls to create jobs with private objects stored in AWS S3
The AWS Simple Storage Service (S3) can be used as a source for jobs. If your
audio already resides on S3, you can process it there directly with pyannoteAI
APIs by providing temporary access credentials to your data.
## Pre-Signed URLs
A pre-signed URL can be a convenient way to provide a URL containing temporary
credentials as a basic string. pyannoteAI Jobs APIs need to GET the `url` parameter
when reading it. Therefore, to make a request with pre-signed URLs, you'll need
to generate it.
Here's an example for how to do this:
```python generate-signed-url.py theme={null}
# pip install boto3
import boto3
from botocore.exceptions import ClientError
def create_presigned_url(bucket_name, object_name,
operation='get_object', expiration=3600):
client = boto3.client('s3')
try:
return client.generate_presigned_url(operation,
Params={'Bucket': bucket_name, 'Key': object_name},
ExpiresIn=expiration
)
except ClientError as e:
print(e)
# Use the presigned URL as the input parameter
signed_url = presign.create_presigned_url('your-bucket-name', 'object/name.mp3')
request = {
'url': signed_url,
}
# ... send the request to the Job API
```
For further examples, see the AWS documentation on [Creating a pre-signed URL for Amazon S3](https://docs.aws.amazon.com/AmazonS3/latest/userguide/ShareObjectPreSignedURL.html).
# Receiving Webhooks
Source: https://docs.pyannote.ai/webhooks/receiving-webhooks
pyannoteAI can send an HTTP POST request to a specified URL when a job reaches a terminal status.
To receive webhooks, you must specify a webhook URL in the `webhook` field when
creating a [diarization](/api-reference/diarize), [identify](/api-reference/identify)
or [voiceprint](/api-reference/voiceprint) job.
The webhook URL must use **HTTPS**. Plain HTTP URLs are not accepted. HTTPS
ensures that webhook payloads, which may contain sensitive transcription or
speaker data, are encrypted in transit and cannot be intercepted or tampered
with.
You can also set `webhookStatusOnly` to `true` when creating the job (default:
`false`). When this option is enabled, webhook payloads only include `jobId`
and `status` (the `output` field is excluded), which is useful for large
payloads.
Webhooks are sent when the job status is one of:
* `succeeded`
* `failed`
* `canceled`
Webhooks are not sent for `pending`, `created`, or `running` statuses.
The request body will be a JSON object containing job data. Here's an example of what the request body will look like for a succeeded diarization job:
```json Request body theme={null}
{
"jobId": "job_id",
"status": "succeeded",
"output": {
"diarization": [
{
"start": 0.0,
"end": 1.0,
"speaker": "speaker_1"
}
...
]
}
}
```
The `output` field will be different depending on the type of job you created.
* [Diarization](/api-reference/diarize) jobs will have a [diarization schema](/api-reference/schemas/diarizationschema) as the value of the `output` field.
* [Identify](/api-reference/identify) jobs will have an [identify schema](/api-reference/schemas/identifyschema) as the value of the `output` field.
* [Voiceprint](/api-reference/voiceprint) jobs will have a [voiceprint schema](/api-reference/schemas/voiceprintschema) as the value of the `output` field.
If a job fails, the `status` field will be `failed` and there will be no
`output` field. If a job is canceled, the `status` field will be `canceled`
and there will be no `output` field. If `webhookStatusOnly` is `true`, there
will be no `output` field for any status, including `succeeded`.
## Retries
pyannoteAI will retry sending the webhook up to 3 times. The first retry is immediate, the second retry after 1 minute, and the third retry after 5 minutes.
With each retry attempt, you'll also be given a `x-retry-num` HTTP header indicating the attempt number: `1`, `2`, or `3`.
An attempt will be considered unsuccessful if your webhook doesn't behave as expected.
This will be communicated to you in the `x-retry-reason` HTTP header, where you'll find a string describing the reason why the previous attempt failed.
Here is a list of all the possible failure codes and their reason:
* `http_timeout`: We didn't receive a response from your server within 10 seconds.
* `too_many_redirects`: The request was redirected more than twice.
* `connection_failed`: We couldn't connect to your server.
* `ssl_error`: We couldn't verify the authenticity of your SSL certificate.
* `http_error`: Your server responded with an HTTP status code that was not in the HTTP 200 OK range.
* `unknown_error`: We encountered an unknown error.
## Example webhook payloads
For a successful diarization job, the webhook payload will look like the following:
```json Request body theme={null}
{
"jobId": "123e4567-e89b-12d3-a456-426614174000",
"status": "succeeded",
"output": {
"diarization": [
{
"start": 0.0,
"end": 1.0,
"speaker": "speaker_1"
}
...
]
}
}
```
For a failed job, the webhook payload will look like the following:
```json Request body theme={null}
{
"jobId": "123e4567-e89b-12d3-a456-426614174000",
"status": "failed"
}
```
For a canceled job, the webhook payload will look like the following:
```json Request body theme={null}
{
"jobId": "123e4567-e89b-12d3-a456-426614174000",
"status": "canceled"
}
```
When you create a job with `webhookStatusOnly: true`, the webhook payload will
look like the following even if the job succeeded:
```json Request body theme={null}
{
"jobId": "123e4567-e89b-12d3-a456-426614174000",
"status": "succeeded"
}
```
# Verifying Webhooks
Source: https://docs.pyannote.ai/webhooks/verifying-webhooks
To prevent undesired requests to your webhooks, you can verify the webhook signature with a unique secret for your team. This secret can always be used to verify that the webhook sender is valid.
## Why verify webhooks?
A webhook is an HTTP POST request from an API service to a URL you control. You might want to verify these requests to ensure they're from pyannoteAI and not from a malicious actor.
## How to verify webhooks
Each webhook request includes 2 HTTP headers that you can use to verify the request:
* `X-Signature`: The base64 encoded signature of the webhook request.
* `X-Request-Timestamp`: The timestamp of the webhook request in seconds since the Unix epoch.
### 1. Creating the signed content
As the webhook receiver, you need to create the signed content by concatenating the `timestamp` and the raw `body` of the request with a colon (`:`) and prefixing the result with `v0:`.
In code, this looks like:
```python Python theme={null}
signed_content = f"v0:{timestamp}:{body}"
```
Make sure to use the raw body of the request (without headers or other
metadata) before serializing to JSON or any other format.
### 2. Retrieving your webhook secret
You can find your webhook secret in the pyannoteAI dashboard.
1. Sign in to your dashboard at [https://dashboard.pyannote.ai](https://dashboard.pyannote.ai)
2. Click on the "Webhooks" page in the sidebar.
3. Click on the button with the eye icon to reveal your webhook secret.
4. Copy the secret to your clipboard.
On the webhook page, you can also rotate your webhook secret. This is useful if you think your secret has been compromised.
### 3. Determining the expected signature
pyannoteAI uses the HMAC-SHA256 algorithm to sign webhook requests. To determine the expected signature, you need to:
1. Create the signed content by concatenating the `timestamp` from the HTTP header and the raw `body` with a colon (`:`), then prefix the result with `v0:`.
2. Compute the HMAC-SHA256 hash of the result using your webhook secret as the key.
3. Base64 encode the result.
In code, this looks like:
```python Python theme={null}
import hmac
import base64
import hashlib
def compute_signature(timestamp, body, secret):
signed_content = f"v0:{timestamp}:{body}"
signature = hmac.new(
key=secret.encode('utf-8'),
msg=signed_content.encode('utf-8'),
digestmod=hashlib.sha256
).hexdigest()
return signature
```
### 4. Verifying the signature
Then, simply compare the computed signature with the signature you received in the `X-Signature` header.
```python Python theme={null}
computed_signature = compute_signature(timestamp, body, secret)
signature = request.headers.get("X-Signature")
if computed_signature == signature:
print("The signature is valid")
else:
print("The signature is invalid")
```
Reject the request if the signature is invalid.
### Example FastAPI server
Here's an example of how you can verify a webhook request in a FastAPI server:
```python Python theme={null}
import os
import hmac
import hashlib
from fastapi import FastAPI, Request, HTTPException
app = FastAPI()
# Get key from your environment or edit default value
key = os.getenv("PYANNOTEAI_WEBHOOK_SIGNING_SECRET", "whs_....")
@app.post("/webhook")
async def validate_signature(request: Request):
body = await request.body()
headers = request.headers
timestamp = headers.get("x-request-timestamp")
received_signature = headers.get("x-signature")
if not timestamp or not received_signature:
raise HTTPException(status_code=400, detail="Missing headers")
signed_content = f"v0:{timestamp}:{body.decode('utf-8')}"
calculated_signature = hmac.new(
key=key.encode("utf-8"),
msg=signed_content.encode("utf-8"),
digestmod=hashlib.sha256,
).hexdigest()
if not hmac.compare_digest(calculated_signature, received_signature):
raise HTTPException(status_code=403, detail="Invalid signature")
# Do something with the payload, now that we know it's valid
return {"status": "success"}
```