AWS has launched WhisperX, a GPU-optimized deep learning container for speaker-labeled transcription on Amazon SageMaker AI.
AWS has introduced WhisperX, a deep learning container designed for speaker-labeled transcription, now available on Amazon SageMaker AI. WhisperX integrates OpenAI's Whisper model with wav2vec2 for forced alignment and speaker diarization, providing a comprehensive solution for transcribing audio with speaker identification.
The WhisperX container is optimized for GPU instances and can be deployed on SageMaker AI's real-time and asynchronous endpoints. This setup enables users to process audio files efficiently, generating transcripts that include word-level timestamps and speaker labels. The container supports various instance types, such as `ml.g4dn.xlarge` and `ml.g5.2xlarge`, ensuring flexibility based on performance requirements.
To deploy WhisperX, users need an AWS account with appropriate SageMaker AI execution roles and access to the WhisperX deep learning container image available in Amazon Elastic Container Registry (ECR). Detailed instructions for setting up the model and endpoint configurations are provided, including necessary parameters like `InferenceAmiVersion` set to `al2-ami-sagemaker-inference-gpu-3-1` to ensure compatibility with CUDA 12.8.
For real-time transcription, the system processes audio clips within a 60-second limit, returning transcripts synchronously. For longer audio files, the asynchronous endpoint allows processing without time constraints, with results stored in an S3 bucket specified by the user.
A practical example provided involves transcribing a 40-second segment from the 2009 "Miracle on the Hudson" emergency landing, demonstrating the model's capability to handle complex, multi-speaker audio scenarios effectively.
This development offers a robust solution for applications requiring accurate transcription with speaker identification, benefiting sectors like media, legal, and customer service where understanding who said what is crucial.