How to Install OpenAI Whisper on a Dedicated Server (Ubuntu 24.04 Guide 2026)

This guide covers every practical way to self-host Whisper — the original implementation, the two faster runtimes people actually use in production, and a ready-made REST API you can drop into an existing app — along with the real hardware requirements for each.

What Whisper Actually Is

Whisper is OpenAI's open-source automatic speech recognition (ASR) model, released under the MIT license. It was trained on 680,000 hours of multilingual audio and handles transcription, translation, and language identification in one model. Because it's MIT-licensed, you can run it commercially, offline, and without any API key or usage cap.

There are two current model families worth knowing:

• large-v3: The most accurate open Whisper checkpoint, released November 2023.
• large-v3-turbo: A distilled version of large-v3 released October 2024, roughly 5x faster with only a minor accuracy trade-off.

Choosing a Model: Size vs. VRAM vs. Accuracy

This is the table that actually matters when you're picking server specs. These figures come straight from the official OpenAI Whisper repository:

Practical Deployment Notes:

• VRAM Headroom: These are baseline figures for the reference PyTorch implementation. VRAM use scales up with longer audio files and larger beam sizes, so treat the numbers above as a floor, not a hard ceiling — leave headroom.
• Quantization: Running large-v3 through faster-whisper or whisper.cpp with INT8 quantization can drop its footprint into the 1.5–4 GB range, which makes the largest model usable on 8 GB consumer cards instead of requiring a 10+ GB professional GPU.
• The Go-To Model: turbo is the practical default for most people in 2026. It's a distilled version of large-v3, so you get most of the accuracy at roughly 5–8x the speed and about 6 GB VRAM instead of 10 GB.
• English-Only: If you're only transcribing English audio, the .en-suffixed models (e.g., small.en) tend to slightly outperform their multilingual counterparts, especially at the tiny and base sizes.
• Server Sizing Rule of Thumb: For turbo or large-v3 in production with real concurrency, plan for a GPU with at least 12–16 GB VRAM so you have room for multiple simultaneous jobs, longer files, and OS/driver overhead. For single-user or batch offline transcription, an 8 GB card running turbo or a quantized large-v3 is enough.

Base Server Requirements

Before installing anything:

• OS: Ubuntu 22.04 or 24.04 LTS (used throughout this guide).
• GPU: Any modern NVIDIA card with CUDA support — required for real-time or near-real-time performance. CPU-only transcription works but is dramatically slower (several times real-time for larger models).
• NVIDIA driver + CUDA: Install via ubuntu-drivers and confirm with nvidia-smi before touching Python.
• Python: 3.10 or 3.12. Whisper and its ecosystem do not yet reliably support Python 3.13.
• ffmpeg: Required by the original openai-whisper package to decode audio formats.

Install the system-level prerequisites:

sudo apt update
sudo apt install -y python3-pip python3-venv ffmpeg git

# Confirm the GPU and driver are visible before proceeding
nvidia-smi

If nvidia-smi doesn't return a driver and CUDA version, install the recommended driver first:

sudo ubuntu-drivers autoinstall
sudo reboot

There are three practical ways to run Whisper on your server. Pick based on what you're building.

Option A: faster-whisper (Recommended for Most Deployments)

faster-whisper is a reimplementation of Whisper using CTranslate2, a fast inference engine for transformer models. It's up to 4x faster than the original OpenAI implementation at the same accuracy, uses less memory, and supports 8-bit quantization on both CPU and GPU. This is the version most production deployments and downstream tools (including WhisperX below) are built on.

Set up a virtual environment and install it:

python3 -m venv whisper-env
source whisper-env/bin/activate
pip install faster-whisper

Unlike the original Whisper package, faster-whisper doesn't need a system-wide ffmpeg install — it decodes audio through the bundled PyAV library.

Basic transcription in Python:

from faster_whisper import WhisperModel

model_size = "large-v3"

# GPU, half-precision — fastest option on supported hardware
model = WhisperModel(model_size, device="cuda", compute_type="float16")

# CPU fallback with INT8 quantization
# model = WhisperModel(model_size, device="cpu", compute_type="int8")

segments, info = model.transcribe("audio.mp3", beam_size=5)

print(f"Detected language: {info.language} (probability {info.language_probability:.2f})")

for segment in segments:
    print(f"[{segment.start:.2f}s -> {segment.end:.2f}s] {segment.text}")

If you'd rather have a command-line tool that mirrors the original whisper CLI syntax exactly, install whisper-ctranslate2, which wraps faster-whisper:

pip install -U whisper-ctranslate2
whisper-ctranslate2 audio.mp3 --model medium

(Add --vad_filter True to strip silent segments before transcription, and --batched True for an additional 2–4x speed boost on longer files by transcribing segments in parallel.)

Option B: whisper.cpp (Best for CPU-only or Lightweight Servers)

whisper.cpp is a C/C++ port of Whisper built for efficient CPU inference (with optional GPU acceleration), using quantized GGML model files. It's the right choice if you're deploying on a server without a dedicated GPU, or want the smallest possible resource footprint.

Disk and memory usage for the GGML models:

Build and run it:

git clone https://github.com/ggerganov/whisper.cpp.git
cd whisper.cpp

# Download a model (e.g. medium)
sh ./models/download-ggml-model.sh medium

# Build
cmake -B build
cmake --build build --config Release

# Transcribe a WAV file (16kHz mono)
./build/bin/whisper-cli -m models/ggml-medium.bin -f audio.wav

whisper.cpp expects 16 kHz mono WAV input. Convert other formats first with ffmpeg:

ffmpeg -i input.mp3 -ar 16000 -ac 1 -c:a pcm_s16le audio.wav

Option C: whisper-asr-webservice (REST API, Best for Apps)

If you want Whisper running as a background service that other applications call over HTTP — rather than a script you run manually — whisper-asr-webservice packages faster-whisper (or WhisperX, or the original OpenAI implementation) behind a REST API with Swagger documentation, output in text/JSON/VTT/SRT/TSV, and Docker images for both CPU and GPU.

GPU deployment with Docker:

docker pull onerahmet/openai-whisper-asr-webservice:latest-gpu

docker run -d --gpus all -p 9000:9000 \
  -e ASR_MODEL=large-v3 \
  -e ASR_ENGINE=faster_whisper \
  -v $PWD/cache:/root/.cache/ \
  onerahmet/openai-whisper-asr-webservice:latest-gpu

CPU-only deployment:

docker run -d -p 9000:9000 \
  -e ASR_MODEL=medium \
  -e ASR_ENGINE=faster_whisper \
  onerahmet/openai-whisper-asr-webservice:latest

Once it's running, open http://your-server-ip:9000/docs for the interactive Swagger UI, or call it directly:

curl -X POST "http://your-server-ip:9000/asr?output=json" \
  -F "[email protected]"

Key environment variables:

• ASR_ENGINE — openai_whisper, faster_whisper, or whisperx
• ASR_MODEL — any valid model name (tiny through large-v3)
• ASR_MODEL_PATH — custom directory for cached models, useful for persisting downloads across container restarts
• ASR_DEVICE — cuda or cpu
• MODEL_IDLE_TIMEOUT — how long an unused model stays loaded in memory before being unloaded

Option D: WhisperX (Add Speaker Diarization and Precise Timestamps)

Standard Whisper output gives you segment-level timestamps, not word-level, and it has no concept of "who is speaking." If you're transcribing interviews, meetings, or podcasts and need to know which speaker said what, WhisperX adds forced word-level alignment and speaker diarization on top of faster-whisper.

pip install whisperx

whisperx audio.mp3 --model large-v3 --device cuda --compute_type float16 --diarize

Which Option Should You Actually Use?

• Just need to transcribe files from the command line or a script: faster-whisper
• No GPU, or want the lightest possible footprint on a small VPS: whisper.cpp
• Building this into an app or need a persistent API endpoint: whisper-asr-webservice
• Need to know who said what (interviews, meetings, podcasts): WhisperX

Troubleshooting Common Issues

• "CUDA out of memory" on large-v3 with an 8 GB card: This is expected for large-v3 in FP16 wants roughly 10 GB in production scenarios. Either switch to turbo (~6 GB), drop to medium (~5 GB), or use INT8 quantization with faster-whisper (compute_type="int8"), which can bring large-v3 down into the range an 8 GB card can handle.
• faster-whisper fails to find CUDA libraries at runtime: This is almost always a CUDA/cuDNN version mismatch. Recent ctranslate2 releases require CUDA 12 and cuDNN 9 — check your installed versions with nvcc --version and nvidia-smi, and downgrade ctranslate2 if you're still on CUDA 11.
• Model downloads on every container restart: Mount a persistent volume for the cache directory (as shown in the Docker examples above) so models don't re-download every time the container restarts.
• Python 3.13 install errors: Whisper's dependency chain doesn't fully support 3.13 yet. Use a 3.10 or 3.12 virtual environment instead.

Why Run This on a Dedicated Server Instead of the Cloud?

Whisper's own footprint is modest, even large-v3 only needs single-digit GB of VRAM depending on precision and quantization — so the deciding factor for most people isn't raw model size, it's what you're doing with the output. A dedicated GPU server gets you:

• No per-minute transcription costs, which matters once volume goes beyond occasional use.
• Audio and transcripts never leave infrastructure you control — highly relevant for legal, medical, or internal business recordings.
• Full control over which model and engine you run, and the ability to fine-tune or swap models without waiting on a vendor.
• Concurrent processing headroom — a GPU with 16–24 GB VRAM can run several transcription jobs in parallel instead of queuing them one at a time.

If you're already running LLM inference workloads (Ollama, vLLM) on a GPU server, Whisper is lightweight enough to run alongside them on the same card, since it uses only a fraction of the VRAM a 7B+ parameter LLM needs.