How to Install NVIDIA Drivers and CUDA on an Ubuntu Dedicated Server

Many tutorials online recommend using default Ubuntu repositories or editing local user files to set up CUDA. However, for a dedicated production server, those methods often lead to outdated packages, broken cron jobs, and application errors.

This guide walks you through the enterprise-standard method for installing NVIDIA drivers and the CUDA toolkit on an Ubuntu dedicated server (specifically targeting Ubuntu 22.04 LTS and 24.04 LTS).

Prerequisites: The Pre-Flight Check

Before making system changes, verify that your server is detecting your GPU and that you have the necessary Linux kernel headers installed.

1. Check GPU Detection

Connect to your server via SSH and run:

lspci | grep -i nvidia

2. Install Build Tools and Kernel Headers

To allow the NVIDIA driver to build its kernel modules successfully, update your package lists and install the required base dependencies:

sudo apt update && sudo apt upgrade -y
sudo apt install -y build-essential dkms software-properties-common linux-headers-$(uname -r)

Step 1: Clean the Slate and Disable Nouveau

Conflicts with old driver installations or Ubuntu's default open-source driver (Nouveau) are the leading causes of failed CUDA setups.

1. Remove Old NVIDIA/CUDA Packages

If you or your OS previously attempted an installation, wipe it out completely to prevent conflicts. (Note: It is completely normal to see a large list of files being deleted here—let it run!)

# Safely target and purge all existing NVIDIA, CUDA, and associated library files
sudo apt-get --purge remove "^nvidia-.*" "^cuda-.*" "^libnvidia-.*" -y
sudo apt-get --purge remove "*cublas*" "*cufft*" "*curand*" -y

# Clean up any leftover, unused dependencies automatically
sudo apt-get autoremove -y
sudo apt-get autoclean -y

2. Blacklist the Nouveau Driver

sudo bash -c "echo 'blacklist nouveau' > /etc/modprobe.d/blacklist-nouveau.conf"
sudo bash -c "echo 'options nouveau modeset=0' >> /etc/modprobe.d/blacklist-nouveau.conf"

Apply the changes to your boot image and reboot your server:

sudo update-initramfs -u
sudo reboot

Wait a few moments for the server to come back online, then log back in via SSH to proceed.

Step 2: Add the Official NVIDIA Network Repository

Download and install the NVIDIA CUDA Keyring. (If you are on Ubuntu 22.04, replace ubuntu2404 with ubuntu2204 in the URL below).

wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2404/x86_64/cuda-keyring_1.1-1_all.deb
sudo dpkg -i cuda-keyring_1.1-1_all.deb
sudo apt-get update

Step 3: Install the Drivers and CUDA

Depending on your server's purpose, you have two paths. Installing unnecessary packages consumes valuable disk space, so choose the one that fits your use case.

Option A: The Production Path (Drivers + Runtime)

If your server is dedicated purely to running pre-compiled AI models or inference applications (like vLLM or Ollama), you only need the drivers and the runtime library.

sudo apt-get install -y cuda-drivers cuda-runtime-12-4

Option B: The Development Path (Full Toolkit)

If you are actively developing, training models, compiling custom CUDA code (using nvcc), or setting up PyTorch/TensorFlow from scratch, install the full toolkit.

sudo apt-get install -y cuda-drivers cuda-toolkit-12-4

Step 4: System-Wide Environment Configuration

Many guides instruct you to add your CUDA paths to ~/.bashrc. Do not do this on a dedicated server. That method only applies the paths to your specific user session. If a background service, Docker container, or Cron job tries to access the GPU, it will fail.

Instead, configure the environment system-wide:

# Create a system-wide profile script
sudo bash -c 'echo "export PATH=/usr/local/cuda/bin:\$PATH" > /etc/profile.d/cuda.sh'
sudo bash -c 'echo "export LD_LIBRARY_PATH=/usr/local/cuda/lib64:\$LD_LIBRARY_PATH" >> /etc/profile.d/cuda.sh'

# Make it executable
sudo chmod +x /etc/profile.d/cuda.sh

# Load the variables into your current session
source /etc/profile.d/cuda.sh

Step 5: Verify Your Installation

Let's ensure the operating system and the GPU are communicating flawlessly.

1. Check the Driver

Run the System Management Interface tool:

nvidia-smi

You should see a detailed table listing your GPU, its temperature, VRAM usage, and the driver version.

2. Check the CUDA Compiler (If Full Toolkit was Installed)

nvcc --version

This will return the active CUDA compiler version, confirming your path variables are set up correctly.

Congratulations! Your Ubuntu server is now fully equipped for high-performance GPU computing.