Ensure that the NVIDIA drivers are correctly installed on the host system.
See: Setup.
This toolkit extends Docker to leverage NVIDIA GPUs fully, ensuring that the GPU capabilities can be used within containers without any hitches.
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey -o /tmp/nvidia-gpgkey
gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg /tmp/nvidia-gpgkey
curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list -o /tmp/nvidia-list
sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' /tmp/nvidia-list > /etc/apt/sources.list.d/nvidia-container-toolkit.list
apt update
Having the NVIDIA Container Toolkit in place, the next essential task is configuring Docker to recognize and utilize NVIDIA GPUs.
Configure the Docker runtime to use NVIDIA Container Toolkit by using the nvidia-container-cli command.
nvidia-container-cli configure --runtime=docker
NOTE: Behind the scenes, this command makes alterations to the /etc/docker/daemon.json file.
systemctl restart docker
With all the required setups in place, the exciting part begins: running a Docker container with NVIDIA GPU support.
NVIDIA maintains a series of CUDA images on Docker Hub.
Pull the specific NVIDIA CUDA image:
docker pull nvidia/cuda:12.2.0-base-ubuntu22.04
NOTE: Always check for the latest tags at NVIDIA CUDA Docker Hub to stay updated.
docker run --gpus all -it nvidia/cuda:12.2.0-base-ubuntu22.04 bash
NOTE: This command runs the Docker container with full GPU access (–gpus all) and provides an interactive shell inside the container.