| CUDA 13.4 Memory Default Change: Action Required |
 |
CUDA 13.4 (driver r615) updates the default memory management mode on coherent NVIDIA systems (GH200, GB200, GB300, Vera Rubin) from NUMA mode to CDMM mode (Coherent Driver-based Managed Memory). In NUMA mode, Linux manages GPU memory as part of system memory and can move data into it automatically. In CDMM mode, the NVIDIA driver manages placement in GPU memory. A system runs in either mode, set per node.
How This Affects Your Workloads
- No Action Needed: Applications using standard cudaMalloc and explicit cudaMemcpy calls will continue working normally without changes.
- Action Recommended: Applications using Unified Memory, managed memory, or automatic page migration (common with NVHPC, OpenACC, or OpenMP offload) require NUMA mode to maintain automatic page movement. Automatic migration is disabled by default in CDMM mode under r615.
How to Check Your Current Setting
Run the following command on your node:
|
|
|
|
BashBash
grep Coherent /proc/driver/nvidia/params |
|
|
|
- CoherentGPUMemoryMode: "driver" = CDMM mode
- CoherentGPUMemoryMode: "numa" = NUMA mode
How to Force NUMA Mode (Recommended Before Upgrading)
To keep or revert to NUMA mode across your node, apply these settings and reboot:
|
|
|
BashBash
echo 'options nvidia NVreg_CoherentGPUMemoryMode=numa' \
| sudo tee /etc/modprobe.d/nvidia-openrm.conf
echo 'options nvidia-uvm uvm_disable_sam_migration=false' \
| sudo tee -a /etc/modprobe.d/nvidia-openrm.conf |
|
|
|
Note: This is a system-wide administrative setting. Changes take effect after a reboot or driver reload.
For complete technical details, driver reload steps, and deep dives, refer to the full Release Notes and this NVIDIA Technical Blog. |
|
|
|
|
|
|
