MEPION

", launched the basics of CUDA programming by showing how to put in writing a simple program that allotted two arrays of numbers in memory accessible to the GPU and then added them collectively on the GPU. To do that, I introduced you to Unified Memory, which makes it very easy to allocate and access knowledge that can be utilized by code running on any processor within the system, CPU or GPU. I completed that publish with a number of easy "exercises", one in every of which encouraged you to run on a current Pascal-based mostly GPU to see what occurs. I was hoping that readers would try it and touch upon the results, and some of you did! I steered this for two reasons. First, because Pascal GPUs such as the NVIDIA Titan X and the NVIDIA Tesla P100 are the primary GPUs to include the Web page Migration Engine, which is hardware support for Unified Memory web page faulting and migration.

The second reason is that it gives an ideal alternative to study extra about Unified Memory. Fast GPU, Fast Memory… Proper! But let’s see. First, I’ll reprint the outcomes of running on two NVIDIA Kepler GPUs (one in my laptop and one in a server). Now let’s strive running on a very quick Tesla P100 accelerator, primarily based on the Pascal GP100 GPU. Hmmmm, that’s under 6 GB/s: slower than running on my laptop’s Kepler-based mostly GeForce GPU. Don’t be discouraged, though; we are able to fix this. To grasp how, I’ll should let you know a bit more about Unified Memory. What's Unified Memory? Unified Memory is a single memory deal with space accessible from any processor in a system (see Figure 1). This hardware/software program know-how permits functions to allocate data that can be read or written from code running on both CPUs or GPUs. Allocating Unified Memory is so simple as replacing calls to malloc() or new with calls to cudaMallocManaged(), an allocation perform that returns a pointer accessible from any processor (ptr in the next).

When code running on a CPU or GPU accesses knowledge allotted this fashion (usually known as CUDA managed knowledge), the CUDA system software and/or the hardware takes care of migrating memory pages to the memory of the accessing processor. The important level right here is that the Pascal GPU architecture is the primary with hardware support for digital memory page faulting and migration, through its Page Migration Engine. Older GPUs based on the Kepler and Maxwell architectures also assist a more restricted type of Unified Memory. What Occurs on Kepler After i name cudaMallocManaged()? On techniques with pre-Pascal GPUs just like the Tesla K80, calling cudaMallocManaged() allocates measurement bytes of managed memory on the GPU system that is lively when the call is made1. Internally, the driver additionally units up web page desk entries for all pages covered by the allocation, so that the system is aware of that the pages are resident on that GPU. So, in our example, operating on a Tesla K80 GPU (Kepler architecture), x and y are both initially fully resident in GPU memory.

Then within the loop beginning on line 6, the CPU steps by way of each arrays, initializing their parts to 1.0f and 2.0f, respectively. Since the pages are initially resident in gadget memory, a web page fault happens on the CPU for every array web page to which it writes, and the GPU driver migrates the page from machine memory to CPU memory. After the loop, all pages of the 2 arrays are resident in CPU memory. After initializing the info on the CPU, this system launches the add() kernel so as to add the weather of x to the weather of y. On pre-Pascal GPUs, upon launching a kernel, the CUDA runtime should migrate all pages previously migrated to host memory or to a different GPU back to the machine memory of the gadget operating the kernel2. Since these older GPUs can’t web page fault, all information must be resident on the GPU just in case the kernel accesses it (even if it won’t).