Streams con thrust y alguito mas... Clase 15.5 Códigos para la clase Códigos: async_thrust.cu, async_reduce_thrust.cu ● ● cp -r /share/apps/codigos/alumnos_icnpg2016/clase_15/thrust_streams . ; cd thrust18 Thrust 1.8: streams, new execution policies, dynamic paralellism, etc. Streams with Thrust Streams Ver async.cu, clase streams... Streams con Thrust thrust::cuda::par is the parallel execution policy associated with Thrust's CUDA backend system. Ahora puede tomar como argumento el stream! Notación: thrust::cuda::par.on(stream[i]), On the other hand, it's safe to use thrust::cuda::par with raw pointers allocated bycudaMalloc, even when the pointer isn't wrapped by thrust::device_ptr: Streams con Thrust ● ● ● ● Ejemplo de clase de streams reciclado, mas nuevo material de demo ... cp -r /share/apps/codigos/alumnos_icnpg2016/thrust_streams . ; cd thrust_streams Incluimos thrust 1.8 headers (no necesario con cuda 7 toolkit en adelante), y el macro THRUST18 (omitir para cambiar a la version async.cu). nvcc async_thrust.cu -I /share/apps/codigos/thrust-master -DTHRUST18 Ejecutamos nvprof -o a.prof ./a.out qsub submit_gpu.sh Verificar concurrencia de streams (overlaps copy-kernel, kernel-kernel...) nvpp → import a.prof Streams con Thrust // baseline case - sequential transfer and execute // asynchronous version 1: loop over {copy, kernel, copy} Streams con Thrust A optimizar! ... Algorithm invocation in CUDA __device__ code Algorithm invocation in CUDA __device__ code template<typename Iterator, typename T, typename BinaryOperation, typename Pointer> __global__ void reduce_kernel(Iterator first, Iterator last, T init, BinaryOperation binary_op, Pointer result) { *result = thrust::reduce(thrust::cuda::par, first, last, init, binary_op); } int main() { size_t n = 1 << 20; thrust::device_vector<unsigned int> data(n, 1); thrust::device_vector<unsigned int> result(1, 0); // method 1: call thrust::reduce from an asynchronous CUDA kernel launch // create a CUDA stream cudaStream_t s; cudaStreamCreate(&s); // launch a CUDA kernel with only 1 thread on our stream reduce_kernel<<<1,1,0,s>>>(data.begin(), data.end(), 0, thrust::plus<int>(), result.data()); // wait for the stream to finish cudaStreamSynchronize(s); // our result should be ready assert(result[0] == n); cudaStreamDestroy(s); return 0; } Ejemplo de la librería: /share/apps/codigos/thrust-master/examples/cuda/async_reduce.cu template<typename Iterator, typename T, typename BinaryOperation, typename Pointer> __global__ void reduce_kernel(Iterator first, Iterator last, T init, BinaryOperation binary_op, Pointer result) { *result = thrust::reduce(thrust::cuda::par, first, last, init, binary_op); } int main() { size_t n = 1 << 18; thrust::device_vector<unsigned int> data(n, 1); thrust::device_vector<unsigned int> result(1, 0); // method 1: call thrust::reduce from an asynchronous CUDA kernel launch // create a CUDA stream cudaStream_t s; cudaStreamCreate(&s); // launch a CUDA kernel with only 1 thread on our stream reduce_kernel<<<1,1,0,s>>>(data.begin(), data.end(), 0, thrust::plus<int>(), result.data()); nvtxRangePushA("la CPU hace su propia reduccion... quien gana?"); unsigned res=0; unsigned inc=1; for(int i=0;i<n;i++) res+=inc; nvtxRangePop(); // wait for the stream to finish cudaStreamSynchronize(s); Ejemplo de la librería modificado para la clase: /share/apps/codigos/alumnos_icnpg2016/thrust_streams/async_reduce_thrust.cu Algorithm invocation in CUDA __device__ code Si todavia no lo hicieron... ● cp -r /share/apps/codigos/alumnos_icnpg2015/thrust18 . ; cd thrust18 Incluimos thrust 1.8 headers, NVTX, y flags para paralelismo dinamico... ● nvcc -arch=sm_35 -rdc=true async_reduce_thrust.cu -lcudadevrt -lnvToolsExt -I /share/apps/codigos/thrust-master -o a.out Ejecutamos nvprof -o a.prof ./a.out ● qsub submit_gpu.sh Algorithm invocation in CUDA __device__ code Ejemplo de reduccion asincrónica usando GPU size_t n = 1 << 18; Prueben aumentarlo... Paralelismo dinámico Dynamic Parallelism in CUDA is supported via an extension to the CUDA programming model that enables a CUDA kernel to create and synchronize new nested work. Basically, a child CUDA Kernel can be called from within a parent CUDA kernel and then optionally synchronize on the completion of that child CUDA Kernel. The parent CUDA kernel can consume the output produced from the child CUDA Kernel, all without CPU involvement. Dynamic Parallelism http://devblogs.nvidia.com/parallelforall/introduction-cuda-dynamic-parallelism/ Early CUDA programs had to conform to a flat, bulk parallel programming model. Programs had to perform a sequence of kernel launches, and for best performance each kernel had to expose enough parallelism to efficiently use the GPU. For applications consisting of “parallel for” loops the bulk parallel model is not too limiting, but some parallel patterns—such as nested parallelism—cannot be expressed so easily. Nested parallelism arises naturally in many applications, such as those using adaptive grids, which are often used in real-world applications to reduce computational complexity while capturing the relevant level of detail. Flat, bulk parallel applications have to use either a fine grid, and do unwanted computations, or use a coarse grid and lose finer details. Dynamic Parallelism http://devblogs.nvidia.com/parallelforall/introduction-cuda-dynamic-parallelism/ Dynamic parallelism is generally useful for problems where nested parallelism cannot be avoided. This includes, but is not limited to, the following classes of algorithms: ● algorithms using hierarchical data structures, such as adaptive grids; ● algorithms using recursion, where each level of recursion has parallelism, such as quicksort; ● algorithms where work is naturally split into independent batches, where each batch involves complex parallel processing but cannot fully use a single GPU. Mandelbrot Set En negro NVIDIA_CUDA-6.5_Samples/2_Graphics/Mandelbrot/ Mandelbrot Set The Escape Time Algorithm Mandelbrot Set The Escape Time Algorithm Mandelbrot Set Recursion: Subdivision de cuadrados Grilla Madre Grilla Hija Grilla Nieta Mariani-Silver Algorithm Dynamic Parallelism http://devblogs.nvidia.com/parallelforall/introduction-cuda-dynamic-parallelism/ De un solo paso seria asi: nvcc -arch=sm_35 -rdc=true myprog.cu -lcudadevrt -o myprog.o Dynamic Parallelism http://devblogs.nvidia.com/parallelforall/cuda-dynamic-parallelism-api-principles/ Dynamic Parallelism http://devblogs.nvidia.com/parallelforall/cuda-dynamic-parallelism-api-principles/ Grid Nesting and Synchronization Memory Consistency child grids always complete before the parent grids that launch them, even if there is no explicit synchronization, Passing Pointers to Child Grids ● ● ● Device Streams and Events Recursion Depth and Device Limits Etc... Dynamic Parallelism Cool Thing You Could Do with Dynamic Parallelism - Intro to Parallel Programming https://www.youtube.com/watch?v=QVvHbsMIQzY Problems Suitable for Dynamic Parallelism - Intro to Parallel Programming https://www.youtube.com/watch?v=8towMTm82DM http://devblogs.nvidia.com/parallelforall/introduction-cuda-dynamic-parallelism/ http://devblogs.nvidia.com/parallelforall/cuda-dynamic-parallelism-api-principles/ Intro to Parallel Programming UDACITY, Lesson 7.2 https://www.udacity.com/course/cs344. Dinamica Molecular con interacciones de largo alcance, por ejemplo, gravitatorias... Esquema PP: O(N*N) ● Esquema PM: O(M*log M) ● Esquema PPPM=P3M ● Tree codes: ● Algorithm invocation in CUDA __device__ code farber1-11_sync.cu CPU ociosa farber1-11_async.cu 1 thread de la CPU ocioso Piense como mejorarlo.... ahora que sabe reducción asincrónica