Building BOINC applications with Cuda and OpenCL
We have built sample BOINC-Cuda and BOINC-OpenCL applications as templates for making your own app. The source files are in the BOINC source tree in samples/nvcuda and samples/openclapp.
Before starting, you must install the appropriate SDK for the GPU you wish to support: NVIDIA SDK, ATI / AMD SDK or Intel Ivy Bridge. This document assumes version 5 of the NVIDIA SDK or version 2.8 of the AMD SDK. (Exception: no SDK is required for the OpenCL sample on Mac OS X, which has OpenCL support built in.)
OpenCL Sample Notes
The openclapp sample calls boinc_get_opencl_ids(), which reads an init_data.xml file to determine which GPU to use. Actual project applications must also do this. Please see the ReadMe.txt file for more information.
This same sample is designed to run with AMD, NVIDIA and Intel Ivy Bridge GPUs. It is supplied with 3 minimal init_data.xml files, one for each of these 3 vendors (GPU "types".) Copy the appropriate init_data.xml file into the directory containing the openclapp executable.
Windows
The 32-bit Windows samples compile under Visual Studio.
NVIDIA Cuda
The VS project file, example_app_nvcuda.vcproj, is in boinc/win_build/. Note that NVIDIA Cuda SDK 3.0 or older supports device emulation mode. If your machine doesn't have a CUDA-enabled GPU, then you should build the sample application in the device emulation mode. Do so by setting Build -> Configuration Manager -> Configuration to either Emudebug or Emurelease.
The executable file can be found at boinc/win_build/Build/Win32/build_mode/ (build_mode could be release, debug, emurelease or emudebug).
NVIDIA OpenCL
The project file, example_app_nvopencl.vcproj, is in boinc/win_build/.
The executable file can be found at boinc/win_build/Build/Win32/build_mode/ (build_mode could be release or debug).
Upon running the executable file, you might encounter the following error:
This application has failed to start because OpenCL.dll was not found. Re-installing the application may fix this problem.
To fix this problem, you will need to install the Cuda Developer Driver for Windows, which can only be done when your machine has an NVIDIA Cuda-enabled GPU.
Please read the OpenCL Sample Notes section above.
ATI Stream OpenCL
The project file, example_app_atiopencl.vcproj, is in boinc/win_build/. It uses the same source files as NVIDIA OpenCL.
Please read the OpenCL Sample Notes section above.
Intel Ivy Bridge OpenCL
Project file to be added. It uses the same source files as NVIDIA OpenCL.
Please read the OpenCL Sample Notes section above.
Linux
The OpenCL samples assume 32-bit Ubuntu. You may need to modify them for other Linux configurations.
NVIDIA Cuda
Before running the Makefile, you will need to install gcc 4.3 and g++ 4.3. This is because the NVIDIA Cuda SDK 3.0 has not yet worked with gcc 4.0 and g++ 4.0. There should be no issue compiling cuda files with gcc 4.3 and g++ 4.3 on newer NVIDIA Cuda SDK versions. For a successful compilation, please follow these steps:
- Install gcc-4.3 and g++-4.3:
$ sudo aptitude install gcc-4.3 g++-4.3
- Go to SDK source directory:
$ cd ~/NVIDIA_GPU_Computing_SDK/C
- Create a directory and create symlinks to gcc-4.3/g++-4.3
$ mkdir mygcc
$ cd mygcc
$ ln -s $(which g++-4.3) g++
$ ln -s $(which gcc-4.3) gcc
The Makefile for Linux is made with the assumption that the NVIDIA Cuda SDK is installed at the default location $(ROOT)/home/USER_NAME/NVIDIA_GPU_Computing_SDK/. On my Linux machine, for example, it would be $(ROOT)/home/tuanle/NVIDIA_GPU_Computing_SDK/. Since the absolute path is used in Makefile, you will probably need to edit file common.mk before compiling the sample application. Do so by looking for /home/tuanle/NVIDIA_GPU_Computing_SDK in file common.mk and replace this path with an appropriate path on your machine.
You will also need to build the necessary Cuda SDK libraries by running makefile in ~/NVIDIA_GPU_Computing_SDK/C/. The resulting libraries such as libcutil_i386 are created in ~/NVIDIA_GPU_Computing_SDK/C/lib/. Notice that if these libraries are not built, running nvcuda-sample-app makefile could result in the following error:
/usr/bin/ld: cannot find -lcutil_i386
If your machine doesn't have an NVIDIA Cuda-enabled GPU, you can compile and run your app in emulation mode. As far as we know, NVIDIA Cuda SDK 3.1 no longer supports emulation mode. Thus, you will need to install an older Cuda SDK version, like version 3.0, on your machine. To build the sample application in emulation mode, simply type:
make emu=1
The executable file is created in /boinc/samples/nvcuda/linux/build_mode/ (build_mode could be release or emurelease). Also, note that you will need to define some environment variables before running the executable file.
export PATH=/usr/local/cuda/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda/lib:$LD_LIBRARY_PATH
This will resolve the following error:
./example_app_nvcuda: error while loading shared libraries: libcudart.so.3: cannot open shared object file: No such file or directory
NVIDIA OpenCL
It appears that NVIDIA OpenCL SDK is missing shared library file libOpenCL.so which is needed at compile time. This library file comes with the kernel driver package which can be installed only when your machine has an NVIDIA Cuda-enabled GPU. Thus, any attempt to compile OpenCL sample codes on a non-cuda-enabled GPU machine might result in the following error:
/usr/bin/ld: cannot find -lOpenCL
The Makefile for Linux is made with the assumption that the NVIDIA Cuda SDK is installed at the default location /usr/local/cuda/ with subdirectories include and lib. Some versions of Ubuntu have a bug which can prevent the LD_LIBRARY_PATH environment variable from being honored as described here, so the Makefile hard codes the path to the standard libraries as '/lib/i386-linux-gnu'. You may need to modify the Makefile if any of these are at different locations.
Please read the OpenCL Sample Notes section above.
ATI Stream OpenCL
The Makefile for Linux is made with the assumption that the environment variable AMDAPPSDKROOT contains the path to the ATI Stream OpenCL SDK. This environment variable should be set automatically when you install the SDK.
Please read the OpenCL Sample Notes section above.
Intel Ivy Bridge OpenCL
Project file to be added. It uses the same source files as NVIDIA OpenCL.
Please read the OpenCL Sample Notes section above.
Mac OS X
The Mac versions are built using gcc/make (not xcode).
NVIDIA Cuda
Unlike Windows and Linux, the 'nvcc' cuda compiler for Mac has some trouble compiling .cu files that contain both BOINC and Cuda code. Any attempt to compile such .cu files might result in errors like these:
/usr/lib/gcc/i686-apple-darwin9/4.0.1/include/mmintrin.h(55): error: identifier `__builtin_ia32_emms` is undefined
/usr/lib/gcc/i686-apple-darwin9/4.0.1/include/mmintrin.h(68): error: identifier `__builtin_ia32_vec_init_v2si` is undefined
/usr/lib/gcc/i686-apple-darwin9/4.0.1/include/mmintrin.h(111): error: identifier `__builtin_ia32_vec_ext_v2si` is undefined
...
The solution that we came up with is to compile BOINC and Cuda separately, meaning that compile .c files that have BOINC codes with gcc and compile .u files that have Cuda codes such as kernel definitions with nvcc (This can simply be done by putting .cu and .c file names at corresponding CUFILES := and CCFILES := entries in Makefile_mac). To handle the results computed by the kernels, you will need to write external functions. The function definitions that make kernel calls will be put in .cu file while the function headers with the extern prefix are put in .c file and can be called in main function or elsewhere.
If your machine doesn't have an NVIDIA Cuda-enabled GPU, you can compile and run your app in emulation mode. As far as we know, NVIDIA Cuda SDK 3.1 no longer supports emulation mode. Thus, you will need to install an older Cuda SDK version, like version 3.0, on your machine.
The makefile for Mac is made with the assumption that the NVIDIA Cuda SDK is installed at the default location: $ROOT/Developer/GPU Computing/ on Mac. If it is installed at a different location on your machine, then you will need to edit file common_mac.mk. Do so by looking for ROOTDIR ?= /Developer/GPU\ Computing in file common_mac.mk and replace this path with an appropriate path on your machine.
To build the sample application in emulation mode, simply type:
make -f Makefile_mac emu=1
The executable file is created in /boinc/samples/nvcuda/darwin/build_mode/ (build_mode could be release or emurelease). Also, note that you will need to define some environment variables before running the executable file.
export PATH=/usr/local/cuda/bin:$PATH
export DYLD_LIBRARY_PATH=/usr/local/cuda/lib:$DYLD_LIBRARY_PATH
OpenCL
Mac OS X has built in support for OpenCL. No additional SDK is needed to build and run the OpenCL sample, provided that you have an OpenCL-capable GPU. Note: at this time, Mac OS X does not yet support OpenCL on Intel Ivy Bridge GPUs.
Use the Terminal application to build the sample application. In Terminal, cd to the samples/openclapp directory and type:
$ make -f Makefile_mac
Please read the OpenCL Sample Notes section above.