GPGPU High Performance Computing using OpenCL -- Work with Multiple Vendor Platforms using ICD

Sometimes you have different workloads that are better handled with different vendor OpenCL implementations.
For example if your data presents access locality or coherence, the number of calculation per data element transferred on PCI  is high and the number of calculation per memory access is also high, you can use NVIDIA's OpenCL implementation on its GPGPU.
On the other hand, if your data doesn't possess all of the characteristic mentioned above, you can use ATI/AMD's OpenCL implementation on CPU so that you at least can take advantage of all your CPU cores.

OpenCL has an extension "cl_khr_icd". It is technically called OpenCL ICD (Installable Client Driver) that is a means of allowing multiple OpenCL implementations to co-exist and applications to select between them at runtime.

In my case since I have both CPU and GPGPU-friendly data, I installed on a 64-bit Windows 7 workstation both NVIDIA GPU Computing SDK 3.2 that support OpenCL 1.0, and ATI Stream SDK 2.2 that support OpenCL 1.1.

1. Find out whether your Vendor's SDK Supports OpenCL ICD
You can compile and run NVIDIA's oclDeviceQuery program. If you see "cl_khr_icd" in section "CL_DEVICE_EXTENSIONS" it means NVIDIA supports Opencl ICD.
You can also run ATI's "clinfo" program. It actually prints out all OpenCL platforms. Again if you see "cl_khr_icd" in section "Extension" it means the corresponding platform supports Opencl ICD.

2. How does ICD Work?
Although I setup all my Visual Studio projects with NVIDIA GPU Computing SD, I basically can still dynamically load ATI's CPU OpenCL implementation.
The project setup basically needs two configurations. One is to specify the OpenCL include directory that contains vendor-neutral headers such as cl.h, cl_ext.h and some vendor-specific headers such as cl_agent_amd.h for AMD.
Unless your SDK's have different OpenCL versions, all vendor-neutral headers should be the same. If you need vendor-specific headers, of course you have to setup your project with that vendor's SDK.

The other configuration is to specify the OpenCL library directory where a vendor-neutral stub library "OpenCL.lib" is located. This "OpenCL.lib" is called ICD loader that is responsible to dynamically load a vendor specific ICD library.
When you install a OpenCL SDK, the SDK registers its ICD DLL in the register key
HKEY_LOCAL_MACHINE\SOFTWARE\Khronos\OpenCL\Vendors on Windows if it supports ICD.
So the ICD loader can scan the above register key and dynamically load your vendor's DLL library.

For more information, please refer to this resource. 

3. How does ICD affect your Code?
Your application is now responsible for selecting which of the OpenCL platforms present on a system it wishes to use, instead of just requesting the system default.
This means using the clGetPlatformIDs() and clGetPlatformInfo() functions to examine the list of available OpenCL implementations and selecting the one that best suits your requirements.

For example, previously you could call the following function without specifying the specific platform value:
context = clCreateContextFromType(
     0, CL_DEVICE_TYPE_GPU, NULL, NULL, &status);

With ICD, you must specify a specific platform:

cl_context_properties cps[3] = {
              CL_CONTEXT_PLATFORM,
              (cl_context_properties)platform,
              0};
context = clCreateContextFromType(
     cps, CL_DEVICE_TYPE_GPU, NULL, NULL, &status);