ATS-GPU-OCT Programmer's guide ============================== .. note:: This is the documentation for AlazarTech's ATS-GPU version |version|. Please visit our `documentation homepage `_ to find documentation for other versions or products. .. highlight:: c License Agreement ----------------- .. include:: ../../LICENSE Introduction ------------ ATS-GPU-OCT provides a framework to allow real-time OCT data processing from AlazarTech PCIe digitizers on a CUDA compatible GPU. ATS-GPU-OCT internally calls ATS-CUDA-OCT, which is a low-level library that performs all the necessary operations to perform the Fourier Transform. ATS-CUDA-OCT is described in later in this guide in the section `ATS-CUDA-OCT`_. This document assumes that the reader is familiar with ATS-SDK, the standard interface for programming AlazarTech digitizers. Having a copy of the ATS-SDK manual available can be helpful, since many references to ATSApi functions are done here. The latest version of the ATS-SDK manual can be downloaded free of charge from `AlazarTech's website `_. Prerequisites ------------- System requirements ~~~~~~~~~~~~~~~~~~~ This software requires a PC with a CUDA-enabled GPU, and sufficient CPU resources to supply data to the GPU at the desired data acquisition rate. It also requires a working installation of the same version of ATS-GPU-BASE. It was tested with GeForce GTX Titan X (Maxwell), GeForce GTX980 and Quadro P5000. DDR4 memory and a modern chipset (X99, X299) will greatly improve transfer speed and overall performance. Supported operating systems Windows and Linux operating systems are supported. Please verify that your Linux distribution is `supported by NVIDIA `_ which supplies the CUDA toolkit required to use ATS-GPU. Compiler support CMake is required to build C/C++ code. CMake files are provided. On Linux, a C++11 compiler is required to build the library. On older Red Hat distributions, a devtoolset can be obtained to use a more recent version of gcc that supports C++11. NVCC is required to compile the example code, this compiler is included with CUDA toolkit. CUDA driver requirements In order to use ATS-GPU, you must install the appropriate driver for your CUDA-enabled GPU. Drivers can be downloaded at https://www.nvidia.com/Download/index.aspx. .. note:: Under Windows operating systems, dynamic link libraries releated to ATS-GPU-OCT are installed by default in %WINDIR%\System32. For applications to link approripately to them, %WINDIR%\System32 must be added to the Windows PATH Environment Variable. ATS-GPU-OCT ----------- ATS-GPU-OCT leverages ATS-GPU-BASE to transfer data from an ATS PCIe digitizer to a GPU in a highly efficient manner. It then takes care of doing OCT processing on the data before sending it back to the host computer's RAM. Usage ~~~~~ ATS-GPU-OCT acquisitions are very similar to standard ATSApi acquisitions. Only the differences are listed here for brevity. The central function of the ATS-GPU-OCT interface is :cpp:func:`ATS_GPU_OCT_Setup()`. This function calls its ATS-GPU-BASE counterpart ``ATS_GPU_Setup()`` internally, which in turns calls ``AlazarBeforeAsyncRead()``. It takes a few extra parameters: - ``OCTFlags``: Used to define which data type, such as amplitude and phase, to obtain from the acquisition. - ``FFTLength``: This is used to select the length of the Fourier transform done on the GPU. This value must be a power of two, and it also must be equal to or larger than the record length. .. code-block:: c++ rc = ATS_GPU_OCT_Setup( boardHandle, channelMask, -(int) preTriggerSamples, samplesPerRecordPerChannel, recordsPerBuffer, buffersPerAcquisition * recordsPerBuffer, autoDMAFlags, OCTOptions, FFTLength, NULL, &fftBytesPerBuffer); We then choose the window function applied to the acquired data before the FFT processing phase. The most common usage pattern is to first generate a window function using :cpp:func:`ATS_GPU_OCT_GenerateWindowFunction()`, then to download it to the board using :cpp:func:`ATS_GPU_OCT_SetWindowFunction()`. It is possible however to use entirely custom window functions instead of the ones generated by the API. It is also possible to use complex window functions by way of downloading two arrays of points: the first for the real part of the window and the other for the imaginary one. .. code-block:: c++ rc = ATS_GPU_OCT_GenerateWindowFunction( FFT_WINDOW_HANNING, &window[0], samplesPerRecordPerChannel); // Error handling rc = ATS_GPU_OCT_SetWindowFunction( boardHandle, samplesPerRecordPerChannel, &window[0], NULL); // Error handling We then allocate memory on the GPU and CPU for data to be transferred to, and we post those buffers to the board. For this purpose, we use :cpp:func:`ATS_GPU_OCT_AllocBuffer`. This function allocates buffers on the GPU, and sets up all the intermediary state necessary for ATS-GPU-OCT to successfully transfer data. It also allocates data on the CPU to receive the processed OCT data. .. code-block:: c++ for (int i = 0; i < numberOfBuffers; i++) { buffers[i] = (float*) ATS_GPU_OCT_AllocBuffer( boardHandle, bytesPerResultBuffer, NULL); rc = ATS_GPU_OCT_PostBuffer( boardHandle, buffers[i], bytesPerResultBuffer); // Error handling } We can then start the acquisition with :cpp:func:`ATS_GP_OCT_StartCapture()`. Once acquisition is started, :cpp:func:`ATS_GPU_OCT_GetBuffer` must be called as often as possible to retrieve a buffer containing processed data on the CPU. The data can then be used by the calling application. When no longer needed, the buffer needs to be posted back. .. code-block:: c++ for (size_t i; i < buffers_per_acquisition; i++) { rc = ATS_GPU_OCT_GetBuffer( boardHandle, buffers[bufferIndex],timeout_ms); // Error handling // TODO: Process sample data in this buffer. rc = ATS_GPU_OCT_PostBuffer( boardHandle, buffers[bufferIndex], bytesPerResultBuffer); // Error handling } When acquisition is complete, :cpp:func:`ATS_GPU_OCT_AbortCapture` must be called. Buffers allocated with :cpp:func:`ATS_GPU_OCT_AllocBuffer` should then be freed with :cpp:func:`ATS_GPU_OCT_FreeBuffer`. .. code-block:: c++ ATS_GPU_OCT_AbortCapture(boardHandle); if (gpuFile != NULL) fclose(gpuFile); // Free buffers for (int i = 0; i < numberOfBuffers; i++) { ATS_GPU_OCT_FreeBuffer(boardHandle, buffers[i]); } LabVIEW Programming ^^^^^^^^^^^^^^^^^^^ LabVIEW applications must use the managed interface which allows the API to allocate and manage a list of buffers available to be filled by the board. These applications should call :cpp:func:`ATS_GPU_OCT_Setup()` with the AMDA_ALLOC_BUFFERS option selected in the “autoDMAFlags” parameter. This option will cause the API to allocate and manage a list of buffers available to be filled by the board. It is therefore not necessary for the application to call :cpp:func:`ATS_GPU_OCT_AllocBuffer()` or :cpp:func:`ATS_GPU_OCT_FreeBuffer()`. The application must call :cpp:func:`ATS_GPU_OCT_ManageGetBuffer()` to wait for a buffer to be filled. When the board receives sufficient trigger events to fill a buffer, the API will copy the data from the internal buffer to the user-supplied buffer. :cpp:func:`ATS_GPU_OCT_ManageGetBuffer()` internally calls :cpp:func:`ATS_GPU_OCT_GetBuffer()` and :cpp:func:`ATS_GPU_OCT_PostBuffer()` so applcation should not use these API calls when using the managed interface. LabVIEW users might find it convenient to edit the VI search paths to locate the appropriate subVIs for the different ATS-GPU packages and ATS-SDK. The VI Search Path can be set in the "Tools" menu under "Options", in the "Path" category. Then select the "VI Search Path" from the drop down list. By unselecting "Use default" custom VI search paths can be added. API Reference ~~~~~~~~~~~~~ .. note:: Errors from ATS-GPU-OCT will be logged in ATS_GPU.log. Relevant information about the error will be logged here and can be useful for debugging. For Windows users log file is located in %TEMP%. For Linux users log file is located in /tmp/. .. doxygenenum:: ATS_GPU_OCT_OPTIONS .. raw:: latex \clearpage .. doxygenenum:: ATS_GPU_OCT_WINDOWS .. raw:: latex \clearpage .. doxygenfunction:: ATS_GPU_OCT_AbortCapture .. raw:: latex \clearpage .. doxygenfunction:: ATS_GPU_OCT_AllocBuffer .. raw:: latex \clearpage .. doxygenfunction:: ATS_GPU_OCT_EnableVerificationMode .. raw:: latex \clearpage .. doxygenfunction:: ATS_GPU_OCT_FreeBuffer .. raw:: latex \clearpage .. doxygenfunction:: ATS_GPU_OCT_GenerateWindowFunction .. raw:: latex \clearpage .. doxygenfunction:: ATS_GPU_OCT_GetBuffer .. raw:: latex \clearpage .. doxygenfunction:: ATS_GPU_OCT_GetVersion .. raw:: latex \clearpage .. doxygenfunction:: ATS_GPU_OCT_ManageGetBuffer .. raw:: latex \clearpage .. doxygenfunction:: ATS_GPU_OCT_PostBuffer .. raw:: latex \clearpage .. doxygenfunction:: ATS_GPU_OCT_SetBuffer .. raw:: latex \clearpage .. doxygenfunction:: ATS_GPU_OCT_SetWindowFunction .. raw:: latex \clearpage .. doxygenfunction:: ATS_GPU_OCT_Setup .. raw:: latex \clearpage .. doxygenfunction:: ATS_GPU_OCT_StartCapture ATS-CUDA-OCT ------------ ATS-CUDA-OCT provides a framework to allow OCT data processing on a CUDA-enabled GPU. ATS-CUDA-OCT should be used with ATS-CUDA for buffer and stream manipulation. ATS-CUDA-OCT requires an AlazarTech board on the system in order to be used. API Reference ~~~~~~~~~~~~~ .. note:: Errors from ATS-CUDA-OCT will be logged in ATS_GPU.log. Relevant information about the error will be logged here and can be useful for debugging. For Windows users log file is located in %TEMP%. For Linux users log file is located in /tmp/. .. doxygenfunction:: ATS_CUDA_OCT_CreateFFTPlan .. raw:: latex \clearpage .. doxygenfunction:: ATS_CUDA_OCT_DestroyFFTPlan .. raw:: latex \clearpage .. doxygenfunction:: ATS_CUDA_OCT_GenerateGPUWindowFunction .. raw:: latex \clearpage .. doxygenfunction:: ATS_CUDA_OCT_FFT .. raw:: latex \clearpage .. doxygenfunction:: ATS_CUDA_OCT_GetVersion .. raw:: latex \clearpage .. doxygenfunction:: ATS_CUDA_OCT_PostFFT .. raw:: latex \clearpage .. doxygenfunction:: ATS_CUDA_OCT_PreFFT ATS-CUDA-PSOCT ------------ ATS-CUDA-PSOCT is an add-on library to ATS-GPU-OCT that allows to output the signals of interest in polarization sensitive OCT. Using ATS-CUDA-PSOCT requires a seperate license. API Reference ~~~~~~~~~~~~~ .. doxygenenum:: ATS_GPU_PSOCT_OPTIONS .. raw:: latex \clearpage .. doxygenfunction:: ATS_CUDA_PSOCT_PostFFT