Robo.Fish Wiki

OpenCV

Page Discussion

Introduction


OpenCV (http://opencv.org) is a successful, open-source framework full of solutions for computer vision and related tasks. The robo.fish projects expect OpenCV version 4.1, or a newer 4.x release. The OpenCV installation path is assumed to be /usr/local/opencv-4.x.y.

Building From Source

If your system does not provide an up-to-date version of OpenCV, including Python 3 modules, you can always resort to building it from source. Make sure that cmake is installed. On Ubuntu Linux open a shell terminal and make sure cmake is installed. 

sudo apt install cmake


Get the source code of OpenCV from Github: 

git clone https://github.com/opencv/opencv.git OpenCV


Also get the OpenCV extra modules: 

git clone https://github.com/opencv/opencv_contrib.git OpenCV-Contrib


Change to the new OpenCV folder, look at the list of version tags, decide which tagged release version you want to build, then check out that version by creating a branch for it: 

cd OpenCV
git tag
git checkout -b v4.1.1 4.1.1

where we created the branch v4.1.1 for the release tagged 4.1.1.
Don't forget to repeat this procedure for OpenCV-Contrib using the same tag.

Make sure all dependencies for building the OpenCV source code are installed: 

sudo apt install gcc g++ cmake cmake-curses-gui python3.6-dev \
libavformat-dev libavcodec-dev libavutil-dev libswscale-dev libavresample-dev libswresample-dev \
libopenblas-dev libatlas-base-dev libatlas-dev liblapack-dev liblapacke-dev python3-numpy \
doxygen qt5-default python3-pyqt5 libqt5opengl5-dev libgphoto2-dev \
libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev libv4l-dev

where I assume a Ubuntu 18.04 Bionic system with Python 3.6 packages available from the default repositories. On Ubuntu 16.04 you can install Python 3.6 via miniconda3. Also, there should be no previously installed OpenCV libraries in /usr/lib/, which could confuse the linker. You can check by typing 

find /usr/lib/ -name 'libopencv*'


Create the build subfolder in which we will run cmake. 

mkdir build
cd build


Running cmake with the options shown below and will generate the make files for building GUI-less OpenCV binaries. 

cmake -DCMAKE_BUILD_TYPE=RELEASE \
  -DCMAKE_INSTALL_PREFIX=/usr/local/opencv-3.4.1 \
  -DOPENCV_EXTRA_MODULES_PATH=../../OpenCV-Contrib/modules -DBUILD_opencv_legacy=OFF \
  <<< Python settings >>>
  -DBUILD_TIFF=ON \
  -DBUILD_opencv_java=OFF \
  -DWITH_QT=ON \
  -DWITH_OPENGL=ON \
  -DWITH_OPENCL=ON \
  <<< CUDA settings >>>
  -DWITH_IPP=ON \
  -DWITH_TBB=ON \
  -DWITH_EIGEN=ON \
  -DWITH_V4L=ON \
  -DWITH_VTK=OFF \
  -DBUILD_TESTS=OFF \
  -DBUILD_PERF_TESTS=OFF \
  ..


Python Settings

This part of the build settings is responsible for generating the Python bindings, so that OpenCV can be used when programming in Python. Since it's 2018 and we are not interested in Python 2.7 anymore, we will build OpenCV exclusively for Python 3.6. These two settings are therefore fixed: 

  -DBUILD_opencv_python2=OFF \
  -DBUILD_opencv_python3=ON \


With that settled, we need to specify the special directories of the Python installation.

  • DPYTHON_DEFAULT_EXECUTABLE and DPYTHON3_EXECUTABLE are both paths to the Python 3 binary
  • PYTHON_INCLUDE_DIR and PYTHON3_INCLUDE_DIR are the folder in which the C API header file is located
  • DPYTHON3_PACKAGES_PATH is the folder where the shared library files for Python modules need to be installed
  • DPYTHON_LIBRARY and DPYTHON3_LIBRARY are the shared library file for Python itself


On Ubuntu 18.04, where Python 3.6 is available as a main package, this works fine: 

 -DPYTHON_DEFAULT_EXECUTABLE=/usr/bin/python3 \
 -DPYTHON_INCLUDE_DIR=/usr/include/python3.6m \
 -DPYTHON3_INCLUDE_DIR=/usr/include/python3.6m \
 -DPYTHON3_NUMPY_INCLUDE_DIRS=/usr/lib/python3/dist-packages/numpy/core/include \
 -DPYTHON3_PACKAGES_PATH=/usr/lib/python3/dist-packages \
 -DPYTHON3_EXECUTABLE=/usr/bin/python3 \
 -DPYTHON_LIBRARY=/usr/lib/x86_64-linux-gnu/libpython3.6m.so \
 -DPYTHON3_LIBRARY=/usr/lib/x86_64-linux-gnu/libpython3.6m.so \

Note that both DPYTHON_INCLUDE_DIR and DPYTHON3_INCLUDE_DIR need to point to the Python 3 directory.

The directory paths can also be extracted from the Python runtime. For example: 

-DPYTHON3_EXECUTABLE=$(which python3) \
-DPYTHON_DEFAULT_EXECUTABLE=$(which python3) \
-DPYTHON_INCLUDE_DIR=$(python3 -c "from distutils.sysconfig import get_python_inc; print(get_python_inc())") \
-DPYTHON_INCLUDE_DIR2=$(python3 -c "from os.path import dirname; from distutils.sysconfig import get_config_h_filename; print(dirname(get_config_h_filename()))") \
-DPYTHON3_INCLUDE_DIR=$(python3 -c "from distutils.sysconfig import get_python_inc; print(get_python_inc())") \
-DPYTHON3_NUMPY_INCLUDE_DIRS=$(python3 -c "import numpy; print(numpy.get_include())") \
-DPYTHON3_PACKAGES_PATH=$(python3 -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())") \
-DPYTHON_LIBRARY=$(python3 -c "from distutils.sysconfig import get_config_var;from os.path import dirname,join ; print(join(dirname(get_config_var('LIBPC')),get_config_var('LDLIBRARY')))") \


One more note on Python: If you choose to install the Python module "cv2" to a different packages folder, don't forget to add that folder to the PYTHONPATH environment variable.

CUDA Settings

I am assuming that your system is Ubuntu 18.04.3 with CUDA 10.1 installed (via the CUDA Toolkit download page). To enable support for CUDA in OpenCV use this setting 

  -DWITH_CUDA=ON \


You probably also want to enable Fast Math and cuBLAS: 

-DENABLE_FAST_MATH=ON \
-DCUDA_FAST_MATH=ON \
-DWITH_CUBLAS=ON \


To shorten the build time, you should restrict the target architectures for the CUDA compiler (NVCC) to the architecture (or compute capability level) of the installed NVIDIA card. Use the cmake option 

-DCUDA_ARCH_BIN=61

where 61 specifies compute capability level 6.1 and is given as an example. The compute capability versions supported by your NVIDIA hardware are listed in the cmake output when you run it without setting CUDA_ARCH_BIN. There is also a list at Wikipedia that shows which compute capability level is supported by which NVIDIA hardware.

Fixing the eigen3 Header Path

In the file modules/core/include/opencv2/core/private.hpp, edit line 66 

#  include <Eigen/Core>

by inserting eigen/ before Eigen like this 

#  include <eigen3/Eigen/Core>


Intel Optimizations

The build option -DWITH_IPP=ON is for workstations with Intel processors and enables accelerated computation by using the Intel IPP-ICV library. The library will be downloaded automatically into the subfolder 3rdparty/ippicv during the build process.

More Options

All the build options can be explored with the ccmake tool 

ccmake ..


Gtk+ 3 development files need to be present if you want imshow() to work. On Debian based distros look for the libgtk-3-dev package.

Make & Install

After the configuration completes successfully simply enter 

make -j7

to build OpenCV, where the optional parameter -j<n> specifies the number of parallel build processes to run. After OpenCV has been built type 

sudo make install

to install the headers (into /usr/local/include/) and the library files (into /usr/local/lib/).

To uninstall, run 

sudo make uninstall

Just make sure you don't delete the build subfolder before uninstalling. The file install_manifest.txt inside the build folder is needed by the uninstaller script. Otherwise you will need to manually delete the copied files.

Test

Testing whether OpenCV was successfully installed is easy. Just check the contents of the installation folder that you provided with the setting CMAKE_INSTALL_PREFIX, which defaults to /usr/local.

To check whether the Python library was successfully installed, look for the file cv2.cpython-36m-x86_64-linux-gnu.so in the Python libraries directory that you specified (PYTHON3_PACKAGES_PATH) and run the test script 

import os
import cv2                     # loads the OpenCV module
from cv2 import __version__
os.path.dirname(cv2.__file__)  # prints the directory from which OpenCV was loaded
__version__                    # prints the version of the loaded OpenCV module


Embedding OpenCV-Python In Your C++ Application

Boost Python is often used for make integration of the Python C API into your C++ project easier. It takes care of memory management and with its more compact syntax makes your code more readable.

Recent versions of OpenCV-Python can import/export matrices from/to Numpy arrays. That's why in C++, when you want to inject an OpenCV matrix to the Python environment, you will most likely have to deal with the Numpy C API. And there is a gotcha regarding the Numpy C API: Calling the macro import_array() only once is not enough if your Python interaction code spans multiple C++ compile units, i.e., cpp file. You either need to call import_array() in each cpp file or you need to use the #define statements PY_ARRAY_UNIQUE_SYMBOL and NO_IMPORT_ARRAY as explained in the Numpy documentation.

Mat vs. GpuMat vs. UMat

OpenCV stores images data in matrix form. In addition to the core Mat class, there are two matrix classes that you will be potentially dealing with.

GpuMat is defined in <opencv/core/cuda.hpp> and represents image data stored in the memory space of a CUDA-capable parallel processing hardware (graphics processing unit). To convert from or into a Mat object use the GpuMat methods 

void GpuMat::upload(InputArray m)
void GpuMat::download(OutputArray m)

The full class documentation can be found here.

UMat is defined in <opencv2/core/mat.hpp> and bridges Mat to the image data structure used in the OpenCL-accelerated functions of OpenCV (make sure OpenCV was built with the flag WITH_OPENCL set to ON). Support for UMat is integrated much deeper into OpenCV functions than for GpuMat. To convert from or into a Mat object use 

cv::UMat m = cv::Mat::getUMat(int accessFlags, UMatUsageFlags usageFlags) 
cv::Mat m = cv::UMat::getMat(int flags)

The full documentation can be found here.

Retrieved from "https://Robo.Fish/wiki/index.php?title=OpenCV&oldid=2923"
Debug data: