OpenCV C++ API

In this section, I will introduce you some fundamental concepts of OpenCV C++ API. These concepts will help you to understand and write OpenCV C++ example programs in this tutorial more easily.


Header files


You will need to include only the opencv2/opencv.hpp header file in your program. That header file will include all the other necessary header files for your application. Therefore you don't need to bother thinking which header files should be included for your program any more.
e.g. -
#include <opencv2/opencv.hpp>


Namespace


All OpenCV classes and functions are in cv namespace. So, you have to do one of following

  • Add the using namespace cv; line just after including header files (I have used this method in all my example programs in this tutorial)
e.g. -
#include <opencv2/opencv.hpp> 
using namespace cv;
 
int main(int argc, char** argv) 
{ 
 // Read the image file 
 Mat image = imread("D:/My OpenCV Website/Eagle.jpg"); 
 return 0; 
}


  • Append the cv:: specifier at the beginning of every OpenCV classes, functions and data structures in your source code
e.g. -
#include <opencv2/opencv.hpp> 

int main(int argc, char** argv) 
{ 
 // Read the image file 
 cv::Mat image = cv::imread("D:/My OpenCV Website/Eagle.jpg"); 
 return 0; 
}


Data Types of Arrays


Data type of an array defines the number of channels, the number of bits allocated for each element and how the value of an element is represented using those bits. If an array represents an image, each elements of the array are pixels of the image.

Any single channel array should belong to one of following data types.
  • CV_8U - 8 bit unsigned integer
  • CV_8S - 8 bit signed integer
  • CV_16U - 16 bit unsigned integer
  • CV_16S - 16 bit signed integer
  • CV_32S - 32 bit signed integer
  • CV_32F - 32 bit floating point number
  • CV_64F - 64 bit float floating point number

Here I have illustrated an array of which the data type is CV_8U. It has one channel. Each element in the channel is 8 bit unsigned integers. Hence each element should have a value range from 0 to 255. A common analogy example for a single channel array is a black and white image. (Pixel value 0 represents black and 255 represents white. Pixel values in between 0 and 255 represents color in between black and white.)

Array of which the data type is CV_8U
Array of which the data type is CV_8U


We can define all of above data types for multi channel arrays. OpenCV supports up to 512 channels. Here I am going to show you how to define CV_8U data type for multi channel arrays.
  • CV_8UC1 - Single channel array with 8 bit unsigned integers which is exactly same as CV_8U
  • CV_8UC2 - 2 channel array with 8 bit unsigned integers
  • CV_8UC3 - 3 channel array with 8 bit unsigned integers
  • CV_8UC4 - 4 channel array with 8 bit unsigned integers
  • CV_8UC(n) - n channel array with 8 bit unsigned integers (n can be from 1 to 512) )

In the same way, you can derive multi channel data types using any other single-channel data types. (e.g. - CV_16SC3, CV_32FC4, CV_64FC(27), etc)


Example 1:

Here I have illustrated an array of which the data type is CV_8UC3. It has 3 channels. Each element in each channel is 8 bit unsigned integers. Hence each element should have a value range from 0 to 255. Because this is a 3 channel array, array consists of tuples with 3 elements. The first tuple is {54, 0, 34}, second tuple is {58, 78, 185} and so on. A common analogy example for a 3 channel array is a RGB image which consists of red, green and blue channels.

Array of which data type is CV_8UC3
Array of which data type is CV_8UC3


Example 2:

Here I have illustrated an array of which the data type is CV_8SC2. It has 2 channels. Each element in each channel is 8 bit signed integers. Hence each element should have a value range from -128 to 127. Because this is a 2 channel array, array consists of tuples with 2 elements. The first tuple is {-85, -127}, second tuple is {25, 23} and so on.

Array of which data type is CV_8SC2
Array of which data type is CV_8SC2


Example Usage :

  • Mat img1(3, 5, CV_32F ); //Creating a 3 x 5 single-channel array with 32 bit floating point numbers
  • Mat img2(23, 53, CV_64FC(5) ); //Creating a 23 x 53 5-channel array with 64 bit floating point numbers
  • Mat img3(Size(100, 200), CV_16UC2 ); //100 x 200 2-channel array with 16 bit unsigned integers


Note :

  1. After reading this tutorial, it should be obvious to you that the following 3 datatypes are exactly the same.
      • CV_8U
      • CV_8UC1
      • CV_8UC(1)

  2. Although CV_32FC4 is a valid data type, CV_32FC5 is not a valid data type. For arrays which consist of more than 4 channels, parentheses should be used to enclose the channel number. e.g. - CV_32FC(5).
  3. Some OpenCV functions can handle only a subset of above data types. So, read the documentation before using OpenCV functions.



Some Insight into Depth and Channels of Images


Any digital image consists of pixels. Every pixel should have some value. The minimum value for a pixel is 0 and it represents black.When the value of the pixel is increased, the intensity of that pixel is also increased. The maximum value which can be assigned for a pixel depends on the number of bits allocated for each pixels. If the number of bits allocated per pixel is 8, then the maximum value of that pixel is 255 (11111111 in binary)

Now what is depth of an image? The image depth means the number of bits allocated for each pixel. If it is 8, each pixel can have a value between 0 and 255. If it is 4, each pixel can have a value between 0 to 15 (1111 in binary).


Gray-scale Image


Here is a simple model of an image with a depth of 8 bits. Each small box represents a pixel. So, each box may contain a value between 0 to 255.

Gray-scale Image with a depth of 8
Gray-scale Image with a depth of 8



Here is some important properties of the above image.
  • Image depth is 8 bits.
  • Image consists of single channel.
  • The height of the image is 4 pixel.
  • The width of the image is 5 pixels.
  • The resolution of this image is 5 x 4.

This is a gray-scale image (black and white image) because it only consists of a single channel. Therefore this image does not contain any color information. If the value of this pixel is higher, it will be shown more brighter. If the value is low, it will be shown more darker.


Color Image


Following image is a simple model of a color image. Color image should consist of at least 3 planes; Red, Green and Blue. Any pixel is a combination of the three values. Any color can be created by combining these 3 basic colors.

Examples -
  • (255, 0, 0) represent pure red. 
  • (0, 255, 0) represent pure green. 
  • (0, 0, 255) represents pure blue. 
  • (255, 0, 255) represents pure violate. 

R, G, B planes of a color image
R, G, B planes of a color image

In the above image, top left pixel is (23, 231, 46). It will be shown as a greenish color because the green value (231) of that pixel is significantly larger than the red (23) and blue (46) value.

Here is some important properties of the above image.
  • Image depth is 24 bits. (because each pixel is represented with 8 x 3 bits (8 bits from each channel)
  • The image consists of 3 channels.
  • The height of the image is 4 pixel. 
  • The width of the image is 5 pixels.
  • The resolution of this image is 5 x 4.

Note -
OpenCV library functions usually read images in BGR format which means blue plane first, green color plane next and the red plane at the end which is exactly the reverse order of the above image.



Next Lesson : Load & Display Image



Load & Display Image


Load Image from File and Display


In this section, I will show you how to load an image from a file and display the image in a window using OpenCV library functions.

First of all, open your C++ IDE and create a new project. Then you have to configure the new project for OpenCV. If you have not installed OpenCV or configured the visual studio projects for OpenCV yet, please refer to Install OpenCV with Visual Studio.

//Uncomment the following line if you are compiling this code in Visual Studio
//#include "stdafx.h"

#include <opencv2/opencv.hpp>
#include <iostream>

using namespace cv;
using namespace std;

int main(int argc, char** argv)
{
 // Read the image file
 Mat image = imread("D:/My OpenCV Website/Lady with a Guitar.jpg");

 // Check for failure
 if (image.empty()) 
 {
  cout << "Could not open or find the image" << endl;
  cin.get(); //wait for any key press
  return -1;
 }

 String windowName = "The Guitar"; //Name of the window

 namedWindow(windowName); // Create a window

 imshow(windowName, image); // Show our image inside the created window.

 waitKey(0); // Wait for any keystroke in the window

 destroyWindow(windowName); //destroy the created window

 return 0;
}


Copy and paste above code snippet into your IDE and run it. Please note that you have to replace "D:/My OpenCV Website/Lady with a Guitar.jpg" in the code with a valid location to an image in your computer. Then you should see a output like the below image.


Loading and Displaying an image with OpenCV


Explanation


Let's go through the above OpenCV program line by line.


//#include "stdafx.h" 
This is the pre-compiled header file generated by Visual Studio IDE. If you are using Visual Studio, please uncomment this line to avoid a compile error.



#include <opencv2/opencv.hpp>
#include <iostream>
opencv2/opencv.hpp header file itself will include all other neccessary header files in OpenCV library. You have to include only this header file in order to compile your OpenCV codes. For more information, please refer OpenCV C++ API.

iostream header file should also be included because we use cout and cin in our program to print an error message to the console.



using namespace cv;
using namespace std;
All OpenCV functions, classes and data structures are declared inside cv namespace. So, we have to add the 'using namespace cv' line in the top of our program. Otherwise we have to append 'cv::' specifier before each OpenCV functions, classes and data structures. (e.g - cv::Mat, cv::imread() , etc). For more information, please refer OpenCV C++ API.

We have to use the std namespace also because cout, endl and cin functions are inside the std namespace.



Mat image = imread("D:/My OpenCV Website/Lady with a Guitar.jpg");
The function loads an image from the file "D:/My OpenCV Website/Lady with a Guitar.jpg" and returns it as a Mat object.
In your code, you have to replace the "D:/My OpenCV Website/Lady with a Guitar.jpg" with a valid location to an image file in your computer.

Mat imread(const String& filename, int flags = IMREAD_COLOR)

This function loads an image from the specified file and return is as a Mat object. If the function cannot read the file, it will return an empty Mat object.
  1. filename - You have to give the relative or absolute path of an image file. If you are giving the relative path, it should be relative to your cpp file. jpeg, jpg, bmp, png, tiff and tif image file types are always supported. Other image file types are supported depending on your platform and installed codecs.
  2. flags - There are several possible values for the flag argument. In the above program, I did not pass any value to this argument such that default IMREAD_COLOR argument will be used.
    • IMREAD_UNCHANGED - The image will be loaded as it is. If you want to get the alpha channel in your input image (if it is available), you have to use this flag.
    • IMREAD_GRAYSCALE - The image will be load as a gray-scale image (i.e. - Single channel image, Black and white image)
    • IMREAD_COLOR - The image will be loaded as a BGR image (i.e. - 3 channel image, color image)



 // Check for failure
 if (image.empty()) 
 {
   cout << "Could not open or find the image" << endl;
   cin.get(); //wait for any key press
   return -1;
 }
If imread() function fails to load the image, the returned Mat object will be empty. If the Mat object is empty, image.empty() function will return true. In such scenarios, our program will print an error message to the console and wait for any key press. When the user press any key in the keyboard, the program will exit returning -1. It's a good practice to check whether the image is empty and exit the program. Otherwise your program will crash while trying to execute the imshow() function.



namedWindow(windowName); // Create a window

This function creates a window with the name of "The Guitar". The name of the window is used later in this code to identify the window.

void namedWindow(const String& winname, int flags = WINDOW_AUTOSIZE)

This function creates a window which can be used to place images and track bars. If a window already exists with the given name, this function does nothing.
  1. winname - Name of the window. That name will display in the title bar of the newly created window. This name is also the identifier for this window and it will be used in the later OpenCV function calls to identify the window.
  2. flags - Determine the size of the window. In the above program, I did not pass any value to this argument such that default WINDOW_AUTOSIZE argument will be used.
    • WINDOW_AUTOSIZE - User cannot resize the window. Image will be displayed in its original size. 
    • WINDOW_NORMAL- User can resize the window.



imshow(windowName, image); // Show our image inside the created window.
This function display the image in the window created in a previous step.

void imshow(const String& winname, InputArray mat)


This function shows the image in a window specified by winname. If the window is created with WINDOW_AUTOSIZE flag, image will be displayed in its original size. Otherwise image may be scaled to the size of the window. If the window has not been created by calling to namedWindow() function, this function will create a window with the WINDOW_AUTOSIZE flag.

This function call should be followed by waitKey(int) function call in order to provide sufficient time to paint and display the image in the window for the specified time duration in milliseconds. If you do not call waitKey(int) function, the image will not be displayed in the window.
  1. winname -  Name of the window which created by namedWindow() function.
  2. mat - Mat object which holds the image



waitKey(0); // Wait for any keystroke in the window
waitKey(0) function waits for a key press forever. When any key is pressed, this function returns the ASCII value of the key and your program will continue.

If positive value is passed to the waitKey() function, it waits for a key press for only the time duration, specified by passed value in milliseconds. If any key is pressed during this period, this function returns the ASCII value of the key and your program will continue. If no key is pressed during this period, it will return -1 and program will continue.

This function only works if there is at least one active HIGHGUI window opened by the program.



destroyWindow(windowName); //destroy the created window
This function closes the opened window identified with the name, windowName and de-allocate any associated memory usage. This function is not essential for the above program because when the program exits, operating system usually close all the opened windows and de-allocate any associated memory usage.



Summary


In the above section, you have learned,

  • How to load an image from a file
  • How to handle the error scenario when the loading of the image is failed
  • How to create a window and display the image
  • How to wait without exiting the program until the user press a key
  • How to destroy created windows




Create a Blank Image & Display


The following program is also very much similar to the previous program. The only difference is that this program creates a blank image instead of loading an existing image from a file.

//Uncomment the following line if you are compiling this code in Visual Studio
//#include "stdafx.h"

#include <opencv2/opencv.hpp>
#include <iostream>

using namespace cv;
using namespace std;

int main(int argc, char** argv)
{
 //create a new image which consists of 
 //3 channels 
 //image depth of 8 bits 
 //800 x 600 of resolution (800 wide and 600 high)
 //each pixels initialized to the value of (100, 250, 30) for Blue, Green and Red planes respectively.
 Mat image(600, 800, CV_8UC3, Scalar(100, 250, 30)); 
 
 String windowName = "Window with Blank Image"; //Name of the window

 namedWindow(windowName); // Create a window

 imshow(windowName, image); // Show our image inside the created window.

 waitKey(0); // Wait for any keystroke in the window

 destroyWindow(windowName); //destroy the created window

 return 0;
}

Copy and paste above simple code snippet into your IDE and run it. Then you should see a output like the below image.


Creating a blank image and display



Explanation for New OpenCV Functions Found in this Section


Mat image(600, 800, CV_8UC3, Scalar(100, 250, 30)); 
This is one of many constructors available in Mat class. It creates an image which is 600 pixels high and 800 pixels wide. 24 bits are allocated for each pixel in the image. 24 bits will consist of three unsigned 8 bit integers representing blue, green and red planes respectively. The value of the three integers should be from 0 to 255.

Then this constructor initializes each pixels of the created image with the 3 unsigned integers (100, 250, 30). So, it initializes the blue channel with 100, green channel with 250 and red channel with 30. Because the value of the green channel is significantly larger than that of other channels, the output image is greenish. You can try different combinations of these three values and see the output image.

Mat::Mat(int rows, int cols, int type, const Scalar& s)

This constructor will create a Mat object with specified number of rows and number of cols and initialize each element with the value given in s.
  1. rows - Number of rows in the 2D array ( i.e. - height of the image in pixels )
  2. cols - Number of columns in the 2D array ( i.e. - width of the image in pixels )
  3. type - Data type of the 2D array which specify the depth and the data type of each element in each channel and number of channels. (Of course, if number of channels is more than 1, 2D array becomes 3D array). More details of data types can be found in the OpenCV C++ API. Possible values for this argument are,
    • CV_8UC1
    • CV_8UC2
    • CV_8UC3
    • CV_8UC4
    • CV_8UC(n)
    •  
    • CV_8SC1
    • CV_8SC2
    • CV_8SC3
    • CV_8SC4
    • CV_8SC(n)
    •  
    • CV_16UC1
    • CV_16UC2
    • CV_16UC3
    • CV_16UC4
    • CV_16UC(n)
    •  
    • CV_16SC1
    • CV_16SC2
    • CV_16SC3
    • CV_16SC4
    • CV_16SC(n)
    •  
    • CV_32SC1
    • CV_32SC2
    • CV_32SC3
    • CV_32SC4
    • CV_32SC(n)
    •  
    • CV_32FC1
    • CV_32FC2
    • CV_32FC3
    • CV_32FC4
    • CV_32FC(n)
    •  
    • CV_64FC1
    • CV_64FC2
    • CV_64FC3
    • CV_64FC4
    • CV_64FC(n)
  4. s - Initialize each array element (pixels) with the value given by s for each channel. 

Summary

In the above section, you have learned how to create an image of desired size and type and initialize all pixels to a specified value.



Next Tutorial : Play Video from File or Camera



Install OpenCV with Visual Studio


In this lesson, I will show you how to install OpenCV 3.3.1 with Visual Studio 2015 on Microsoft Windows 7. But it is almost same for other versions of OpenCV, Microsoft Visual Studio and Microsoft Windows.

If you want, you can refer to my older posts

There are 2 ways to install OpenCV into your computer. One way is to build the OpenCV from the latest source in the GitHub and use it. But for a beginner, the most suitable way is to use OpenCV pre-built libraries. Therefore I am going to discuss with you how to install and configure OpenCV pre-built libraries with your the most popular C++ IDE, Visual Studio.



Finding the System Type of your Windows 7 Computer


Right click on My Computer icon on your Desktop and click Properties in the context menu. Then you will see a window like this.

Finding the System Type in the Properties of My Computer


Here the system type is 64 bit operating system. So the system architecture is x64. If your system type is 32-bit operating system, please refer my old post Installing & Configuring OpenCV 2.4.5 with Visual Studio 2012.



What you should download


First of all, you should have to download Microsoft Visual Studio 2015, if it is not installed in your computer. I download Visual Studio Community 2015 which is a free, fully-featured Visual Studio edition for students, open-source and individual developers for their commercial and non-commercial development activities. (You will need to register by giving your email address to obtain the free product key).

Then you have to download the OpenCV pre-built, self-extracting executable on Microsoft Windows from SourceForge. You can choose whatever version, you want. But I choose OpenCV 3.3.1 which was the latest stable release when I wrote this tutorial.



Installation


First you have to Install Visual Studio 2015 if it is not installed in your computer. Because this step is straightforward, I'm not going to explain it here.

Then double click the downloaded opencv-3.3.1-vc14.exe file which was downloaded from SourceForge and OpenCV pre-built libraries will be extracted to whatever location, you give. I gave C:\ as the Extract To location in the popped up dialog box.

The default compiler type of the Microsoft Visual Studio 2015 is VC14. And the opencv-3.3.1-vc14.exe file also contains OpenCV libraries, built with VC14 compiler. Therefore if you are using the OpenCV 3.3.1 downloaded from SourceForge and Visual Studio 2015, you are good to go.

You can find supporting compiler types of the installed OpenCV pre-built libraries, if you go to C:\opencv\build\x64. In this particular OpenCV pre-built version, the only supporting compiler type is VC14.

If you want to use a newer version of Microsoft Visual Studio, you should change the compiler type of your Visual Studio project to VC14. Please refer this page on MSDN.

The next step is to set up environment variables correctly in your computer.



Set up Environment Variables


  • Open start menu and search for 'edit environment variables' in the search bar in the bottom of the start menu. 

Opening the Environment Variables Window from the Start Menu



  • Then click the 'Edit the system environment variables' link in the top of the search results as highlighted in the above image to open the 'System Properties' dialog box. 
  • In the 'System Properties' dialog box, click the 'Environment Variables...' button window as highlighted in the below image to open the 'Environment Variables' dialog box. 


Opening the Environment Variables Window from the System Properties window



  • Click 'New' button at the bottom of the Environment Variables window to create a new environment variable.

Creating a new Environment Variable



  • Type OPENCV_DIR against Variable name field and type the location C:\opencv\build\ against the Variable value field in the popped up dialog box. Please note that I have installed OpenCV pre-built libraries into C: location. If you have used another location as the OpenCV installation location, the Variable value field should be updated accordingly. (e.g. OPENCV_DIR=<OpenCV installation location>\opencv\build\). 

Inserting an Environment Variable Name and Value



Hooray!! You have completed the installation of OpenCV. Next thing you have to do is to configure Visual Studio to use OpenCV pre-built libraries in your computer vision application.



Configure Visual Studio Project


  • Start Microsoft Visual Studio 
  • Go to menu item 'File' > 'New' > 'Project...' to open the 'New Project' window 
  • Do everything as highlighted in the below image and click OK. 

Creating a new Visual Studio Project



  • Click 'Next' button in the popped up 'Win32 Application Wizard' dialog box. 
  • Then click 'Finish' button in the next window as shown below. 

Finished Creating a new Visual Studio Project



  • Then right click on the project name (you have entered this name in a previous step) on the solution explorer and then click 'Properties' in the opened context menu to open the Property Pages window. 

Opening the Property Pages of the Visual Studio Project



  • In Property Pages window, choose 'All Configurations' as the configuration and 'x64' as the Platform as shown in the below image. 

Selecting the Configuration and Platform in the Property Pages window



  • Click the 6 places in the given order in the Property Pages window as shown in the below image. Then copy and paste $(OPENCV_DIR)\include in the opened dialog box and click OK. 

Opening the dialog box to update Additional Include Directories



Updating Additional Include Directories



  • Click the 6 places in the given order in the Property Pages window as shown in the below image. Then copy and paste $(OPENCV_DIR)\x64\vc14\lib in the opened dialog box and click OK. 

Opening the dialog box to update Additional Library Directories


Update Additional Library Directories



  • Click the 5 places in the given order in the Property Pages window as shown in the below image. Then copy and paste PATH=$(OPENCV_DIR)\x64\vc14\bin;%PATH% in the opened dialog box and click OK. 

Opening the dialog box to update Environment Variables of the Visual Studio Project


Update Environment Variables of the Visual Studio Project



  • Click the Configuration Manager in the top right corner of the Property Pages window to open the Configuration Manager dialog box. 

Opening the Configuration Manager of the Visual Studio Project



  • In the Configuration Manager dialog box, set the solution platform to x64 in both locations as highlighted in the below image. Then click Close button to close the Configuration Manager dialog box. 

Updating the Platform of the Visual Studio project in the Configuration Manager Window



  • Change the Configuration of the Property Pages window to 'Debug' as shown in the below image. 

Changing the Configuration of the Property Pages window



  • Click the 6 places in the given order in the Property Pages window as shown in the below image. Then copy and paste the opencv_world331d.lib in the opened dialog box and click OK. Please verify that this filename can be found in the %OPENCV_DIR%\x64\vc14\lib location. 

Opening the Dialog box to update the Additional Dependencies of the Visual Studio Project


Updating the Additional Dependencies of the Visual Studio Project



  • Change the Configuration of the Property Pages window to 'Release' as shown in the below image. 

Changing the Configuration of the Property Pages to Release



  • Click the 6 places in the given order in the Property Pages window as shown in the below image. Then copy and paste the opencv_world331.lib in the opened dialog box and click OK. Please verify that this filename can be found in the %OPENCV_DIR%\x64\vc14\lib location. 

Opening the Dialog box to update the Additional Dependencies of the Release Configuration


Updating the Additional Dependencies of the Release Configuration



  • Click OK button in the bottom right corner of the Property Pages window to save all configurations you have done in previous steps. 

Pressing OK button to save the changes done in Property Pages window



Congratulations!! You have done with all of the configurations of your visual studio IDE. Now you are ready to write your first HelloWorld OpenCV application.



Your First OpenCV HelloWorld Application


#include "stdafx.h"

#include <opencv2/opencv.hpp>

#include <iostream>
#include <string>

using namespace cv;
using namespace std;

int main(int argc, char** argv)
{
 // Read the image file
  Mat image = imread("D:/My OpenCV Website/Eagle.jpg");

  if (image.empty()) // Check for failure
  {
   cout << "Could not open or find the image" << endl;
   system("pause"); //wait for any key press
   return -1;
  }

  String windowName = "My HelloWorld Window"; //Name of the window

  namedWindow(windowName); // Create a window

  imshow(windowName, image); // Show our image inside the created window.

  waitKey(0); // Wait for any keystroke in the window

  destroyWindow(windowName); //destroy the created window

  return 0;
}


Copy and paste above simple code snippet into the text area of your visual studio project and run it. Please note that you have to replace "D:/My OpenCV Website/Eagle.jpg" in the code with a valid location to an image in your computer. Then you should see a output like the below image.

(If this is not working, try restarting the Visual Studio IDE)


Output of the First HelloWorld OpenCV application in Visual Studio



Next Lesson : OpenCV C++ API