OpenCV Tutorial C++

How to Detect Mouse Clicks and Moves

OpenCV supports for detecting mouse events. Mouse events include mouse clicks and movements over an attached OpenCV window.

OpenCV Example Code

It is very simple to do that. All you have do is to define a callback function in the OpenCV C++ code attaching to the OpenCV window. That callback function will be called every time, mouse events occur. That callback function will also give the coordinates of the mouse events. (e.g - (x, y) coordinate of a mouse click).

Here is the simple OpenCV code to detect left, right and middle mouse clicks and mouse movements with its coordinates

////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
#include "opencv2/highgui/highgui.hpp"
#include <iostream>

using namespace std;
using namespace cv;

void CallBackFunc(int event, int x, int y, int flags, void* userdata)
{
if ( event == EVENT_LBUTTONDOWN )
{
cout << "Left button of the mouse is clicked - position (" << x << ", " << y << ")" << endl;
}
else if ( event == EVENT_RBUTTONDOWN )
{
cout << "Right button of the mouse is clicked - position (" << x << ", " << y << ")" << endl;
}
else if ( event == EVENT_MBUTTONDOWN )
{
cout << "Middle button of the mouse is clicked - position (" << x << ", " << y << ")" << endl;
}
else if ( event == EVENT_MOUSEMOVE )
{
cout << "Mouse move over the window - position (" << x << ", " << y << ")" << endl;

}
}

int main(int argc, char** argv)
{
// Read image from file
Mat img = imread("MyPic.JPG");

//if fail to read the image
if ( img.empty() )
{
cout << "Error loading the image" << endl;
return -1;
}

//Create a window
namedWindow("My Window", 1);

//set the callback function for any mouse event
setMouseCallback("My Window", CallBackFunc, NULL);

//show the image
imshow("My Window", img);

// Wait until user press some key
waitKey(0);

return 0;

}

////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

You can download this OpenCV visual C++ project from here

Detect Mouse Clicks and Moves Over the OpenCV Window

Summary

In the above OpenCV sample code, "CallbackFunc" function will be called on any mouse event (Moving a mouse over the attached OpenCV window is also a mouse event). By using suitable if - else blocks, I printed only left, right and middle mouse clicks and mouse movements over the window.

Here are new OpenCV functions, found in the above example code. If you are not familiar with the other OpenCV functions as well, please go through the other lessons in this tutorial.

void setMouseCallback(const string& winname, MouseCallback onMouse, void* userdata = 0)

This function sets a callback function to be called every time any mouse events occurs in the specified window. Here is the detailed explanation of the each parameters of the above OpenCV function.

winname - Name of the OpenCV window. All mouse events related to this window will be registered
onMouse - Name of the callback function. Whenever mouse events related to the above window occur, this callback function will be called. This function should have the signature like the following

void FunctionName(int event, int x, int y, int flags, void* userdata)

event - Type of the mouse event. These are the entire list of mouse events

EVENT_MOUSEMOVE
EVENT_LBUTTONDOWN
EVENT_RBUTTONDOWN
EVENT_MBUTTONDOWN
EVENT_LBUTTONUP
EVENT_RBUTTONUP
EVENT_MBUTTONUP
EVENT_LBUTTONDBLCLK
EVENT_RBUTTONDBLCLK
EVENT_MBUTTONDBLCLK

x - x coordinate of the mouse event
y - y coordinate of the mouse event
flags - Specific condition whenever a mouse event occurs. See the next OpenCV example code for the usage of this parameter. Here is the entire list of enum values which will be possesed by "flags"

EVENT_FLAG_LBUTTON
EVENT_FLAG_RBUTTON
EVENT_FLAG_MBUTTON
EVENT_FLAG_CTRLKEY
EVENT_FLAG_SHIFTKEY
EVENT_FLAG_ALTKEY

userdata - Any pointer passes to the "setMouseCallback" function as the 3rd parameter (see below)

userdata - This pointer will be passed to the callback function

OpenCV Example to Detect Mouse Clicks While Pressing a Key

I am going to explain you how to detect a mouse event while pressing a key of the keyboard.

The following OpenCV example code will detect left mouse clicks while pressing the "CTRL" key , right mouse clicks while pressing the "SHIFT" key and movements of the mouse over the OpenCV window while pressing the "ALT" key.

/////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
#include "opencv2/highgui/highgui.hpp"
#include <iostream>

using namespace std;
using namespace cv;

void CallBackFunc(int event, int x, int y, int flags, void* userdata)
{
if ( flags == (EVENT_FLAG_CTRLKEY + EVENT_FLAG_LBUTTON) )
{
cout << "Left mouse button is clicked while pressing CTRL key - position (" << x << ", " << y << ")" << endl;
}
else if ( flags == (EVENT_FLAG_RBUTTON + EVENT_FLAG_SHIFTKEY) )
{
cout << "Right mouse button is clicked while pressing SHIFT key - position (" << x << ", " << y << ")" << endl;
}
else if ( event == EVENT_MOUSEMOVE && flags == EVENT_FLAG_ALTKEY)
{
cout << "Mouse is moved over the window while pressing ALT key - position (" << x << ", " << y << ")" << endl;
}
}

int main(int argc, char** argv)
{
// Read image from file
Mat img = imread("MyPic.JPG");

//if fail to read the image
if ( img.empty() )
{
cout << "Error loading the image" << endl;
return -1;
}

//Create a window
namedWindow("My Window", 1);

//set the callback function for any mouse event
setMouseCallback("My Window", CallBackFunc, NULL);

//show the image
imshow("My Window", img);

// Wait until user press some key
waitKey(0);

return 0;
}
/////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

You can download this visual C++ OpenCV project from here.

No new OpenCV functions in the above example. If you have some doubt about any OpenCV functions in the above example, please refer to previous lessons.

Next Tutorial : Rotate Image & Video

Previous Tutorial : How to Add Trackbar

Rotate Image & Video

Rotate Image

Rotating images by a given angle is a common image processing task. Although it seems little bit complicated, OpenCV provides some built-in functions making it easy to do it. Here is a simple OpenCV C++ example code to rotate an image. Here I use a track bar to change the rotating angle dynamically.

////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
#include "opencv2/highgui/highgui.hpp"
#include "opencv2/imgproc/imgproc.hpp"

using namespace cv;

int main( int argc, char** argv )
{
// Load the image
Mat imgOriginal = imread( "MyPic.JPG", 1 );

//show the original image
const char* pzOriginalImage = "Original Image";
namedWindow( pzOriginalImage, CV_WINDOW_AUTOSIZE );
imshow( pzOriginalImage, imgOriginal );

const char* pzRotatedImage = "Rotated Image";
namedWindow( pzRotatedImage, CV_WINDOW_AUTOSIZE );

int iAngle = 180;
createTrackbar("Angle", pzRotatedImage, &iAngle, 360);

int iImageHieght = imgOriginal.rows / 2;
int iImageWidth = imgOriginal.cols / 2;

while (true)
{
Mat matRotation = getRotationMatrix2D( Point(iImageWidth, iImageHieght), (iAngle - 180), 1 );

// Rotate the image
Mat imgRotated;
warpAffine( imgOriginal, imgRotated, matRotation, imgOriginal.size() );

imshow( pzRotatedImage, imgRotated );

int iRet = waitKey(30);
if ( iRet == 27 )
{
break;
}
}

return 0;
}
///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
You can download this visual C++ project from here.

Original Image

Rotated Image

Explanation

Here is the explanation for new OpenCV functions which are not found in previous lessons.

Mat getRotationMatrix2D( Point2f center, double angle, double scale )

This function returns 2x3 affine transformation matrix for the 2D rotation.

Arguments -

center - The center of the rotation of the the source image.
angle - Angle of rotation in degrees (Positive values for counter-clockwise direction and negative values for clockwise rotation)
scale - The scaling factor of the image. (Scaling factor of 1 means its original size)

Try different values for center, angle and scale and observe the output image.

void warpAffine( InputArray src, OutputArray dst, InputArray M, Size dsize, int flags = INTER_LINEAR, int bordreMode=BORDER_CONSTANT, const Scalar& borderValue=Scalar() )

This OpenCV function applies affine transformation to an image.

Arguments -

src - Source Image
dst - Destination image which should have the same type as the source image(The transformed image is stored in this location)
M - 2x3 affine transformation matrix
dsize - Size of the destination image
flags - Interpolation methods
borderMode - pixel extrapolation method. (Try these values; BORDER_REPLICATE, BORDER_CONSTANT, BORDER_REFLECT, BORDER_WRAP, BORDER_REFLECT_101, BORDER_TRANSPARENT and BORDER_ISOLATED)
borderValue - If you use BORDER_CONSTANT for borderMode, this argument define the value used for the border

Another OpenCV Example to Rotate an Image

Here is another way to rotate an image. Here I use a callback function to apply the rotation instead of using a infinite while loop. Other than the rotation, you can change the scale of the image and border extrapolation method dynamically.

Here is the OpenCV C++ code.

///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
#include "opencv2/highgui/highgui.hpp"
#include "opencv2/imgproc/imgproc.hpp"

using namespace cv;

int iAngle = 180;
int iScale = 50;
int iBorderMode = 0;
Mat imgOriginal ;
int iImageCenterY = 0;
int iImageCenterX = 0;
const char* pzRotatedImage = "Rotated Image";

void CallbackForTrackBar(int, void*)
{
Mat matRotation = getRotationMatrix2D( Point( iImageCenterX, iImageCenterY ), (iAngle - 180), iScale / 50.0 );

// Rotate the image
Mat imgRotated;
warpAffine( imgOriginal, imgRotated, matRotation, imgOriginal.size(), INTER_LINEAR, iBorderMode, Scalar() );

imshow( pzRotatedImage, imgRotated );

}

int main( int argc, char** argv )
{
// Load the image
imgOriginal = imread( "MyPic.JPG", 1 );

iImageCenterY = imgOriginal.rows / 2;
iImageCenterX = imgOriginal.cols / 2;

//show the original image
const char* pzOriginalImage = "Original Image";
namedWindow( pzOriginalImage, CV_WINDOW_AUTOSIZE );
imshow( pzOriginalImage, imgOriginal );

//create the "Rotated Image" window and 3 trackbars in it
namedWindow( pzRotatedImage, CV_WINDOW_AUTOSIZE );
createTrackbar("Angle", pzRotatedImage, &iAngle, 360, CallbackForTrackBar);
createTrackbar("Scale", pzRotatedImage, &iScale, 100, CallbackForTrackBar);
createTrackbar("Border Mode", pzRotatedImage, &iBorderMode, 5, CallbackForTrackBar);

int iDummy = 0;

CallbackForTrackBar(iDummy, &iDummy);

waitKey(0);

return 0;
}
///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

You can download this visual C++ project from here.

Rotated Image which is down-scaled and whose border values have been extrapolated

All the OpenCV functions have been discussed previously.

Rotate a Video

Rotating a video is also simple. The code is just like the 1st example in this lesson. Here is the OpenCV C++ code.

///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
#include "opencv2/highgui/highgui.hpp"
#include "opencv2/imgproc/imgproc.hpp"
#include <iostream>

using namespace cv;
using namespace std;

int main(int argc, char* argv[])
{
// open the video file for reading
VideoCapture cap("C:/Users/SHERMAL/Desktop/MyVideo.mp4");

// if not success, exit program
if ( !cap.isOpened() )
{
cout << "Cannot open the video file" << endl;
return -1;
}

const char* pzOriginalWindowName = "Original Video";
namedWindow(pzOriginalWindowName, CV_WINDOW_AUTOSIZE);

const char* pzRotatingWindowName = "Rotated Video";
namedWindow( pzRotatingWindowName, CV_WINDOW_AUTOSIZE );

int iAngle = 180;
createTrackbar("Angle", pzRotatingWindowName, &iAngle, 360);

while (true)
{
Mat matOriginalFrame;

// read a new frame from video
bool bSuccess = cap.read(matOriginalFrame);

//if not success, break loop
if (!bSuccess)
{
cout << "Cannot read the frame from video file" << endl;
break;
}

imshow(pzOriginalWindowName, matOriginalFrame);

//get the affine transformation matrix
Mat matRotation = getRotationMatrix2D( Point(matOriginalFrame.cols / 2, matOriginalFrame.rows / 2), (iAngle - 180), 1 );

// Rotate the image
Mat matRotatedFrame;
warpAffine( matOriginalFrame, matRotatedFrame, matRotation, matOriginalFrame.size() );

imshow( pzRotatingWindowName, matRotatedFrame );

//wait for 'esc' key press for 30 ms. If 'esc' key is pressed, break loop
if (waitKey(30) == 27)
{
cout << "esc key is pressed by user" << endl;
break;
}
}

return 0;
}
///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
You can download this visual C++ project from here.

All the OpenCV functions, found in the above example have been discussed earlier.

Next Tutorial : Object Detection & Tracking using Color

Previous Tutorial : How to Detect Mouse Clicks and Moves

How to Add Trackbar

Trackbars are very useful in lots of occasions. It enables users to change various parameters while the OpenCV application is running.

( If you have not install and configure OpenCV yet, please refer to Installing & Configuring with Visual Studio. )

Simple Use of Trackbars

Whenever you change the position of a trackbar, the value of an integer variable is changed. Using that value, we can change a property of an image or a video. The following example will show you how to do it with OpenCV.

OpenCV Example of How to Change Brightness and Contrast of an Image with Trackbars

In the following example, I have added two trackbars to change the brightness and contrast of an image. It is iterating in a infinite while loop and applying the brightness and contrast to the image periodically because I want to apply the changes to the image whenever the user changes the position of the trackbar.

//////////////////////////////////////////////////////////////////////////////////////////////////////////////////
#include "opencv2/imgproc/imgproc.hpp"
#include "opencv2/highgui/highgui.hpp"
#include <iostream>

using namespace std;
using namespace cv;

int main( int argc, char** argv )
{
// Read original image
Mat src = imread("MyPic.JPG");

//if fail to read the image
if (!src.data)
{
cout << "Error loading the image" << endl;
return -1;
}

// Create a window
namedWindow("My Window", 1);

//Create trackbar to change brightness
int iSliderValue1 = 50;
createTrackbar("Brightness", "My Window", &iSliderValue1, 100);

//Create trackbar to change contrast
int iSliderValue2 = 50;
createTrackbar("Contrast", "My Window", &iSliderValue2, 100);

while (true)
{
//Change the brightness and contrast of the image (For more infomation http://opencv-srf.blogspot.com/2013/07/change-contrast-of-image-or-video.html)
Mat dst;
int iBrightness = iSliderValue1 - 50;
double dContrast = iSliderValue2 / 50.0;
src.convertTo(dst, -1, dContrast, iBrightness);

//show the brightness and contrast adjusted image
imshow("My Window", dst);

// Wait until user press some key for 50ms
int iKey = waitKey(50);

//if user press 'ESC' key
if (iKey == 27)
{
break;
}
}

return 0;
}
//////////////////////////////////////////////////////////////////////////////////////////////////////////////////

You can download this OpenCV visual c++ example from here.

Explanation of New OpenCV Functions

int createTrackbar(const string& trackbarname, const string& winname, int* value, int count, TrackbarCallback onChange = 0, void* userdata = 0)

This OpenCV function creates a trackbar and attached that trackbar to a specified window

trackbarname - The name of the trackbar
winname - The name of the window to which the trackbar is attached
value - This integer, pointed by this pointer, holds the value associated with the position of the trackbar
count - The maximum value of the trackbar. The minimum value is always zero.
onChange - This function will be called everytime the position of the trackbar is changed. The prototype of this function should be "FunctionName(int, void*)". The "int" value is the value associate with the position of the trackbar. And "void*" is any pointer value which you pass as the "userdata" (See the next parameter).
userdata - This pointer variable will be passed as the second parameter of the above function

All other functions have been discussed in the previous lessons. If you have not followed them yet, please visit

which have the all the other OpenCV functions in the above example code.

Trackbar with Callback Function

In the above example, I have used only 4 parameters for the "createTrackbar" function. But there are 2 more parameters. Here I am going to explain, how to use a callback function using the 5th and 6th parameters of "createTrackbar". The advantage of using the callback function is that it is not required to iterate in a while loop periodically as in the above example.

In the following OpenCV example, I have added two trackbars to change the brightness and contrast of an image. And a callback function is implemented for each trackbar.

///////////////////////////////////////////////////////////////////////////////////////

#include "opencv2/imgproc/imgproc.hpp"
#include "opencv2/highgui/highgui.hpp"
#include <iostream>

using namespace std;
using namespace cv;

Mat src;

void MyCallbackForBrightness(int iValueForBrightness, void *userData)
{
Mat dst;
int iValueForContrast = *( static_cast<int*>(userData) );

//Calculating brightness and contrast value
int iBrightness = iValueForBrightness - 50;
double dContrast = iValueForContrast / 50.0;

//Calculated contrast and brightness value
cout << "MyCallbackForBrightness : Contrast=" << dContrast << ", Brightness=" << iBrightness << endl;

//adjust the brightness and contrast
src.convertTo(dst, -1, dContrast, iBrightness);

//show the brightness and contrast adjusted image
imshow("My Window", dst);
}

void MyCallbackForContrast(int iValueForContrast, void *userData)
{
Mat dst;
int iValueForBrightness = *( static_cast<int*>(userData) );

//Calculating brightness and contrast value
int iBrightness = iValueForBrightness - 50;
double dContrast = iValueForContrast / 50.0;

//Calculated contrast and brightness value
cout << "MyCallbackForContrast : Contrast=" << dContrast << ", Brightness=" << iBrightness << endl;

//adjust the brightness and contrast
src.convertTo(dst, -1, dContrast, iBrightness);

//show the brightness and contrast adjusted image
imshow("My Window", dst);
}

int main(int argc, char** argv)
{
// Read original image
src = imread("MyPic.JPG");

//if fail to read the image
if (src.data == false)
{
cout << "Error loading the image" << endl;
return -1;
}

// Create a window
namedWindow("My Window", 1);

int iValueForBrightness = 50;
int iValueForContrast = 50;

//Create track bar to change brightness
createTrackbar("Brightness", "My Window", &iValueForBrightness, 100, MyCallbackForBrightness, &iValueForContrast);

//Create track bar to change contrast
createTrackbar("Contrast", "My Window", &iValueForContrast, 100, MyCallbackForContrast, &iValueForBrightness);

imshow("My Window", src);

// Wait until user press some key
waitKey(0);

return 0;
}

///////////////////////////////////////////////////////////////////////////////////////

You can download this OpenCV visual c++ project from here.

Trackbar with Callback Function

Explanation

I have used 2 callback functions; "MyCallbackForBrightness(int, void*)" for the "Brightness" trackbar and "MyCallbackForContrast(int, void*)"for the "Contrast" trackbar.

I have used a global variables, src because it should be accessed from all the 3 methods.

Examine closely the 5th and 6th parameters of "createTrackbar" method in the "main" method.

Whenever the position of the "Brightness" trackbar is changed, "MyCallbackForBrightness(int, void*)" will be called. The 1st integer argument holds the value of the position of the "Brightness" trackbar. The position of the "Contrast" trackbar is passed as the 2nd argument. (Observe the 5th and 6th parameter; createTrackbar("Brightness", "My Window", &iValueForBrightness, 100, MyCallbackForBrightness, &iValueForContrast);)

Whenever the position of the "Contrast" trackbar is changed, "MyCallbackForContrast(int, void*)" will be called. The 1st integer argument holds the value of the position of the "Contrast" trackbar. The position of the "Brightness" trackbar is passed as the 2nd argument. (Observe the 5th and 6th parameter; createTrackbar("Contrast", "My Window", &iValueForContrast, 100, MyCallbackForContrast, &iValueForBrightness))

Next Tutorial : How to Detect Mouse Clicks and Moves

Previous Tutorial : Filtering Images

OpenCV, MATLAB or AForge ??

If you are new to computer vision, you may be wondering where to start. First you have to understand the basic principles of image processing and computer vision. Then you have to choose a suitable language to develop your computer vision application. Some of the most popular methods are using OpenCV with C/C++, MATLAB and AForge. If you don't really know why you would choose one over the other, here is my explanation.

MATLAB is the most easiest way to process images. There are lots of in-built image processing functionalities which come with the MATLAB IDE which you can just drag and drop into your code. But it is not as efficient as you would expect for a real time image processing application. And it's not a free tool either.

On the other hand, OpenCV is a free open source library which allows you to process images efficiently. OpenCV together with C/C++ interface is designed with a strong focus on real time image procession applications. Learning OpenCV has become hardest way only because there is no proper documentation and error handling codes. But OpenCV has lots of basic inbuilt image processing functions so that those who want to learn computer vision can develop their applications through proper understanding about what they do.

AForge.NET is a open source C# framework which provides a convenient way of processing images. In terms of complexity and computational efficiency, AForge.NET stands in between that of OpenCV and MATLAB frameworks.

So, I think that it is worthy to learn computer vision with OpenCV to get a better understanding about what you do. Therefore in this blog, basic image processing functions and computer vision applications written in OpenCV/C++ are presented with line by line explanations.