(For more resources related to this topic, see here.)
First you should download the OpenCV framework for iOS from the official website at http://opencv.org. In this article, we will use Version 2.4.6.
The following are the main steps to accomplish the task:
Let’s implement the described steps:
You can add the framework as a resource. This is a straightforward approach. Alternatively, the framework can be added through project properties by navigating to Project | Build Phases | Link Binary With Libraries. To open project properties you should click to the project name in the Project Navigator area.
#ifdef __cplusplus
#import <opencv2/opencv.hpp>
#endif
This is needed, because OpenCV redefines some names, for example, min/max functions.
UIImage* MatToUIImage(const cv::Mat& image)
{
NSData *data = [NSData dataWithBytes:image.data length:image.
elemSize()*image.total()];
CGColorSpaceRef colorSpace;
if (image.elemSize() == 1) {
colorSpace = CGColorSpaceCreateDeviceGray();
} else {
colorSpace = CGColorSpaceCreateDeviceRGB();
}
CGDataProviderRef provider = CGDataProviderCreateWithCFData((__
bridge CFDataRef)data);
// Creating CGImage from cv::Mat
CGImageRef imageRef = CGImageCreate(image.cols, //width
image.rows, //height
8, //bits per
component
8*image.elemSize(),//bits
per pixel
image.step.p[0], //
bytesPerRow
colorSpace, //colorspace
kCGImageAlphaNone|kCGBitmapByteOrderDefault,//
bitmap info
provider, //
CGDataProviderRef
NULL, //decode
false, //should
interpolate
kCGRenderingIntentDefault
//intent
);
// Getting UIImage from CGImage
UIImage *finalImage = [UIImage imageWithCGImage:imageRef];
CGImageRelease(imageRef);
CGDataProviderRelease(provider);
CGColorSpaceRelease(colorSpace);
return finalImage;
}
void UIImageToMat(const UIImage* image, cv::Mat& m,
bool alphaExist = false)
{
CGColorSpaceRef colorSpace = CGImageGetColorSpace(image.
CGImage);
CGFloat cols = image.size.width, rows = image.size.height;
CGContextRef contextRef;
CGBitmapInfo bitmapInfo = kCGImageAlphaPremultipliedLast;
if (CGColorSpaceGetModel(colorSpace) == 0)
{
m.create(rows, cols, CV_8UC1);
//8 bits per component, 1 channel
bitmapInfo = kCGImageAlphaNone;
if (!alphaExist)
bitmapInfo = kCGImageAlphaNone;
contextRef = CGBitmapContextCreate(m.data, m.cols, m.rows,
8,
m.step[0], colorSpace,
bitmapInfo);
}
else
{
m.create(rows, cols, CV_8UC4); // 8 bits per component, 4
channels
if (!alphaExist)
bitmapInfo = kCGImageAlphaNoneSkipLast |
kCGBitmapByteOrderDefault;
contextRef = CGBitmapContextCreate(m.data, m.cols, m.rows,
8,
m.step[0], colorSpace,
bitmapInfo);
}
CGContextDrawImage(contextRef, CGRectMake(0, 0, cols, rows),
image.CGImage);
CGContextRelease(contextRef);
}
#import "opencv2/highgui/ios.h"
viewDidLoad() method:
- (void)viewDidLoad
{
[super viewDidLoad];
UIImage* image = [UIImage imageNamed:@"lena.png"];
// Convert UIImage* to cv::Mat
UIImageToMat(image, cvImage);
if (!cvImage.empty())
{
cv::Mat gray;
// Convert the image to grayscale
cv::cvtColor(cvImage, gray, CV_RGBA2GRAY);
// Apply Gaussian filter to remove small edges
cv::GaussianBlur(gray, gray,
cv::Size(5, 5), 1.2, 1.2);
// Calculate edges with Canny
cv::Mat edges;
cv::Canny(gray, edges, 0, 50);
// Fill image with white color
cvImage.setTo(cv::Scalar::all(255));
// Change color on edges
cvImage.setTo(cv::Scalar(0, 128, 255, 255), edges);
// Convert cv::Mat to UIImage* and show the resulting
image
imageView.image = MatToUIImage(cvImage);
}
}
Now run your application and check whether the application finds edges on the image correctly.
Frameworks are intended to simplify the process of handling dependencies. They encapsulate header and binary files, so the Xcode sees them, and you don’t need to add all the paths manually. Simply speaking, the iOS framework is just a specially structured folder containing include files and static libraries for different architectures (for example, armv7, armv7s, and x86). But Xcode knows where to search for proper binaries for each build configuration, so this approach is the simplest way to link external library on the iOS. All dependencies are handled automatically and added to the final application package.
Usually, iOS applications are written in Objective-C language. Header files have a *.h extension and source files have *.m. Objective-C is a superset of C, so you can easily mix these languages in one file. But OpenCV is primarily written in C++, so we need to use C++ in the iOS project, and we need to enable support of Objective-C++. That’s why we have set the language property to Objective-C++. Source files in Objective-C++ language usually have the *.mm extension.
To include OpenCV header files, we use the #importdirective. It is very similar to #include in C++, while there is one distinction. It automatically adds guards for the included file, while in C++ we usually add them manually:
#ifndef __SAMPLE_H__
#define __SAMPLE_H__
…
#endif
In the code of the example, we just convert the loaded image from a UIImage object to cv::Matby calling the UIImageToMat function. Please be careful with this function, because it entails a memory copy, so frequent calls to this function will negatively affect your application’s performance.
Please note that this is probably the most important performance tip—to be very careful while working with memory in mobile applications. Avoid memory reallocations and copying as much as possible. Images require quite large chunks of memory, and you should reuse them between iterations. For example, if your application has some pipeline, you should preallocate all buffers and use the same memory while processing new frames.
After converting images, we do some simple image processing with OpenCV. First, we convert our image to the single-channel one. After that, we use the GaussianBlur filter to remove small details. Then we use the Canny method to detect edges in the image. To visualize results, we create a white image and change the color of the pixels that lie on detected edges. The resulting cv::Mat object is converted back to UIImage and displayed on the screen.
The following is additional advice.
There is one more way to add support of Objective-C++ to your project. You should just change the extension of the source files to .mm where you plan to use C++ code. This extension is specific to Objective-C++ code.
If you don’t want to use UIImage, but want to load an image to cv::Mat directly, you can do it using the following code:
// Create file handle
NSFileHandle* handle =
[NSFileHandle fileHandleForReadingAtPath:filePath];
// Read content of the file
NSData* data = [handle readDataToEndOfFile];
// Decode image from the data buffer
cvImage = cv::imdecode(cv::Mat(1, [data length], CV_8UC1,
(void*)data.bytes),
CV_LOAD_IMAGE_UNCHANGED);
In this example we read the file content to the buffer and call the cv::imdecode function to decode the image. But there is one important note; if you later want to convert cv::Mat to the UIImage, you should change the channel order from BGR to RGB, as OpenCV’s native image format is BGR.
This article explained how to link the OpenCV library and call any function from it.
Further resources on this subject:
I remember deciding to pursue my first IT certification, the CompTIA A+. I had signed…
Key takeaways The transformer architecture has proved to be revolutionary in outperforming the classical RNN…
Once we learn how to deploy an Ubuntu server, how to manage users, and how…
Key-takeaways: Clean code isn’t just a nice thing to have or a luxury in software projects; it's a necessity. If we…
While developing a web application, or setting dynamic pages and meta tags we need to deal with…
Software architecture is one of the most discussed topics in the software industry today, and…