Camera calibration: Android and ARToolKit

There is something that I believe every augmented reality developer should know. (At least if we are talking about optical Augmented Reality.) And that is camera calibration. I believe you should have a general understanding what camera calibration is, why we need it and what ARToolKit does to make life very easy for us as augmented reality developers.

Why to calibrate your camera?

Let me start with that:

I thought like that and that is because of the underlying technology that makes us forget about such technical details.

The ideal world

(Source: https://au.mathworks.com/help/vision/ug/calibration_applications.png)

Most our consumer cameras are based on the ‘Pinhole Camera Model’. Meaning they should reflect a copy of the real world onto the Image plane.And a plain pinhole camera would do just that. Cameras, however, use lenses to widen (or to narrow) the field of view (The snippet of the real world that is actually visible to the camera. Which leads to the fact that we have image distortions.

The real world

In the real world, the camera is build using lenses and lenses produce distortions. In the case of Android cameras, we get a ‘positive radial distortion’ or ‘barrel distortion’

That is because the sensor of the camera is smaller than the field of view we would like to capture, resulting in a squeezed image to fit onto the sensor. This squeezing has the effect that straight lines on the incoming image appear to be curved inwards.

Here you see an example of a barrel distortion and how the image looks with the distortion removed:

Projected to Augmented Reality

What does this now mean for optical augmented reality? Well, the image that comes in from the camera is not ideal, in fact is rather bad to look for features. Because what happens is that, we feed our augmented reality application with a 2D trackable that ideal and flat. But we retrieve the same image through the camera lens with a barrel distortion applied to it. That leads to the fact that the augmented reality application is looking for feature points in the wrong position. This leads to a bad tracking or in the worst case to no tracking at all.

Print image that we would like to track

Distorted image as the camera sees it

What ARToolKit does

If you are developing wit ARToolKit, life gets a bit easier for you because ARToolKit has established a method to mitigate this camera calibration related issues. Whenever you run an app based on ARToolKit your app reads device specific parameters from your Android device and requests a camera calibration for this specific device. This has the great benefit for you that chances are good that your app not just runs on your Android device but also on other Android devices which you couldn’t test with.

Conclusion

This article has outlined the basics about why it is imported to calibrate cameras and should give you an idea why you augmented reality application might not work as expected on every Android device. Keep that in mind and let me know if you discovered some such behavior or if you have additions.