Camera calibration

From Rsewiki
(Difference between revisions)
Jump to: navigation, search
(→‎Take test images)
 
(16 intermediate revisions by one user not shown)
Line 9: Line 9:
 
The camera calibration test pattern is found here: https://github.com/opencv/opencv/blob/3.4/doc/pattern.png; print it.
 
The camera calibration test pattern is found here: https://github.com/opencv/opencv/blob/3.4/doc/pattern.png; print it.
  
 +
[[File:pattern.png | 300px]]
  
 +
=== Take test images ===
  
−
@todo
+
The images must also have the test pattern ''near the corners'' of the images.
 +
 
 +
Tape the test pattern on a solid frame, as a curved pattern will ruin the calibration.
 +
 
 +
Place the pattern in front of the robot.
 +
 
 +
Check position using [[Robobot_software_description#View_the_stream | motion (Video stremaer)]].
 +
 
 +
View the stream using a browser on port 8081, like http://10.197.216.165:8081/, but with the robot IP, move the pattern (or the robot) for an appropriate view:
 +
 
 +
local@jack:~ $ cd svn/robobot/raubase/build/
 +
local@jack:~/svn/robobot/raubase/build $ '''motion'''
 +
[0:motion] [NTC] [ALL] conf_load: Processing thread 0 - config file /etc/motion/motion.conf
 +
[0:motion] [NTC] [ALL] motion_startup: Logging to file (/home/local/.motion/motion.log)
 +
^C
 +
 
 +
Stop the video streamer (using ctrl-C) and let the '''raubase''' take the image.
 +
 
 +
[[File:test_pattern_and_jack.jpg | 250px]] [[File:img_raw_1.jpg | 250px]]
 +
 
 +
 
 +
local@jack:~/svn/robobot/raubase/build $ '''./raubase -i'''
 +
# UService:: created directory log_20240123_094405.808/
 +
# UService::setup: Ignoring robot hardware (Regbot and GPIO)
 +
# SpyVision:: disabled in robot.ini
 +
# Video device 0: width=1280, height=720, format=MJPG, FPS=25
 +
# Camera is running (to stabilize illumination)
 +
# UCam::run: read frame 0/11
 +
# ready to save
 +
# found '%' in ini[camera][imageName]
 +
# saved image to '''img/img_raw_20240123_094407.424.jpg'''
 +
# saved image to img/img_rec_20240123_094407.424.jpg
 +
# --------- terminating -----------
 +
# UCam::run: camera released
 +
# UCam:: logfile closed
 +
# UService:: configuration saved to robot.ini
 +
 
 +
The raubase saves the image in an '''img''' subdirectory, and the raw image name starts with '''img_raw_2024...'''.
 +
 
 +
Repeat this for 6-10 images with the pattern all around the camera view, something like:
 +
 
 +
[[File:img_raw_1.jpg | 200px]] [[File:cam_cal_2.jpg | 200px]] [[File:cam_cal_3.jpg | 200px]] [[File:cam_cal_4.jpg | 200px]] [[File:cam_cal_5.jpg | 200px]] [[File:cam_cal_6.jpg | 200px]] [[File:cam_cal_7.jpg | 200px]] [[File:cam_cal_8.jpg | 200px]]
 +
 
 +
The run the calibration
 +
 
 +
local@jack:~/svn/robobot/raubase/build $ ./raubase -m
 +
# UService:: created directory log_20240123_105137.289/
 +
# UService::setup: Ignoring robot hardware (Regbot and GPIO)
 +
# SpyVision:: disabled in robot.ini
 +
# Video device 0: '''width=1280, height=720''', format=MJPG, FPS=25
 +
# Camera is running (to stabilize illumination)
 +
# ready to calibrate (stopping camera)
 +
# UCam::run: camera released
 +
#  0 succes    img/img_raw_20240123_101039.126.jpg
 +
#  1 succes    img/img_raw_20240123_103400.651.jpg
 +
#  2 succes    img/img_raw_20240123_103617.847.jpg
 +
#  3 succes    img/img_raw_20240123_103723.998.jpg
 +
#  4 succes    img/img_raw_20240123_103833.044.jpg
 +
#  5 succes    img/img_raw_20240123_103925.440.jpg
 +
#  6 succes    img/img_raw_20240123_104011.273.jpg
 +
#  7 succes    img/img_raw_20240123_105019.243.jpg
 +
# Camera matrix 0:  '''1050.6'''    0.0  '''712.6'''
 +
# Camera matrix 1:    0.0  '''1046.7'''  '''430.6'''
 +
# Camera matrix 2:    0.0    0.0    1.0
 +
# Distortion k1: -0.397866
 +
# Distortion k2: 0.173319
 +
# Distortion p1: 9.58965e-05
 +
# Distortion p2: -0.000660305
 +
# Distortion k3: -0.0385043
 +
# '''Average pixel error is 0.26'''
 +
# Image 0 error 0.29 pixels
 +
# Image 1 error 0.20 pixels
 +
# Image 2 error 0.34 pixels
 +
# Image 3 error 0.32 pixels
 +
# Image 4 error 0.26 pixels
 +
# Image 5 error 0.16 pixels
 +
# Image 6 error 0.19 pixels
 +
# Image 7 error 0.23 pixels
 +
# --------- terminating -----------
 +
# UCam:: logfile closed
 +
# UService:: configuration '''saved to robot.ini'''
 +
local@jack:~/svn/robobot/raubase/build $
 +
 
 +
The code for this is in the scam.cpp file in a function called bool UCam::calibrate().
 +
 
 +
In this case, the average pixel error is calculated to 0.26 pixels.
 +
 
 +
===Camera materix ===
 +
 
 +
The result is a camera matrix ('''CM'''):
 +
 
 +
# Camera matrix 0:  '''1050.6'''    0.0  '''712.6'''
 +
# Camera matrix 1:    0.0  '''1046.7'''  '''430.6'''
 +
# Camera matrix 2:    0.0    0.0    1.0
 +
 
 +
The camera matrix gives the pixel position in the image for a given '''P''' = (x,y,z) position in the real world in meters
 +
(here, x is left, y is down, and z is the distance along the camera centre axis; the camera itself (the focal point) is at (0,0,0) in this coordinate system. i.e.:
 +
 
 +
'''Px''' = '''CM''' * '''P'''
 +
or
 +
px/w = 1050.6 * x + 712.6
 +
py/w = 1046.7 * y + 430.6
 +
w    = z
 +
 
 +
Where px and py are the pixel position in the image; w is the scaling at this distance (z) - from meters to pixels; 1050.6 and 1046.7 are the focal lengths (f) in pixels (should be the same value for most cameras), 712.6 and 430.6 are the pixel positions in the centre of the lens. If the camera chip is mounted correctly, this is also the centre of the image.
 +
 
 +
More details here: https://docs.opencv.org/3.4.20/dc/dbb/tutorial_py_calibration.html.
 +
 
 +
=== Lens distortion ===
 +
 
 +
The lens distortion factors are found to be
 +
 
 +
# Distortion k1: -0.397866
 +
# Distortion k2: 0.173319
 +
# Distortion p1: 9.58965e-05
 +
# Distortion p2: -0.000660305
 +
# Distortion k3: -0.0385043
 +
 
 +
How to intrepid these can be found here: https://docs.opencv.org/3.4.20/dc/dbb/tutorial_py_calibration.html
 +
 
 +
These factors are used to generate a rectified image (saved with the image name ''img_rec_2024...'' when saving an image), and a rectified image is available from the scam module function ''cv::Mat getFrame()'', and used like:
 +
 
 +
  cv::Mat frame = cam.getFrame();
 +
 
 +
The raw distorted frame is also available with:
 +
 
 +
  frame = cam.getFrameRaw();
 +
 
 +
 
 +
=== Configuration file (robot.ini) ===
 +
 
 +
The result of the calibration is saved in the configuration file in the camera section:
 +
 
 +
[camera]
 +
device = 0
 +
width = 1280
 +
height = 720
 +
fps = 25
 +
matrix = 1050.6    0.0  712.6    0.0 1046.7  430.6    0.0    0.0    1.0
 +
distortion = -0.397866 0.173319 9.58965e-05 -0.000660305 -0.0385043

Latest revision as of 11:36, 23 January 2024

Back to Robobot

Back to Robobot software description

Contents

[edit] Camera calibration using openCV

[edit] Test pattern

The camera calibration test pattern is found here: https://github.com/opencv/opencv/blob/3.4/doc/pattern.png; print it.

Pattern.png

[edit] Take test images

The images must also have the test pattern near the corners of the images.

Tape the test pattern on a solid frame, as a curved pattern will ruin the calibration.

Place the pattern in front of the robot.

Check position using motion (Video stremaer).

View the stream using a browser on port 8081, like http://10.197.216.165:8081/, but with the robot IP, move the pattern (or the robot) for an appropriate view: 

local@jack:~ $ cd svn/robobot/raubase/build/
local@jack:~/svn/robobot/raubase/build $ motion
[0:motion] [NTC] [ALL] conf_load: Processing thread 0 - config file /etc/motion/motion.conf
[0:motion] [NTC] [ALL] motion_startup: Logging to file (/home/local/.motion/motion.log)
^C

Stop the video streamer (using ctrl-C) and let the raubase take the image.

Test pattern and jack.jpg Img raw 1.jpg 


local@jack:~/svn/robobot/raubase/build $ ./raubase -i
# UService:: created directory log_20240123_094405.808/
# UService::setup: Ignoring robot hardware (Regbot and GPIO)
# SpyVision:: disabled in robot.ini
# Video device 0: width=1280, height=720, format=MJPG, FPS=25
# Camera is running (to stabilize illumination)
# UCam::run: read frame 0/11
# ready to save
# found '%' in ini[camera][imageName]
# saved image to img/img_raw_20240123_094407.424.jpg
# saved image to img/img_rec_20240123_094407.424.jpg
# --------- terminating -----------
# UCam::run: camera released
# UCam:: logfile closed
# UService:: configuration saved to robot.ini

The raubase saves the image in an img subdirectory, and the raw image name starts with img_raw_2024....

Repeat this for 6-10 images with the pattern all around the camera view, something like:

Img raw 1.jpg Cam cal 2.jpg Cam cal 3.jpg Cam cal 4.jpg Cam cal 5.jpg Cam cal 6.jpg Cam cal 7.jpg Cam cal 8.jpg

The run the calibration 

local@jack:~/svn/robobot/raubase/build $ ./raubase -m
# UService:: created directory log_20240123_105137.289/
# UService::setup: Ignoring robot hardware (Regbot and GPIO)
# SpyVision:: disabled in robot.ini
# Video device 0: width=1280, height=720, format=MJPG, FPS=25
# Camera is running (to stabilize illumination)
# ready to calibrate (stopping camera)
# UCam::run: camera released
#  0 succes    img/img_raw_20240123_101039.126.jpg
#  1 succes    img/img_raw_20240123_103400.651.jpg
#  2 succes    img/img_raw_20240123_103617.847.jpg
#  3 succes    img/img_raw_20240123_103723.998.jpg
#  4 succes    img/img_raw_20240123_103833.044.jpg
#  5 succes    img/img_raw_20240123_103925.440.jpg
#  6 succes    img/img_raw_20240123_104011.273.jpg
#  7 succes    img/img_raw_20240123_105019.243.jpg
# Camera matrix 0:  1050.6     0.0   712.6
# Camera matrix 1:     0.0  1046.7   430.6
# Camera matrix 2:     0.0     0.0     1.0
# Distortion k1: -0.397866
# Distortion k2: 0.173319
# Distortion p1: 9.58965e-05
# Distortion p2: -0.000660305
# Distortion k3: -0.0385043
# Average pixel error is 0.26
# Image 0 error 0.29 pixels
# Image 1 error 0.20 pixels
# Image 2 error 0.34 pixels
# Image 3 error 0.32 pixels
# Image 4 error 0.26 pixels
# Image 5 error 0.16 pixels
# Image 6 error 0.19 pixels
# Image 7 error 0.23 pixels
# --------- terminating -----------
# UCam:: logfile closed
# UService:: configuration saved to robot.ini
local@jack:~/svn/robobot/raubase/build $

The code for this is in the scam.cpp file in a function called bool UCam::calibrate().

In this case, the average pixel error is calculated to 0.26 pixels.

[edit] Camera materix

The result is a camera matrix (CM): 

# Camera matrix 0:  1050.6     0.0   712.6
# Camera matrix 1:     0.0  1046.7   430.6
# Camera matrix 2:     0.0     0.0     1.0

The camera matrix gives the pixel position in the image for a given P = (x,y,z) position in the real world in meters (here, x is left, y is down, and z is the distance along the camera centre axis; the camera itself (the focal point) is at (0,0,0) in this coordinate system. i.e.: 

Px = CM * P
or 
px/w = 1050.6 * x + 712.6
py/w = 1046.7 * y + 430.6
w    = z

Where px and py are the pixel position in the image; w is the scaling at this distance (z) - from meters to pixels; 1050.6 and 1046.7 are the focal lengths (f) in pixels (should be the same value for most cameras), 712.6 and 430.6 are the pixel positions in the centre of the lens. If the camera chip is mounted correctly, this is also the centre of the image.

More details here: https://docs.opencv.org/3.4.20/dc/dbb/tutorial_py_calibration.html.

[edit] Lens distortion

The lens distortion factors are found to be 

# Distortion k1: -0.397866
# Distortion k2: 0.173319
# Distortion p1: 9.58965e-05
# Distortion p2: -0.000660305
# Distortion k3: -0.0385043

How to intrepid these can be found here: https://docs.opencv.org/3.4.20/dc/dbb/tutorial_py_calibration.html

These factors are used to generate a rectified image (saved with the image name img_rec_2024... when saving an image), and a rectified image is available from the scam module function cv::Mat getFrame(), and used like: 

 cv::Mat frame = cam.getFrame();

The raw distorted frame is also available with: 

 frame = cam.getFrameRaw();


[edit] Configuration file (robot.ini)

The result of the calibration is saved in the configuration file in the camera section: 

[camera]
device = 0
width = 1280
height = 720
fps = 25
matrix = 1050.6    0.0  712.6     0.0 1046.7  430.6     0.0    0.0    1.0
distortion = -0.397866 0.173319 9.58965e-05 -0.000660305 -0.0385043
Retrieved from "http://rsewiki.elektro.dtu.dk/index.php?title=Camera_calibration&oldid=6849"
Personal tools
Namespaces

Variants
Actions
Navigation
Toolbox