Camera calibration
(→Test pattern) |
(→Take test images) |
||
| (8 intermediate revisions by one user not shown) | |||
| Line 9: | Line 9: | ||
The camera calibration test pattern is found here: https://github.com/opencv/opencv/blob/3.4/doc/pattern.png; print it. | The camera calibration test pattern is found here: https://github.com/opencv/opencv/blob/3.4/doc/pattern.png; print it. | ||
| − | [[File:pattern.png | + | [[File:pattern.png | 300px]] |
=== Take test images === | === Take test images === | ||
| Line 19: | Line 19: | ||
Place the pattern in front of the robot. | Place the pattern in front of the robot. | ||
| − | Check using [[Robobot_software_description#View_the_stream | motion (Video stremaer)]]. | + | Check position using [[Robobot_software_description#View_the_stream | motion (Video stremaer)]]. |
View the stream using a browser on port 8081, like http://10.197.216.165:8081/, but with the robot IP, move the pattern (or the robot) for an appropriate view: | View the stream using a browser on port 8081, like http://10.197.216.165:8081/, but with the robot IP, move the pattern (or the robot) for an appropriate view: | ||
| Line 31: | Line 31: | ||
Stop the video streamer (using ctrl-C) and let the '''raubase''' take the image. | Stop the video streamer (using ctrl-C) and let the '''raubase''' take the image. | ||
| − | [[File:test_pattern_and_jack.jpg | | + | [[File:test_pattern_and_jack.jpg | 250px]] [[File:img_raw_1.jpg | 250px]] |
| + | |||
local@jack:~/svn/robobot/raubase/build $ '''./raubase -i''' | local@jack:~/svn/robobot/raubase/build $ '''./raubase -i''' | ||
| Line 51: | Line 52: | ||
The raubase saves the image in an '''img''' subdirectory, and the raw image name starts with '''img_raw_2024...'''. | The raubase saves the image in an '''img''' subdirectory, and the raw image name starts with '''img_raw_2024...'''. | ||
| − | Repeat this for 6-10 images with the pattern all around the camera view. | + | Repeat this for 6-10 images with the pattern all around the camera view, something like: |
| + | |||
| + | [[File:img_raw_1.jpg | 200px]] [[File:cam_cal_2.jpg | 200px]] [[File:cam_cal_3.jpg | 200px]] [[File:cam_cal_4.jpg | 200px]] [[File:cam_cal_5.jpg | 200px]] [[File:cam_cal_6.jpg | 200px]] [[File:cam_cal_7.jpg | 200px]] [[File:cam_cal_8.jpg | 200px]] | ||
| + | |||
| + | The run the calibration | ||
| + | |||
| + | local@jack:~/svn/robobot/raubase/build $ ./raubase -m | ||
| + | # UService:: created directory log_20240123_105137.289/ | ||
| + | # UService::setup: Ignoring robot hardware (Regbot and GPIO) | ||
| + | # SpyVision:: disabled in robot.ini | ||
| + | # Video device 0: '''width=1280, height=720''', format=MJPG, FPS=25 | ||
| + | # Camera is running (to stabilize illumination) | ||
| + | # ready to calibrate (stopping camera) | ||
| + | # UCam::run: camera released | ||
| + | # 0 succes img/img_raw_20240123_101039.126.jpg | ||
| + | # 1 succes img/img_raw_20240123_103400.651.jpg | ||
| + | # 2 succes img/img_raw_20240123_103617.847.jpg | ||
| + | # 3 succes img/img_raw_20240123_103723.998.jpg | ||
| + | # 4 succes img/img_raw_20240123_103833.044.jpg | ||
| + | # 5 succes img/img_raw_20240123_103925.440.jpg | ||
| + | # 6 succes img/img_raw_20240123_104011.273.jpg | ||
| + | # 7 succes img/img_raw_20240123_105019.243.jpg | ||
| + | # Camera matrix 0: '''1050.6''' 0.0 '''712.6''' | ||
| + | # Camera matrix 1: 0.0 '''1046.7''' '''430.6''' | ||
| + | # Camera matrix 2: 0.0 0.0 1.0 | ||
| + | # Distortion k1: -0.397866 | ||
| + | # Distortion k2: 0.173319 | ||
| + | # Distortion p1: 9.58965e-05 | ||
| + | # Distortion p2: -0.000660305 | ||
| + | # Distortion k3: -0.0385043 | ||
| + | # '''Average pixel error is 0.26''' | ||
| + | # Image 0 error 0.29 pixels | ||
| + | # Image 1 error 0.20 pixels | ||
| + | # Image 2 error 0.34 pixels | ||
| + | # Image 3 error 0.32 pixels | ||
| + | # Image 4 error 0.26 pixels | ||
| + | # Image 5 error 0.16 pixels | ||
| + | # Image 6 error 0.19 pixels | ||
| + | # Image 7 error 0.23 pixels | ||
| + | # --------- terminating ----------- | ||
| + | # UCam:: logfile closed | ||
| + | # UService:: configuration '''saved to robot.ini''' | ||
| + | local@jack:~/svn/robobot/raubase/build $ | ||
| + | |||
| + | The code for this is in the scam.cpp file in a function called bool UCam::calibrate(). | ||
| + | |||
| + | In this case, the average pixel error is calculated to 0.26 pixels. | ||
| + | |||
| + | ===Camera materix === | ||
| + | |||
| + | The result is a camera matrix ('''CM'''): | ||
| + | |||
| + | # Camera matrix 0: '''1050.6''' 0.0 '''712.6''' | ||
| + | # Camera matrix 1: 0.0 '''1046.7''' '''430.6''' | ||
| + | # Camera matrix 2: 0.0 0.0 1.0 | ||
| + | |||
| + | The camera matrix gives the pixel position in the image for a given '''P''' = (x,y,z) position in the real world in meters | ||
| + | (here, x is left, y is down, and z is the distance along the camera centre axis; the camera itself (the focal point) is at (0,0,0) in this coordinate system. i.e.: | ||
| + | |||
| + | '''Px''' = '''CM''' * '''P''' | ||
| + | or | ||
| + | px/w = 1050.6 * x + 712.6 | ||
| + | py/w = 1046.7 * y + 430.6 | ||
| + | w = z | ||
| + | |||
| + | Where px and py are the pixel position in the image; w is the scaling at this distance (z) - from meters to pixels; 1050.6 and 1046.7 are the focal lengths (f) in pixels (should be the same value for most cameras), 712.6 and 430.6 are the pixel positions in the centre of the lens. If the camera chip is mounted correctly, this is also the centre of the image. | ||
| + | |||
| + | More details here: https://docs.opencv.org/3.4.20/dc/dbb/tutorial_py_calibration.html. | ||
| + | |||
| + | === Lens distortion === | ||
| + | |||
| + | The lens distortion factors are found to be | ||
| + | |||
| + | # Distortion k1: -0.397866 | ||
| + | # Distortion k2: 0.173319 | ||
| + | # Distortion p1: 9.58965e-05 | ||
| + | # Distortion p2: -0.000660305 | ||
| + | # Distortion k3: -0.0385043 | ||
| + | |||
| + | How to intrepid these can be found here: https://docs.opencv.org/3.4.20/dc/dbb/tutorial_py_calibration.html | ||
| + | |||
| + | These factors are used to generate a rectified image (saved with the image name ''img_rec_2024...'' when saving an image), and a rectified image is available from the scam module function ''cv::Mat getFrame()'', and used like: | ||
| + | |||
| + | cv::Mat frame = cam.getFrame(); | ||
| + | |||
| + | The raw distorted frame is also available with: | ||
| + | |||
| + | frame = cam.getFrameRaw(); | ||
| + | |||
| + | === Configuration file (robot.ini) === | ||
| + | The result of the calibration is saved in the configuration file in the camera section: | ||
| − | + | [camera] | |
| + | device = 0 | ||
| + | width = 1280 | ||
| + | height = 720 | ||
| + | fps = 25 | ||
| + | matrix = 1050.6 0.0 712.6 0.0 1046.7 430.6 0.0 0.0 1.0 | ||
| + | distortion = -0.397866 0.173319 9.58965e-05 -0.000660305 -0.0385043 | ||
Latest revision as of 11:36, 23 January 2024
Back to Robobot
Back to Robobot software description
Contents |
[edit] Camera calibration using openCV
[edit] Test pattern
The camera calibration test pattern is found here: https://github.com/opencv/opencv/blob/3.4/doc/pattern.png; print it.
[edit] Take test images
The images must also have the test pattern near the corners of the images.
Tape the test pattern on a solid frame, as a curved pattern will ruin the calibration.
Place the pattern in front of the robot.
Check position using motion (Video stremaer).
View the stream using a browser on port 8081, like http://10.197.216.165:8081/, but with the robot IP, move the pattern (or the robot) for an appropriate view:
local@jack:~ $ cd svn/robobot/raubase/build/ local@jack:~/svn/robobot/raubase/build $ motion [0:motion] [NTC] [ALL] conf_load: Processing thread 0 - config file /etc/motion/motion.conf [0:motion] [NTC] [ALL] motion_startup: Logging to file (/home/local/.motion/motion.log) ^C
Stop the video streamer (using ctrl-C) and let the raubase take the image.
local@jack:~/svn/robobot/raubase/build $ ./raubase -i # UService:: created directory log_20240123_094405.808/ # UService::setup: Ignoring robot hardware (Regbot and GPIO) # SpyVision:: disabled in robot.ini # Video device 0: width=1280, height=720, format=MJPG, FPS=25 # Camera is running (to stabilize illumination) # UCam::run: read frame 0/11 # ready to save # found '%' in ini[camera][imageName] # saved image to img/img_raw_20240123_094407.424.jpg # saved image to img/img_rec_20240123_094407.424.jpg # --------- terminating ----------- # UCam::run: camera released # UCam:: logfile closed # UService:: configuration saved to robot.ini
The raubase saves the image in an img subdirectory, and the raw image name starts with img_raw_2024....
Repeat this for 6-10 images with the pattern all around the camera view, something like:
The run the calibration
local@jack:~/svn/robobot/raubase/build $ ./raubase -m # UService:: created directory log_20240123_105137.289/ # UService::setup: Ignoring robot hardware (Regbot and GPIO) # SpyVision:: disabled in robot.ini # Video device 0: width=1280, height=720, format=MJPG, FPS=25 # Camera is running (to stabilize illumination) # ready to calibrate (stopping camera) # UCam::run: camera released # 0 succes img/img_raw_20240123_101039.126.jpg # 1 succes img/img_raw_20240123_103400.651.jpg # 2 succes img/img_raw_20240123_103617.847.jpg # 3 succes img/img_raw_20240123_103723.998.jpg # 4 succes img/img_raw_20240123_103833.044.jpg # 5 succes img/img_raw_20240123_103925.440.jpg # 6 succes img/img_raw_20240123_104011.273.jpg # 7 succes img/img_raw_20240123_105019.243.jpg # Camera matrix 0: 1050.6 0.0 712.6 # Camera matrix 1: 0.0 1046.7 430.6 # Camera matrix 2: 0.0 0.0 1.0 # Distortion k1: -0.397866 # Distortion k2: 0.173319 # Distortion p1: 9.58965e-05 # Distortion p2: -0.000660305 # Distortion k3: -0.0385043 # Average pixel error is 0.26 # Image 0 error 0.29 pixels # Image 1 error 0.20 pixels # Image 2 error 0.34 pixels # Image 3 error 0.32 pixels # Image 4 error 0.26 pixels # Image 5 error 0.16 pixels # Image 6 error 0.19 pixels # Image 7 error 0.23 pixels # --------- terminating ----------- # UCam:: logfile closed # UService:: configuration saved to robot.ini local@jack:~/svn/robobot/raubase/build $
The code for this is in the scam.cpp file in a function called bool UCam::calibrate().
In this case, the average pixel error is calculated to 0.26 pixels.
[edit] Camera materix
The result is a camera matrix (CM):
# Camera matrix 0: 1050.6 0.0 712.6 # Camera matrix 1: 0.0 1046.7 430.6 # Camera matrix 2: 0.0 0.0 1.0
The camera matrix gives the pixel position in the image for a given P = (x,y,z) position in the real world in meters (here, x is left, y is down, and z is the distance along the camera centre axis; the camera itself (the focal point) is at (0,0,0) in this coordinate system. i.e.:
Px = CM * P or px/w = 1050.6 * x + 712.6 py/w = 1046.7 * y + 430.6 w = z
Where px and py are the pixel position in the image; w is the scaling at this distance (z) - from meters to pixels; 1050.6 and 1046.7 are the focal lengths (f) in pixels (should be the same value for most cameras), 712.6 and 430.6 are the pixel positions in the centre of the lens. If the camera chip is mounted correctly, this is also the centre of the image.
More details here: https://docs.opencv.org/3.4.20/dc/dbb/tutorial_py_calibration.html.
[edit] Lens distortion
The lens distortion factors are found to be
# Distortion k1: -0.397866 # Distortion k2: 0.173319 # Distortion p1: 9.58965e-05 # Distortion p2: -0.000660305 # Distortion k3: -0.0385043
How to intrepid these can be found here: https://docs.opencv.org/3.4.20/dc/dbb/tutorial_py_calibration.html
These factors are used to generate a rectified image (saved with the image name img_rec_2024... when saving an image), and a rectified image is available from the scam module function cv::Mat getFrame(), and used like:
cv::Mat frame = cam.getFrame();
The raw distorted frame is also available with:
frame = cam.getFrameRaw();
[edit] Configuration file (robot.ini)
The result of the calibration is saved in the configuration file in the camera section:
[camera] device = 0 width = 1280 height = 720 fps = 25 matrix = 1050.6 0.0 712.6 0.0 1046.7 430.6 0.0 0.0 1.0 distortion = -0.397866 0.173319 9.58965e-05 -0.000660305 -0.0385043
