Robobot camera

Latest revision as of 12:55, 6 December 2022

Back to Robobot

[edit] camera settings

[edit] Data formats

The used camera is a 'Jieli Technology' camera of type 'USB PHY 2.0' (packed by Sandberg).

The camera has limited control capabilities but supports the following data formats.

• MJPG coded 640 x 320 pixels at 25 or 30 FPS
• MJPG coded 1280 x 720 pixels at 25 or 30 FPS
• MJPG coded 1920 x 1080 pixels at 25 or 30 FPS
• MJPG coded 640 x 480 pixels at 25 or 30 FPS; this mode cuts parts of the image width to fit the format

It also supports "raw" format as YUYV 4:2:2, but it seems like data loss occurs (fragmented images), so do not use this format.

[edit] Use from OpenCV (C++)

First, open and set the camera format:

cv::VideoCapture cap;

void setup()
{
 int deviceID = 0;        
 int apiID = cv::CAP_V4L2;
 // open selected camera using selected API
 cap.open(deviceID, apiID);
 // check if we succeeded
 camIsOpen = cap.isOpened();
 if (not camIsOpen)
 {
   cerr << "ERROR! Unable to open camera\n";
 }
 else
 {
   uint32_t fourcc = cv::VideoWriter::fourcc('M','J','P','G'); 
   cap.set(cv::CAP_PROP_FOURCC, fourcc);
   // possible resolutions in JPEG coding
   // (rows x columns) 320x640, 720x1280
   cap.set(cv::CAP_PROP_FRAME_HEIGHT, 720);
   cap.set(cv::CAP_PROP_FRAME_WIDTH, 1280);
   cap.set(cv::CAP_PROP_FPS, 25);
   // now the camera is ready for frame capture. 
   //
   // debug print of accepted format:
   union FourChar
   { // the four characters are coded into a 32-bit integer.
     uint32_t cc4;
     char ccc[4];
   } fmt;
   fmt.cc4 = cap.get(cv::CAP_PROP_FOURCC); // get format
   printf("# Video device %d: width=%g, height=%g, format=%c%c%c%c, FPS=%g\n", 
          dev, 
          cap.get(cv::CAP_PROP_FRAME_WIDTH), 
          cap.get(cv::CAP_PROP_FRAME_HEIGHT), 
          fmt.ccc[0], fmt.ccc[1], fmt.ccc[2], fmt.ccc[3], 
          cap.get(cv::CAP_PROP_FPS));
 }

Then capture an image to a matrix called 'frame':

cv::Mat frame;
bool gotFrame = false;

cap.read(frame);
// mark as available
gotFrame = not frame.empty();

The frame has 3 channels in BGR format.

To get the most recent image, it is a good idea to keep the buffer empty by discarding unneeded images by:

cap.grab();

In the template code, this is done in a thread (called loop())

[edit] Camera calibration

The camera has significant lens distortion and can be calibrated and rectified using the openCV calibration functions.

But, close to the centre of the image, this may not be needed. The main parameter here is the focal length.

The focal length f is approximately 1008 pixels, when the image format is 1280 x 720.

[edit] Finding 3D position of a known object

The relation between an object with pixel size Px and the focal length f is the same as the actual object size S and the distance D from the camera to the object along the centre axis of the camera, i.e.

Px     S
--- = ---
 f     D

The unit of S and D could be in meters.

This can be used to find the 3D position of a known object.

The distance along the camera axis is thus (when the size of the object is known):

• D = S * f / Px

The 3D position of the object can then be found in camera coordinates as (with x being to the right and y directed down

• xm = (xi - xc) * D / f
• ym = (yi - yc) * D / f

where xm and ym are distances in meters from the camera axis, xi and yi is the image column and row for the (centre of the object), xc and yc are the column and row for the centre of the image, D is the distance to the object in meters and f is the focal length in pixels.