Auball

From Rsewiki
(Difference between revisions)
Jump to: navigation, search
(Hue filtered image)
(Opening)
Line 172: Line 172:
  
 
Figure 54. Result of an opening operation with a 3x3 (all ones) erosion (twice, if 'opening=2') followed by dilation the same number of times.
 
Figure 54. Result of an opening operation with a 3x3 (all ones) erosion (twice, if 'opening=2') followed by dilation the same number of times.
 +
 +
====Code====
 +
 +
The opening code is straight forward - a number of erodes followed by the same number of dilate.
 +
 +
124    cv::Mat buffer;
 +
125    cv::Mat mask = (cv::Mat_<char>(3,3) <<
 +
126    1,  1,  1,
 +
127    1,  1,  1, 
 +
128    1,  1,  1);
 +
129    cv::erode(use, buffer, mask, cv::Point(-1,-1), varOpening->getInt(0));
 +
130    cv::dilate(buffer, use, mask, cv::Point(-1,-1), varOpening->getInt(0));
  
 
=== Smoothing ===
 
=== Smoothing ===

Revision as of 16:10, 11 February 2020

Back to AU Robot Servers


Contents

Ball finder plugin

A plugin intended for finding coloured balls in a camera image. This example is an image from a Kinect camera, looking at my desk.

Ball 5271 18.png

Figure 18. There is 2 or 3 coloured objects, 2 red in the front and a darker blue in the back.

Sourse image

The code in the file ufuncball.cpp to get the sourse image is

66 UImage * img;
67 if (isOK)
68 { // resource is available, get image to analyze
69   // get source image
70   if (extra != NULL)
71     // image is provided (push command)
72     img = (UImage *) extra;
73   else
74     img = imgPool->getImage(varSourceImg->getInt(), false);
75   isOK = img != NULL;
76   if (isOK)
77     isOK = img->height() > 1;
78   if (not isOK)
79   { // no image
80     sendWarning("No source image");
81   }
82 }

The UImage type is a class that holds a version of the image that can be transferred to the client and a cv::Mat version for openCV manipulation.

Line 74 get the image from the image pool with the number specified in the (global) variable "poolImg=18" with the call 'varSourceImg->getInt()'.

convert to HSV

The same image in HSV format

Ball 5271 45 HSV.png

Figure 45. A high hue value shown as blue, a high saturation in green (e.g. the dark red in the front and the blue lit in the back), the intensity (value) is shown in red (ex. the cup in the front). White objects have a hue of 0, the hue of dark objects is unreliable.

Code

101  // split into planes
102  splitIntoHSVChannels(img, imgHSV, imgHue, imgSat, imgInt, imgRGB);

This is a call to a function that converts to HSV and splits the HSV image into 3 grayscale images

502 void splitIntoHSVChannels(UImage * img, UImage * imgHSV, UImage * imgHue, UImage * imgSat, UImage * imgInt, UImage * imgRGB)
...
521   if (img->usableIpl and not img->usableMat)
522     img->toMat();
523   cv::cvtColor(img->mat, imgHSV->mat, cv::COLOR_BGR2HSV);
524   // result is returned in:
525   imgHSV->fromMat("BGR"); // the RGB is to be displayed as RGB, but the colors will represent HSV
526   imgHSV->setName("HSV_format");
527   // split channels
528   cv::Mat channel[3];
529   cv::split(imgHSV->mat, channel);
530   // hue plane
531   if (imgHue->tryLock())
532   { // make hue visible as gray scale image in image pool
533     imgHue->mat = channel[0].clone();
534     imgHue->setName("hue");
535     imgHue->fromMat("GRAY");
536     imgHue->unlock();
537   }

This function takes 5 images as parameters. The source updates the cv::Mat version of the image in line 522, and then makes a color conversion from BGR to HSV using the 'cv::COLOR_BGR2HSV' parameter in line 523. The image is then loaded back to the image pool and given a new name in lines 525 and 526.

The splitting of the color planes are in line 529, and the remaining lines are to get images back to the image pool.

Crop

The image is cropped to the interesting Range Of Interest, here the top 100 rows are removed (parameter 'topLine').

Split into planes

The individual planes shown as grayscale images.

Ball 5271 46 hue.png

Figure 46. Hue, 0 is yellow (dark like the table, green is about 50 (rather dark), blue is about 115 (brighter), red is about 170 (bright). Maximum is 179, then it is back to 0.

Ball 5271 47 sat.png

Figure 47. Saturation, the brighter the more saturated the colour.

Ball 5271 48 val.png

Figure 48. Intensity, this is a grayscale version of the original image.

Hue filtered image

The hue is used as the main filter, here emphasizing the two selected hues 'redHue=175' and 'blueHue=112'. Low saturation and low intensity are further removed.

Ball 5271 54 opened.png

Figure 53. A grayscale image, where all pixels with hue values of the two colours ('redHue' and 'blueHue') in the range specified by 'colLim' are set to white (255), if the hue is further away, then the highest distance from the desired hue is used. If the saturation is below 'limitSat=70' then the value is set to 0 (dark), the same if the intensity is below 'limitVal=70'.

Code

The filter is implemented in a sub-function:
121  // enhance contrast based on known hue for red and blue ball 
122  use = enhanceDesiredHue(use, imgSat->mat, varLimitSat->getInt(), imgInt->mat, varLimitValue->getInt());
The colour and colour span is fetched in 599-602 below, the saturation and intensity values are transferred as parameters.
597 cv::Mat enhanceDesiredHue(cv::Mat use, cv::Mat sat, int limitSat, cv::Mat val, int limitVal)
598 { // enhance contrast based on expected hue (color tone)
599   int redhue  = varRed->getInt();
600   int bluehue = varBlue->getInt();
601   int redLim = varColorLim->getInt(0);
602   int blueLim = varColorLim->getInt(1);
603   //
604   UImage * imgRoi2 = imgPool->getImage(varDebugImg->getInt() + 8, true);
605   imgRoi2->mat = use.clone();
606 ...

The resulting image is set to be available in the image pool as image 45+8 = 53.

Each pixel in the resulting image is based on the same pixel position in the HSV planes. The saturation and intensity limits are tested by line 617. Then the distance from the desired red and blue colour are calculated in lines 619-624 and 625-631 respectively. If within range then the result is set to 255 (max white), else reduced with an increased contrast of a factor 3 (line 638 and 645).

617         if (*cpSat > limitSat and *cpVal > limitVal)
618         {
619           int dred = *cp - redhue;
620           //               int dredc = *cp - redhue;
621           if (dred > 90)
622             dred -= 180;
623           else if (dred < -90)
624             dred += 180;
625           int dblue = *cp - bluehue;
626           //               int dbluec = *cp - bluehue;
627           if (dblue > 90)
628             dblue -= 180;
629           else if (dblue < -90)
630             dblue += 180;
631           v = absi(dblue);
632           if (v > absi(dred))
633           {
634             v = absi(dred);
635             if (v < redLim)
636               v = 255;
637             else
638               v = 155 - v * 3; //fmax;
639           }
640           else
641           {
642             if (v < blueLim)
643               v = 255;
644             else
645               v = 155 - v * 3; //fmax;
646           }
647           if (v < 0)
648             v = 0;
649         }
650         *cp2 = v;

Opening

The filtered image is likely to have smaller areas enhanced, these are removed/reduced by an opening filter.

Ball 5271 53 enhanced.png

Figure 54. Result of an opening operation with a 3x3 (all ones) erosion (twice, if 'opening=2') followed by dilation the same number of times.

Code

The opening code is straight forward - a number of erodes followed by the same number of dilate.

124     cv::Mat buffer;
125     cv::Mat mask = (cv::Mat_<char>(3,3) << 
126     1,  1,  1, 
127     1,  1,  1,  
128     1,  1,  1);
129     cv::erode(use, buffer, mask, cv::Point(-1,-1), varOpening->getInt(0));
130     cv::dilate(buffer, use, mask, cv::Point(-1,-1), varOpening->getInt(0));

Smoothing

The resulting image is then smoothed to get softer edges better suitable for a canny edge detector.

Ball 5271 55 filtered.png

Figure 55. The smoothing is a Gauss blur with a mask size of 5 ('filter="1 5"').

Hough transform

The Hough transform is performed on the filtered image with a number of parameters ('hough="700 70 5").

The used canny filter has a high limit of 700 (and a low limit of 350 (half)). The second parameter 70 represents the voting of there is a circle with a centre at this position. The last parameter 5 is the resolution of the centre position, in this case all centre votes within 5 pixels are counted.

The Hough transform found these circles.

Hough circles found 3 circles
AUBall::  0 (268,278, 39) HSV=179 178 176 - blueish(112)= 67, redish(175)=  4 (OK=1)
AUBall::  1 (524, 58, 37) HSV=112 112 112 - blueish(112)=  0, redish(175)=-63 (OK=2)
AUBall::  2 (230,376, 38) HSV=170 171 169 - blueish(112)= 58, redish(175)= -5 (OK=1)
Ball:: biggest circle at x=268.5, y=277.5, radius=38.9936, color=1 (of 3)

The numbers in brackets are pixel position and circle radius, then the HSV values and how far the hue is from the two colors. In the last bracket is the colour detection (1 for red and 2 for blue).

Ball 5271 50 annotated.png

Figure 50. The found circles (balls?) are shown in the original image with blue and red circles. The removed top part is shown as a red line.

Canny

The Hough includes a canny edge detector, to make the image binary.

Ball 5271 51 canny.png

Figure 51. A replica of the Canny edge filtered image that the Hough transform uses as the basis for estimation.

Parameters

In the demo code there is a - rather high - number of parameters, these are

  poolImg=18                 (r/w) image pool number to use as source
  poolDebugImg=45            (r/w) first image pool number to use for interim images
  redHue=175                 (r/w) hue value (in HSV formet) for red ball range [0-180]
                             (~120=red)
  redCnt=2                   (r) Number of red balls found in last image
  blueHue=112                (r/w) hue value (in HSV formet) for blue ball range
                             [0-180] (~0=blue) 
  blueCnt=1                  (r) Number of blue balls found in last image
  BallSize=0.12              (r/w) Size of the ball (diameter [m])
  topLine=100                (r/w) is the topmost line that could be a ball on the
                             floor.
  size=14 80                 (r/w) size limits of ball in pixels [min max]
  hough=700 70 5             (r/w) params for Hough (canny high [0-1000], hough vote
                             [0..255], Hough resolution 1(fine)..8(rough))
  colLim=13 13               (rw) Color limit (+/-) for circle center hue match [red
                             blue] [0..180]
  mrc=1                      (rw) Should result be send to MRC (smrcl)
  debug=1                    (rw) make more debug images and printout
  filter=1 5                 (rw) smooth image before detect [filter 0-1, size NxN]
  opening=2                  (rw) Opening before Hough circles
  limitSat=70                (rw) do not use pixels with saturation lower than this
                             (0..255)
  limitVal=70                (rw) do not use pixels with a lower V-value (intensity)
                             lower than this (0..255)
Personal tools
Namespaces

Variants
Actions
Navigation
Toolbox