Colored beads sorter

I wanted to try building something involving mechanics with electronic (program) control. I decided to build colored beads sorter for my girlfriend. I thought that the hardest part would be the mechanical sorter, so I started with this first.

Mechanical part

It is made of LEGO, with some additional parts. Basically it is composed of beads container, and two plastic discs. One disc is stable and has two holes in it, allowing beads to fall through. The other disc is rotating on top of the first one and has bead-sized hole in it. By passing the hole through the container it takes one bead and transports it under detector (camera) and directs it to desired output container. The rotating disc is actuated by stepper motor salvaged from old scanner.

Electronics

The stepper motor is driven by ULN2003 transistor array, also salvaged (better said cut out) from the same scanner. It is controlled by a FITKit, which is our school's development platform. It contains MSP430 MCU and Xilinx FPGA. Only the MCU was used in this project. On the breadboard, there are some transistors to control the LEDs (now pretty useless, before it was part of color detector) and potentiometer used with photo-resistor as voltage divider. This voltage used to be measured by ADC to determine color intensity (by lighting up red, green and blue LEDs consecutively), now it only detects bead-under-the-camera position of the rotating disc.

Color detection

First I tried color detection using 3 colored LEDs (red, green, blue) and light dependent resistor. Results were very bad, differences in measured color intensities were too small to reliably detect different colored beads. Even after calibration there were too many errors.

Then I decided to go overkill and replace the tri-color detector by webcam and computer vision software. Although it is now computer dependent (with color LEDs computer was not needed, everything was programmed in FITKit), results improved radically. This was my first computer-vision project, so it was useful to learn something new.

Software

The FITKit is programmed to control the stepper driver, it is standard C++ program for MSP430 microcontroller. It listens on UART interface for NEXT and BACK commands and rotates the upper disc so the bead falls through desired hole. Then it takes new bead from container and positions it under the camera.

For correct position detection, there is black tape mark on the rotating disc and LED-photoresistor detector (I used the former color detector, this is why there are 3 LEDs instead of 1).

Everything is under control of Java program, that captures image from webcam and detects bead colors. It uses BoofCV computer vision library. Detection is done by Canny edge detector - finds position of bead on the captured image. From detected region pixels are thresholded by saturation and mean color hue is calculated.

Screenshot of control application. Image on the left is the webcam captured image, the one on the right is examined region with filtered pixels used for color detection

Calculating mean of hue values is not trivial, because hue represents an angle, with red color on both 0° and 360°. (It took me a while to find out why red beads are detected as turquoise color - turquoise is in the middle of hue spectrum). To solve it, I calculated sine and cosine values of each pixel's hue (X and Y coordinates of corresponding points on unit circle) and then calculated the average of these points, finally converted it back to angle - this is the average hue. (For details, see wikipedia) For each bead, hue is then compared against preset value and bead is directed towards correct bucked. (The sorter only separates one color from all other colors).

Source code