Lego Sorter

Let's Make Robots Robots
1

UPDATE:
I added a video of the 'finished product' that I presented in front of the class. It basically combines everything that I've talked about previously into one algorigthm for 'sorting'. I also uploaded two flow charts as attached files which outline the steps of each program. It's a bit of a proof of concept rather than something helpful but I'm still pretty happy with it.

 

Hi.

Imgur photos here: http://imgur.com/a/AZg0n#0

At the end of grade 12 I had a control systems assignment and as my project I decided to make a Lego sorter. The idea is to eventually get every piece that my school owns to be sorted and it be completely independent, but we only had about 12-15 hours of school time over 4-5 weeks to do it so that was a bit of a stretch.

I used SimpleCV for the recognition. (If you've not heard of it then it's a python library which uses the OpenCV libraries but makes them a bit easier to use). Basically, there is a saving program in which I put the Lego piece on the white plate in every orientation that it is likely to fall on. The program saves pictures for each of these orientations in a directory with the name of the piece. (I have no idea if this is the best way to do machine learning pictures but it seems to work alright, any suggestions are welcome)

The machine learning program then comes along and learns based on the pictures (it uses the SVM machine learning method from SimpleCV, which uses the Orange libraries).

Finally there is the actual robot which is a python to NXT interface achieved through a USB serial link and the nxt-python library. There is an initial setup sequence which checks the camera and the nxt and then it starts sorting. The conveyor moves the piece onto the white plate, the camera classifies it using the machine learning, then the last motor turns around to put the piece in its own bay (which are still under construction). --I am still working on this last part of the program and I have only done the setup sequence as of now--

Once this first phase is all done I will probably start work on a feature extractor made just for the lego which will hopefully improve accuracy.

It has been great fun so far and I hope to do a bit more with SimpleCV and a RPi over the school holidays.

Thanks for reading,
-Riddellikins

 

Sorts lego into their singular piece types

This is a companion discussion topic for the original entry at https://community.robotshop.com/robots/show/lego-sorter
2

This is great!

Does it only have the brick options that you tested it with in the first video? Could you introduce a new brick and it would automatically register it as a different brick type? It’s really good, I presumed from the title it sorted them by colour but it’s much more complex than that!

3

That’s a great idea

It will only recognise pieces that I add manually as of now. Thinking about that new brick adding mechanism it would probably be possible if the machine learning classifying function has a certainty so, if the certainty is lower than a certain threshold then it must be a new piece and it will take a picture, add it to the bank and then give it some arbitrary name. That would be a cool feature.

And yeah, at first I was just going to use a light sensor but they can be a bit boring and I really wanted to use computer vision for something, and this was perfect.

4

Nice work.

Nice work.  I wish I was that skillful with OpenCV.  I wanted to do some sort of visual landmark system (spotting objects around a room and using them as landmarks for localization), but got in over my skill level. 

5

Thanks :slight_smile:

Thanks for the comment. I think that landmarking system sound much more difficult than what I’m doing. Having a white background, ideal lighting and a still camera are main things that make this not all that difficult in terms of computer vision (this was my first CV project).

I am also using SimpleCV (check it out), which is a thousand times easier than OpenCV. I tried OpenCV, it didn’t turn out well :stuck_out_tongue:

6

Problem with first video.

This video is private.

7

Sorry

Sorry about that, I uploaded using my phone and forgot the default is private. Fixed now

8

Hi.Very interesting project.

Hi.

‘Solved’

 

svm = SVMClassifier.load(“SVMClass.xml”) and it works.
Solution for question buttom

Very interesting project.Thank you for this work. :slight_smile:


Please can you help me a little bit.
I’ve created a few Lego pieces picture and create the SMVClass file.
But how i can load this to compare.

In the train programm it does allright.

But when i need to load all pictues at Programmstart it could be hours to load when its over 1000 types/parts of lego.

Is it that the SMV File Stands for?

 

hhfe = HueHistogramFeatureExtractor()
haarfe = HaarLikeFeatureExtractor()
ehfe = EdgeHistogramFeatureExtractor()
#lfe = LegoFeatureExtractor()
extractors = [hhfe, ehfe]#, lfe]#    , haarfe]
props ={
‘KernelType’:‘Poly’, #default is a RBF Kernel
’SVMType’:‘C’, #default is C
’nu’:None, # NU for SVM NU
’c’:None, #C for SVM C - the slack variable
’degree’:None, #degree for poly kernels - defaults to 3
’coef’:None, #coef for Poly/Sigmoid defaults to 0
’gamma’:None, #kernel param for poly/rbf/sigma - default is 1/#samples
}

svm = SVMClassifier(extractors,props)
svm.load(“SVMClass”)

while disp.isNotDone():
    img = cam.getImage()
    if n > 4:
        className = svm.classify(img)
        n = 0
    else:
        n +=1
    img.drawText(className, 10, 10, fontsize=60, color=Color.RED)
    img.show()
    if disp.mouseRight:
        break

 

but svm.classify(img) error.
The SVMClass file is created with content

Traceback (most recent call last):
  File “test.py”, line 50, in <module>
    className = svm.classify(img)
  File “/usr/local/lib/python2.7/dist-packages/SimpleCV/MachineLearning/SVMClassifier.py”, line 153, in classify
    featureVector.extend([self.mClassNames[0]])
IndexError: list index out of range

It seems he didn’t load the SVMClass or whats going here wrong?

Thank you!

Sincerely Bonkersdeluxe

9

Machine Learning.py

MachineLearning.py doesn’t seem to work. I’m getting this error

 

Traceback (most recent call last):

  File “C:\Users\JackSB\SAP\Python\legosorter\machineLearning.py”, line 32, in <module>

    offset = trainPaths[1].replace("/", " “, trainPaths[1].count(”/")-1).find("/")

IndexError: list index out of range

 

Can you please help? I’m slightly new to machine learning but would really like to try this out. Thanks!