This guy did a great job of addressing some of the shortcomings of my machine, specifically in the number of output bins (18 instead of 12), but it is a regression in other ways (built out of Lego so it will break frequently, mechanical part movement from the belts, which will lead to frequent jams and will wear as well as that it is slow and more timing sensitive).
All in all a nice effort and very nice video production but not nearly as much of an improvement that would make it reproducible/mass marketable.
To properly build a machine like this will take a significant budget and a lot of hours behind cad software to make it out of industrial parts rather than Lego. I you want to make something mass marketable then you have a very long road ahead of you.
It seems like increasing the number output bins shouldn't really matter much, since with either machine you could just reprogram it and run parts through multiple times, right?
That's roughly what I got it to with showing it samples of wheels but not all wheels and it detected wheels that it had never seen before just fine and so on. This is a lot easier than it might seem, you'll get a good match on a class but no good hit for any of the subclasses: that's a new part right there.
And yes, that's exactly how I did it, multiple passes. But more bins is certainly useful, I just ran out of room, the machine was already getting quite large, and the belt and frame I used was fixed in size and dictated a lot of the dimensions of the machine (it was a re-purposed running trainer).
First of all, as I mention in the video, this project is inspired by the machines built by akiyuky and jacquesm@. So I'm certainly not claiming this is the world's first LEGO sorting machine!
In the video I try and focus on the unique aspect of this machine, which is the Universal part. Because the CNN is trained on synthetic data, the machine is able to recognize any LEGO part that has ever been produced. This is with the notable exception of ignoring printed parts, mold variations, and oversized parts, as well as parts that are not present in the LDraw part library. These restrictions bring the tens of thousands of part categories down to just under 3000.
Accurately classifying 3000 categories is prohibitively difficult using manual labelling alone, simply because it's very hard to get physical access to all these different parts (not to mention the labelling process itself is quite laborious).
Is it overkill? Totally. But one goal of this project is to be a testbed for the use of synthetic data in a real-world classification system, and I'm quite happy with how well it ended up working.
I have produced another video explaining the AI system in detail: https://youtu.be/-UGl0ZOCgwQ
It has also been noted that the machine would not be mass marketable in its current state. That is totally accurate! This machine should be considered a proof-of-concept/prototype and I have no plans to try and create a mass-produced version.
How long does it run between fixes? How many kg of lego have you put through it?
Beware of lots people that will bug you to give them all your work, to this day not a week goes by that someone contacts me with a thinly veiled request to get a copy of the hard parts 'just for their personal curiosity' and then when you search for a bit on Google you'll find them associated with large parts resellers.
Best of luck with your project, curious about what you'll do next with it.
The machine is quite robust to jams so long as you're careful not to insert oversized parts. That being said,
I'd say there's a problem every 30-60 minutes, so certainly not something you could leave overnight. Running for hours and hours means that it will eventually grind itself into dust anyway (I've already burned out a motor!)
I'll probably end up sharing everything one way or another. I'm sure you'll agree that any real commercial application of something like this is pretty far-fetched so I'm more than happy to just treat this as a contribution to the body of domain randomization application research.
It is odd to claim a first whilst also crediting you. He seemed to emphasize the universality in several places (I guess you didn't explicitly state this?).
At some point the guy that bought Bricklink wanted to buy out the project and I told him to f-off because of the way Bricklink has been run, no surprise to see him sell it, likely he got many times over what he paid for it.
I'm happy to see someone continue with the project though, for me the fun was over as soon as the software worked and it turned into an engineering/business model affair. Bringing this to market is a lot harder than making a prototype.
Not realizing this for almost any product is one of the biggest mistakes technical founders make.
https://xkcd.com/678/ 5 years
I built my V0 out of Lego and it worked but it was very quickly evident that it needed a more industrial approach if you wanted to drop 10 Kg of Lego in the hopper and go have lunch.
It's a pity I don't have the time - or the funds - to really see this through but I have a pretty good understanding of what it would take to do this properly and to date nobody has done one that I would say is ready to be used by anybody but its creators. Once that does happen there will be a rush of follow-ons.
Also, does anyone know of a group working to improve the recycling symbol? It's too small/shallow to even be read by humans, much less machines, on many plastic containers. Short of that, we could break down the task into 1) isolating the symbol on the container and 2) reading the symbol maybe with extra wavelengths like infrared/ultraviolet or their combination with visible light.
Here in Boise, we have regular recycling that only accepts types 1 and 2, then they make participants sort 4, 5, 6, and 7 into an orange bag that gets sent to Nevada to be turned into diesel fuel. I see a ton of recycling in the trash because people aren't motivated to sort it manually:
EDIT: Added JPG: https://imgur.com/a/puSmAzO
PDF (old): https://www.cityofboise.org/media/1632/detailedplasticsguide...
It would be nice just to know what to do with specific materials rather than guessing half the time.
Also image recognition has been a solved problem since about 2010, so any roughly 1 GHz processor can do it in under a second for something like a plastics recycling symbol.
I just can't believe that there is no money available for projects like these. I was thinking about it just now at the gym and I'd like to see society move away from donating time at a personal level, and towards organizational donating where the workers involved are paid the going rate on their labor. So instead of working a full-time job and being expected to put in a couple nights and weekends per month on volunteering, we'd instead donate money to foundations that could go deep on problems with real backing. It seems to me that this would also be a good fit with open source in general. These problems are so much easier than the yak shaving we do at work that we could solve most of them in a matter of a year or two.
I took a VERY similar approach for generalized universal lego sorter and got to about 120 classes of parts recognizing acceptably well for about 70 of the classes. I used Mathias' articles as a guide.
I was picking and choosing physically varied lego to find difficult pieces to sort. Beveled pieces tended to be the worst. Small pieces hid important information maybe? I suspect any solution will require mirrors to get better information.
I also got discouraged because I wanted to sort into about 2-300 different "buckets" and found the mechanical piece of that intimidating and got distracted by some other fun project. Maybe I'll return to it.
edit: I followed up by reading the longer linked article. I can't believe how similar the software the two of us wrote was. There are some labeling enhancements that I were probably the next step that I appreciate he took.
Well done nonetheless, seems to work pretty smooth (though I see a lot of mixed materials in the buckets?).
I tried to buy a bunch of 44 specific lego pieces (the parts were listed from a book and fairly common) through Brickseek and couldn't do it.
Brickseek has a "Easy Buy" where you can add items to your list and have it find sellers, but it didn't work at all for me. Manually trying to find each piece is not worth the effort.
You buy or just get donated legos from people that no longer use them and then you can sell custom sets for people by running through the semi sorted bins and creating the order. So you don't want to sell the machine but rather, sell Legos. Perhaps the main issue is whether resale rights are allowed but seems solvable.
I would suspect this is a nice little business selling 40% discounted complete Lego sets.
It's been a subject of some research and here is the most damning piece of evidence: