Robot Musings: Robotic Complexity Rating (RCR)

Let's Make Robots
1

I have been thinking for a while about how one might rate a robot for its level of complexity or sophistication, perhaps assigning it a number or code that classifies it based on its advancement.  If they can assign colors bands to resistors, there should be some way to rate a robot as well.

What first got me thinking about this was the blog on robot tricks/skills.  My first thought was that the more tricks a robot could perform then the higher its “Robot Complexity Rating” (RCR) would be.  At first this seemed great.  Dumbbot can do 2 tricks and thus is a RCR 2.  Smartbot can do 8 tricks and is an RCR 8.  MegaBrilliantbot can cook you dinner, balance your check book and apply the Heimlich maneuver if you choke on a hot dog, RCR 136 at maybe.

But when you look through the robots on LMR it quickly becomes clear this is too simple an approach.  For one thing, some tricks/skills are harder than others.  STOP, WANDER, GUARD, none of those are exceptionally hard.  On the other hand walking on legs, or walking on two legs, or balancing on two wheels are much more advanced skills or challenges than just standing there trying not to catch on fire.

Robotix 5 - https://www.robotshop.com/letsmakerobots/node/32281

EquipoiseBot - https://www.robotshop.com/letsmakerobots/node/32012

Red Dragon - https://www.robotshop.com/letsmakerobots/node/32012

Are just a few examples from many of robots I consider demonstrating advanced features.  Perhaps tricks/skills should be rated by difficulty and the highest level trick a robot can perform becomes part of its RCR code?  For instance:

STOP, WANDER, GUARD – all examples of Class 1 or perhaps Class A tricks, simple tricks.

WALK, FIND (object), FOLLOW (person or object) – are Class B tricks.

BALANCE (on two feet or wheels) is a Class C trick.

Using this, a walking robot that can do 5 tricks one of which is FOLLOW would become categorized as a RCR 5-B robot.

But is that enough?  Just controlling a lot of actuators should count for something, as some of these robots are like watching a Swiss watch in motion, graceful, sophisticated, and amazingly well crafted.  Perhaps the number of actuators should be part of the rating.  In that case, a hexapod with 3 servos per leg that can do 5 tricks would be a 5-B-18 robot.

I can’t say I have come to a complete idea formed on this.  Might craftsmanship or finish be a forth rating factor?

Anyway this is all of my thoughts on this for now.  What ideas, comments, or suggestions might the rest of you have?

 

2

I believe you did strike on a useful point.

With respect to your 5-B-18 example above, but, instead of 5-B-18 just realize that you are talking about a bot with 3DOF and that it is a hexapod. So, rather than 18 being part of your rating scheme, a 3DOF hexapod would have its own rating vs a 2DOF hex.

3

What we most ponder is

What we most ponder is things like code complexity as well. Is a robot that interacts via “stupid” if-cases as complex as something that has PID-control and Kalmann-filters?

4

My ideas on robot complexity rating

If I could suggest it must come down to two things for complexity at least… Number of sensors and actuators and number of lines of code.

For sophistication I think it comes down to how well the code is written to make use of all the sensors and actuators. That is the foundation for a robot capable of many tasks. Secondly the quality of the sensors( as in how reliable the data they generate) and also the quality of the actuators in the amount of work they are capable of reliably.

Finally for charm I’d say build quality and creative essence. Ok Couldn’t find the word for that but it’s just got to have a look that wins thumbs up and gives it the appearance of soul or being.

So going along with a tri-scoring system I’d say a value for complexity, one for sophistication and one for charm.

In the end it’s going to be a very subjective scoring system no matter how you rate them. One person will rate the robot different to another for a few reasons. The first is knowledge of the person rating the robot. To appreciate how many hours it would take to write a large program with good code, research the hardware and put it together. Second would be bias towards a particular favourite brand of hardware or class of robot. And thirdly if it just looks good to them it might prejudice their ratings for complexity and sophistication. Lastly… How can you be sure you understand the robots value when many postings are very lacking in detail and nobody likes to show videos of their robots being naughty. Of course this is only of concern to rating LMR bots not if you have the opportunity to compare in reality.

I can’t believe how many lines I ended up writing, ideas just started popping in my head. Thanks for bringing up this subject. Rather than turning it into a rating system for approval I think it would be more noble to establish  a rating system for benchmarking, to encourage builders to aim higher and keep lifting the goals.  Maybe even a page on LMR setting out the standards decided with metrics for people to scan and see where they want to go with their project.

5

Rating is useless
Its no good to rate a robot which performs tricks to different levels. All robots aren’t the same. The complexity of a robot is not decided by how many servos it has or how many tricks it can do, it is the creator if the robot who decides that depending on what functions the bot has to perform, what are the parts that are available and how much time the person has. Since for no 2 creators, all three quantities can be same (unless you’re organizing a competition) hence every bot is different. Since only similar things can be compared, it is impossible for you to compare the the m using any rating. Also, ratings can be biased. Maybe you rate a bot excellent or whatever is the highest in your ratings but the creator is not happy with the bot as it doesn’t perform all the functions that were required to be performed and maybe you can rate an excellent robot that took the programmer to ages program but is lacking a few parts due to unavailability or as a challenge. Take for example my case. I want to build a quadped but due to unavailability of pays due to my budget, I can’t build one that had 2 or 3 DOF. As a result, I’ll try to make one which has 1 DOF. Not only will it be difficult to make but it’s code is also going to be more complex to perform a simple function like moving left or right . Hence your rating rates me down but on reality, my project its tougher. Its not logical rate robots and any thoughts of, this type is childish. Sorry, if I’m rude.

6

WOW, Controversial Subject

WOW, I had no idea this would turn out to be a controversial subject.  I really expected this to be a “yawn who cares”, kind of topic. I see many members have some strong feelings about this and the comments make a lot of good points.

Let me say I am not interested in suggesting LMR impose a rating system or that we rank or judge anyone.  This is a hobby after all and suppose to be fun.  What I was really aiming for was some method of trying to measure my own progress.  

A lot of the robots here are examples of “lateral development”.  There is nothing wrong with lateral development, in fact its the smart way to test new systems and code, because you can try …one new thing… on a working design and its easier to debug it and get it right.  That said, I want to try and build on my previous robots to make robots that do more stuff.  The idea of the RCR was to give myself some way of measuring progress.

After reading the comments, the one thing I am sure of now is I don’t like my own system.  The remarks about DOF and baseline are all great.

I much prefer merser’s suggestion of sensors, actuators, code (SAC) as a classification of the robots technical abilities.  (Trying to get you in trouble now  Merser …lol)

When I first came to LMR, I was of course obsessed with the technical details of building the robot.  But the longer I go through the robots section, the more I notice how important the “artistic” aspect of the robot is.  Almost all the robots are interesting and cute, but some of the robots really  have amazing style.

Vishu also makes a very good point as well about the challenge of making a robot without access to the best parts. 

Thanks for all the comments.  It has really been interesting hearing your ideas and prospective on this subject.

 

7

Not rude at all

Hello Vishu, thanks for commenting.  I don’t consider your remarks rude at all, they are honest and direct.  You make many good points and I appreciate you taking the time to respond.  Thanks sharing your understanding of this.

8

Rating robots is hard. You

Rating robots is hard. You need a lot of info about the hardware and software, some things you can see right away, other things are showing up in action over time.  For example, there is this hexapod dancing competition, where everybody is using 3DOF hexapod robots using similar hardware but they pimp up their robots for the competition, so some look great and cute, some just regular. But wait until you see them move. That’s where they differ. So, that is a code competition. Take a standard robot kit and organize a competition, it’s gonna be all in the code. But we create robots that are different one to another (original?). Some are variations on the same theme, some are pure revolutionary. So, how do we rate and classify that? Most of the LMR users have built a SHR at some point, some to start up with, some just to show how can be done a bit differently. All these robots have a few hardware things in common: controller, motors, wheels, batteries, servo, IR distance sensor. The code is the same, a translation of the original, or something similar. Variations to the theme are a chassy, a piezo speaker, IR remote sensor, ultrasonic instead of IR distance sensor, and so on. Now, if we create a SHR class we can rate the different impact each of these robots makes on our viewing pleasure. Say, we give it a 1 if it’s a copy cat with no improvement, and that actually rates the success of the builder in actually building a SHR robot, (perhaps it’s their first robot). Then we see a SHR that has some CDs as chassis, has a piezo speaker to play beeps or a song, an IR remote sensor for external control and an ultrasonic sensor instead of a distance sensor, plus a couple of bumper switch sensors. Is that even a SHR anymore? If it is, the what scrore does it get? Or it evolved to the next class? Or the next class is a robot that has a gripper on top of the SHR hardware? Hard to decide. However, the only thing that moves upeople after they see a cool design is watching the robot in action. Even if it is still a work in progress. At some stages, we can take a short video demonstrating some abillities, following with updates with more progress. Then people can express their impressions about our work and I guess that gets rated, together with the “wow” factor of the robot.

9

Self Rating: no votes

How about a system where the builder can rate his or her own robot using some sort of questionaire?

That way, the rating will not be based on how much other members like certain parts and designs. Also: because no voting is involved, the builder can just add the rating in the robot page without the need to change the LMR website.

It will have to be well thought out questions, but it can be done. With the answers from questions like

are the movements PID or PD controlled? rated from 0=“what do you mean” to 5=“more then 3 PID controlled routines” in a few major categories like:

  • mechanics and movement
  • electronics
  • programming
  • bodywork and craft

Everybody will be able to see that a certain robot has a 3.2 complexity rating for mechanics and a 2.0 for programming. (or something like that)

10

Again I’m opposed to this view

The simple problem with the rating system that you suggest is that not everyone on the LMR site is a competent engineer. While there are some terms that everyone will know, there is going to be some things and some people who are going to find terms from mechanics, electronics or programming too difficult to understand (and if you need example for this, I can find none better that CtC’s conversation with David C. via Shout Box). Also, programming as a specific can’t be judged by the complexity. There are programs which might be made to simplify the implementation of hard to implement things. Such programs on first look will look simple and hence be marked down on your scale but the thing that they are implementing is complex and hence it is as good as useless to rate it. For mechanics and electronics, read my comment above (or below, that actually depends on your settings).

11

Quantify coolness?

I agree that there are no objective criteria to rate complexity just as there are no objective criteria to measure how cool something looks. 

My point is that instead of the LMR society scoring each others robots on complexity, you can rate it yourself. The ratings that a builder gives his creation will still indicate that a certain robot was build with complex programming in mind, not so much the complex electronics or mechanics.

The reader can see/ask why the number 4 was chosen. Because “having multiple processors”, “using advanced power management” and “coping with many sensors simultatiously” or something.

David C. might rate his robot project as “electronics complexity=3” because it has “lots of pushbuttons”, “requires much soldering” and "uses both Mac OSX and Windows"

Both numbers are correct and they are open for debate but they have lots of meaning and purpose even though it is very subjective to the builder. The ratings of different builders can be brought closer together using some form of questionaire as a starting point 

 

12

Your idea is certainly better than community rating
but tell me, what are the chances that you rate your robot what is correct or what it deserves. Maybe A makes a start here robot which for him is very tough and he rates it 5 but for B, who is pretty advanced in creating bots, its another stroll in the park. If B sees the 5 rating, maybe he’ll be interested in, making it but on reading the content he’ll find it pretty easy. Since for no 2 people, the complexity of a project can be same (even in organized competition), you will definitely have lots of problem rating it. The same start here bot will be rated as 5 by A and 1 by B. This is going to create quite a lot of doubt in the mind of a person who is new, to the site. I’m not against rating but if you can devise a method that is fair to everyone, only then rating should be adopted.