Recommendation Engine

Let's Make Robots
1

I have been laid up this winter and have been working from home the past several months.  Idle hands ARE the devil's work so I decided to take on a machine learning challenge at CodeProject.com and build a recommendation engine like Amazon, Facebook etc.  These are interesting pieces of software that in commercial sites can work in conjunction with search engines to lead customers to content, products or items they might never would have looked at otherwise.  In fact, this is something that might be useful on LMR - maybe have a recommendations window for when a user logs in?  This way, older content that is on the site gets recycled and reviewed instead of disappearing into bit heaven.

I am posting this here since I think in robotics, the concepts of how to identify similar things would be useful for a robot.  The article goes through the formulas and approach I took to the problem.  I wrote the code in C#, but this could easily be converted to run as a service in the cloud, or could be a background process that runs a few times a day (more likely- this almost dims the lights when it starts running...)  The point is that this gives a framework and an approach on how to address what is a pretty easy thing for you and me but is hard for a computer.  There are also other examples written in JavaScript, Python and Java.

https://www.codeproject.com/Articles/1233227/Recommendation-Engine

I hope someone finds this interesting and useful.

2

Recommendation Engine

Thanks for posting and I wish you great success but that subject matter is so far over my head that the words just flew past. My typical need for recomendation is “Do you have Harpoon IPA? If not goodbye!!”

3

LOL A Harpoon IPA sounds

LOL  A Harpoon IPA sounds like a great recommendation to me!  Who needs software to tell you that?

Seriously, it seems overwhelming, but it really isn’t that bad.  A few formulas that you plug numbers into and that is it.  You won the firefighter challenge ; this is way easier than that.

 

4

Trust and The Wizards of Oz

Hi Bill,

Once again, great stuff!

This topic of recommendation engines is in vogue lately because a lot of big companies have invested a lot of money in their recommendation engines, but in some spheres, they are recognizing some dangers from skewing of results, or having bad press callling attention to it.  (Facebook, Google - TouTube, etc)

Factor #1  Like - You nailed this factor, nothing more to say.

I would posit that there are other very important factors that the algos don’t adequately consider.

Factor #2  Trust - This area needs more attention in the industry.  Blockchain created a whole new industry around this.

Several of these engines recognize and predict “likes”, without adequately recognizing the trustworthiness of the source of the information or the trustworthiness (even the existence of) others that are liking something.  Effectively, we are talking about fake news, fake people liking fake new, troll farms, bot armies, re-tweets, etc…to manufacture the “trending” of an idea.   In short, a lot of people/companies/state actors are gaming the system to achieve various goals.  Some of the companies haven’t cared to date, as long as we keep clicking, they keep making money.  Not being much of a Facebook user…I really notice this on YouTube.  Amazon handles this much better through reputation.  Google/Facebook have more to gain by ignoring the problem, while Amazon’s interests are more in line with keeping you a happy repeat purchaser.

Often it is not nefarious, simply a weakness of the algo to differentiate from real likes by real people versus technological “rubber-necking”  I don’t like seeing people get killed on the freeway, but I find it hard not to look.  I would suggest that the algos also need to take location more into account…a little more weight on what my nearest neighbors like, or at least people in the same country/hemisphere/language. 

Going too far in this arena though can lead to reenforcing everyone’s individual bubbles…effectively mass-producing confirmation bias…and telling you what the algo thinks you want to hear.  If you are a racist, YouTube will serve up a never ending set of videos to reenforce fears of “them”, or you can simply turn on the TV.  If you like conspiracy theories…it will train you to see them everywhere.  If it doesn’t know you yet, you will see cat videos.

Perhaps we should be allowed to see all the levers that the Wizards of Oz have in their control rooms, and exert individual control over how we are “programmed”.  “Programmed” is Facebook’s lingo for this.  As it stands, the Wizards have HUGE power…perhaps beyond anything Orwell imagined.

 

 

5

There are a number of algos

There are a number of algos one can use; I just picked one and went with it. Like most things in ML, run all the algos and then take what seems to give the best results.  For a computer to figure something that is similar to something else is hard to do, but is something you and I can do pretty easily.  Hence why recommendations we get aren’t that great sometimes.  It is only as good as the algo and the data it gets to process it on.