Memory Systems

I am curious as to what approaches different people would advocate for giving their robots the capacity to remember.

In order to hopefully stimulate some discussion, I'll mention a few approaches:

1.  Who needs memories?  Make stupid robots that just react but never get smarter.

2.  Use Files To Store Memories in Some Way...like pictures of faces, or data logs for sensors.

3.  Use In Memory Data Structures for Short-Term Memories or frequently used memories.  I personally find dictionaries with key/value pairs (like HybridDictionaries in C#) to be useful for many purposes.  However, what good is this in the long term if not persisted?

4.  Use Databases:   If using a database, how to design it?  Can I have a small set of tables that store many different types of memories, or do I want to model each different type of memory as separate table(s)?

5.  Use Neural Nets?  I haven't tried this, so I can't speak to it.

With any of these, it would seem that "Pattern Recognition", "Searching for Similarity", "Nearest Neighbor", etc. type searches would be necessary.  Any approaches/tools that people like for doing this?

There is also the issue of memories stored over time in sequences, Cause and effect learning, etc.  Anyone doing this?

Memories are fundamental to brain function, so I think the topic deserves discussion.  I am curious how others approach this important function.

 

If you talk about Pattern

If you talk about Pattern Recognition and data storage, what about an SOM (Self Organizing Map) it’s not really a way of storing data itself but a system that classifies input data. It’s just a bunch of vectors, you also wouldn’t save the data itself but the trained vectors. It’s not very difficult but still a little too long to explain here. So what you save is a bunch of vectors and probably which output of the SOM belongs to which pattern/data. One advantage of an SOM is that it never changes the size of the stored vectors. Unlike a database of for example faces where you would store each new face, with an SOM you would change the existing vectors to also classify the new face.

I hope it makes sense what I said for more info on SOM’s read the wikipedia page : https://en.wikipedia.org/wiki/Self-organizing_map

Making Intellegent Robots

One of my main interests with robotics is to try and make them intellegent. What that actually means, I am still working on. But for some basic functions I am researching various methods of computer vision to apply to one of my robots. I have been reading a lot on OpenCv, it has some powerful built in tools to do eiganfaces for face detection and HOG + Linear SVM to do basic object detection.  

In terms of memory storage, I too don’t know the best method. However, I am thinking having a large dataset (whether thats a database on the cloud or locally saved files organized by hiearchal folders) that contains thousands of images of different house hold items that are tagged with their name. Then I can use these large data sets to train neural networks so they can recognize and pick up on these objects. The only problem with neural networks, from my initial research, is that its very much a “black box” approach and once its trained for a specific task it can only work for that one task. 

Your point on  “memories stored over time in sequences, Cause and effect learning” is very intersting. I think nerual networks or some form of machine learning might be able to implement this. 

I am currently reading a book, “How to Create a Mind” by Ray Kurzweil. It goes into depth about the different parts of the brain and how researchers are trying to essentialyl model the brain in software. I think you might like it. 

 

Great Subject

I use two different types of tables depending on what type of project I am doing.

For the majority of the time I use simple access tables and they seem to work really nice.

I have used SQL tables and they work great but with all the additional software you have to install such as MS-SQL it tend to weigh down the system and decrease the system resources.

With Access tables you dont have to install Access or any other MS office product to make them work and the connection strings are very simple.

I have used the Access tables for storing GPS locations, Images, Text, Number etc… and they have work out well.

My AI project that I have been working on for some time now started with SQL tables but after doing some testing found that the Access tables did just as good a job and didn’t weigh the system down and cause the resources to drop. Where as SQL is a service that has to run which pulls the system down some. The teeter totter of resources/memory/Processor is one that I am always playing with and find just the right combination is difficult some of the times. My original AI engine that I made used a text file to store the data in and it was getting out of control and there was no real way to organize it well without using several text files.

I have also experminted with use arrays and variables and while they work well, once you turn the software off or turn the computer off all that was learned is gone.

Neural nets can now learn more than one task

The idea that neural nets can be trained to do one job is not strictly true anymore, googles deepmind team have recently published a paper on a new type of neural net called a “differentiable neural computer” - https://deepmind.com/blog/differentiable-neural-computers/

These use an exetenel memory source and learns how to efficiently read/write information and interrupt it correctly. I think this limits the information that needs to be stored for pretty minimal overheads (no need to keep all the training data)

Personally I think neural nets are the way forward, even in the past 5 years they have come on so dramatically, probably brought on by the deep belief boom that kickstarters so much research. By training systems that “learn how to learn” we can craft a single model and teach it new content over time, even having different teams working on teaching the model seperate skills

A few techniques I have used…

I find the SOM (Self Organizing Map) VERY interesting.  I have seen papers expressing emotions as vectors.  I think aspects of visual and verbal could be done with vectors.

Most of my memory experience has been with data that is easy to imagine in relational datastores.  Whether Access (as Jeff discussed) or SQL Server, the concept is the same…to model memory structures and then store lots of them with their relationships to other memory structures.  I think this technique has many uses (its how the world stores data), but other techniques are needed in addition to move forward.

Currently, I use a CacheHelper service that sits in between the code and the databases and decides what to hold “in memory” and what to go to the databases for.  It also takes care of building indexes to access by ID, Name, or other factors, for speed.  This works well for frequently accessed data (like words, phrases, and many others) which is constantly being looked up.  It works well too when their are logic questions to deal with like “Can a penguin fly?” where many memory nodes (possibly hierarchical in nature) have to be evaluated to get an answer.

For NLP parses, I also use a special intermediary “cache helper” (keyed by sentence) and “Parse Dictionary” where I cache features about a given parse of a given sentence.  This lets me avoid hitting the NLP parse routines twice for the same sentence in the same session…this lets you ask questions about a recent or distant conversation without having to incur huge delays as each sentence is found, prioritized, and parsed.  Example:  “What color was the bus?” - where recent references to buses need to be recalled, parsed, and colors identified.

For persistence, I used to use a “generic” database structure that could store any memory type.  I then needed “meta memories” - memories about memories - to define and document the structure of each memory type.  This model made it easy to develop a user interface on top of all memories so I culd see what was going on.

Later I decided to support “specific” databases - meaning “non-generic” - where each specific memory type was modeled as a separate table.  In order to build a UI, I still needed the meta-memories - which I derived from the system tables automaticallly.  As a side benefit…This UI has so many business applications…imagine apps that build themselves.  This model has advantages in that additional indexes can be added for specific tables…a necessity now that I am dealing with some tables that have millions of rows.  I would say it is a necessity as anything over 75K rows.

I layered a generic rules engine on top of this…where the rules (pre-conditions (think patterns or stimulus) and post-conditions (think responses) were modelled in a few tables.  Rules could be layered on top of any table in any db…hundreds of them.  Reflexes are one use of a rules engine.  There are many others.

Other than conversation histories, I haven’t gotten into storing the detailed histories over time of sensor data…tooo much data.  I feel like some intermediary derived data (features, vectors, etc) might be better and smaller and easier to deal with.

I speculated once on creating what I called “Data Cubes” …whereby a “Data Cube” object would monitor a given sensor input and store (in memory) the recent history and summary stats on less recent history in various time increments (minute, hourly, daily, etc.).  This cube would handle storing the summary stats asyncronously to a db when it chose to,   The point of this whole idea was to allow a robot to monitor a set of inputs (through multiple cubes) and attempt to derive knowledge or correlations from observation.  Example:  Getting a robot to observe that it tends to get warmer when the sun comes up.  Beyond correlation, since the cubes would store data over time…perhaps they could be programmed to recognize cause and effect…by recognizing a change in one precedes a change in another.   I think it is all possible, but never built the cubes or tried it.   We used to have to derive formulas for raw data in school.  This basically represents coming up with a hypothesis or idea.  Robots could do this too…which would be really cool I think, a robot coming up with ideas from observation.

I haven’t mentioned visual here…I think its an area where something like the SOM or some other techniques would be useful…I haven’t done anything I would consider useful visually.

Just a few quick thoughts

Just a few quick thoughts that haven’t seen mentioned yet in this thread:

OpenCog has some relational system based on probabilities/likelihoods, not sure if there is a way to persist them to disk.

Ontologies or Semantic Web technologies are good at storing structured logical/symbolic data, exchange them between agents, and reason over them. Thus it is also possible to deduce relationships in time sequences.

There are scalability issues with these approaches, though, they don’t work so well on lower level sensor data and I think in the long run some sort of combination of neural nets and semantic/symbolic methods are necessary.

Hierarchical Temporal Memory

I also find Hierarchical Temporal Memory to be very interesting and promising…

Wiki:

https://en.wikipedia.org/wiki/Hierarchical_temporal_memory

A Video:

https://www.youtube.com/watch?v=6ufPpZDmPKA

I used the sparse distributed representations concept in my own bots (the simple part)…so many more parts yet to try.

I remember seeing some very convincing youtube videos a year or two back saying that major breakthroughs had been made in neoroscience that seemed to substantiate that our brains use something like HTM.

My test case might be, to get a robot to recognize a melody from a few notes and anticipate the next notes (or hum along), no matter what key the song is in.  This case is way beyond my skills right now.

Memory Stats are Important

Great points…Having stats attached to learned memories is very important in my opinion.

I would expand it to having 3 values for each learned memory…plus a sub structure (1 to Many) for source.

1.  N - the number of times this memory has been reenforced - re-learned.

2.  A Mean - this is the robot’s estimate that the memory is correct.

3.  A Standard Deviation - this is the robot’s estimate of the variability in its estimate of the correctness (#2)

Some mathematicians might have other ideas, but I think the principles are useful.  I find this very useful for verbal stuff.  I think its also valuable for the robot to track the source of knowledge and make estimates of the trustworthiness of each source.   

A bot is going to take in a lot of stuff that is wrong, uncertain, conflicting, etc.   This provides a means for handling much of this and allowing the robot to live in the space between 0 and 1.,where most human experience is.  Good knowledge will get reinforced, bad actors will get discovered, and things that are inherently uncertain will get discovered and their variability will become understood and estimated.

Sorry if this is too much detail or too obvious to be interesting.  Cheers.  -Martin

The melody use case is a

The melody use case is a very interesting one and could be applied to predicting action sequences, too. If you bring music theory into this, or actually have the agent recognize some characteristics of the music (without knowing music theory), you’d also have higher level understanding and a general feeling for the type of music.

You might understand movement and the “spirit” of the movement (sequence of actions=sequence of notes) and intention in similar ways as you grasp the deeper meaning and intent of music.

I had some ideas along this path, writing a GUI that let’s you define actions a bit like when writing music scores, with AI helping you to fill in the gaps by recognizing patterns and letting you just specify keypoints instead of all the details. Also higher level analysis of the overal goals you are probably trying to achieve similar to how music can be analyzed (music theory).

I think a major issue with understanding an agent’s memory and rules of behavior/planning/goals etc. is to find representations that are expressive enough to be grasped quickly. Even for ontologies, which in theory are not black boxes (you can look at all the formal rules that are meaningful on their own), the amount of facts and relationships very quickly become impossible to understand. You can perfom queries on ontologies (like on databases) but that’s not going to give you an overall understanding of what a set of rules describe. Interestingly it’s similar to neural nets, it’s in a way too low level to extract the higher level idea that individual rules encode.

So no matter what encoding you find for memory, you really need some way to translate that into various representations. I think this capability is also key to learning, especially in a more abstract way, and to be able to learn and transfer knowledge to other domains, but also to be able to communicate it to other agents.

 

differentiable neural computer

You’re right, a friend of mine told me about this. I don’t fully understand the concept yet but I feel like this is the future in terms of AI. What is your experience with neural networks? Have you used tensorflow? I’ve heard that is a good way to get up and running with neural nets.

 

 

Tensorflow

Yeah Tensorflow is awesome, really good for getting up and running with neural nets quickly, I’ve recently been messing with Keras with a TF backend.

I made a few neural nets at uni using a pretty old language called “CORTEX” and since then made a few for mostly visual tasks (face recognition / tracking / OCR) etc
I’ve only built one outside of following tutorials with tensorflow which was a deep CNN for character recognition, but I don’t think I had quite enough training data to make it work as well as I would like, and got far better results using a KNN classifier and support vector machines.

I made a mouth/jaw tracker a while back using SVM’s as well, they are great for visual tasks. Can be seen here if you are interested:

https://www.youtube.com/watch?v=qjgOhO8VmCw

Deep networks are very data hungry, but if you can provide enough they can hit awesome results. If you are interested in playing with one for OCR i’d suggest the following tutorial: http://www.pyimagesearch.com/2016/08/01/lenet-convolutional-neural-network-in-python/

Pyimagesearch is a fantastic resource for both machine learning and computer vision, I can’t recommend it enough.

 

 

I am envious.

I am envious of your skills/knowledge/experience.  Makes me feel like an obsolete robot that needs an upgraded processing unit.

AVA & Anna

Hah, curiously enough that’s how I feel every time I see a new update for Ava and Anna!

What about NoSQL databases

What about NoSQL databases such as Mongo or any of a whole cadre of others like it?  I have only played with Mongo a bit, but it is easy to work with and to scale horizontally as your data needs expand.  Microsoft’s offerings are a bit more difficult to work with in this regards.  Also, having it in the cloud would allow you to data mine memories etc using standardized big data tools such as 

It sounds like what you have done is to recreate what NoSQL databases give you out of the block although perhaps I am misunderstanding your design.  I know I haven’t thought about this as much as you have by any stretch, but it seems off the cuff that a NoSQL database might be a good approach.  These databases were designed to easily search through data that is not necessarily relational (such as data created on Facebook or data off the web) which kind of sounds like memories.  Basically, data is saved as json name value pairs and searches only find data by that name.  So, using your example, the query as to what color the bus is would generate “sql” that asks if there is a bus memory with a color in it, and if there is something there with a color and a bus in it for today, it would give that back to you.  Otherwise, the system can’t remember.

I am sure you are way ahead of me on this and looked at them already, but just in case, I thought I would mention it.

re: NoSQL Databases

Hi Bill,

Great Points:  I am so glad you posted this.  I really wish I had known more and thought about this.  Unfortunately, my DB skills haven’t gone beyond the more traditional relational approaches where things have to be modelled as tables, columns, and keys.  I have created a lot of in-memory db techniques too.  I think I am probably doing a little re-inventing as you mention, a big company once said I might have reinvented rdf ontologies.  On the other hand, my stuff is several years old now, so I might have invented some things too.  :-)

Along the NoSQL ideas you mentioned:  I think there is great potential in robotics here.  I think a lot of cognitive processes could involve evaluating a set of NVPs (Name/Vale Pairs) …doing a lot of feature detection, adding a lot more NVPs representiong the features to the existing NVP set, doing something, but also storing that set in a db for future use.  I think natural language stuff and visual or other sensor stuff could be done this way, capturing the element of time and therefore sequence of events as well.   At that point,some of your learning algorithms you have written and spoken about in the past might be a great layer to add on top.   I would think using a different type DB as you describe would greatly improve performance / flexilbity.  If I understand this, it also makes it easier to store diverse memories that may each be sparsely populated with a different set of data points.  Relational dbs are not good for this.

Cost Savings:  I use SQL Server on AWS now, which I believe is a lot more expensive than the other db types on AWS. (around $150/month ish per db server).  They have a tool that will migrate a db from one db type to another.  I might need to look into that.  I would like to at the very least buy you a beer at some point for giving me another excellent reason to study this and save bucks.

Moral of the story for any budding coders out there (from Goldilocks):  Sometimes you can spend too little time coding.  Sometimes you can spend too much time coding (me in this case), and not enough time learning whats already been done or emerging (me).  Sometimes, if you are lucky, you can get it just right, melding the best industry tools with your own best problem-solving.

I always say to any

I always say to any teammates who lament the same, “You did what you did when you did it and now it is done.  Next…”  You learned a lot along the way especially about the problem you are trying to solve.  That is gold right there although I am sure there are other nuggets in there from your Ava work.

NoSQL mostly really means Not Only SQL (most dbs support a version of sql) so don’t forget the RDMS skills for the right use case. Use cases where the data needs to be normalized (only one copy of the data can exist in the system), where the data model is known ahead of time and where the relationships between data won’t change the relational database makes sense.  If the data is really complex, highly hierarchical with many layers, the relationships between data change, the model isn’t known ahead of time, etc, then a NoSQL database might be the right solution.  

As to the beer issue:  I owe you probably a cold twelve pack of beers for all the things you have gotten me to think about and consider over the years here at LMR.  The discussion on consciousness a few weeks back, this discussion all make me question my creds as an engineer since I can barely keep up.  But if we ever happen to meet, I will enjoy this lone cold one and remember the one time I brought something useful to the discussion.

As to my learning stuff, it has a place and a valid use case (genetic algorithms or reinforcement learning) although it has been surpassed by Tensorflow.  Google has the resources to make it pretty awesome, easy to use and they have.  Plus a lot of tools are out there that leverage Tensorflow which (from what I have read) make it easier to work with.  I have spent a bunch of time on my version since I have been working on it so long and really wanted to finish it but realize from a technical perspective it is a waste of time.  I have two demos, one where it teaches itself pong and eventually learns to beat it, and a demo where a robot learns what the best strategy is if it runs into a wall.  Both leverage a REST web service in Azure, so the robot tells the service event such and such occurred, the service returns what strategy to try.  I have to do the videos for it but it is otherwise finished.  When it gets hot here, I want to be on the beach, not doing robots so might be a while before I finish my post.  But stay tuned…

re: maelh

A lot of great points you make, which I’ll attempt to respond to while going off on new tangents:

The “prediction” line of thought is one I would like to find the time to explore.

Movement is complex, and describing it in higher and higher levels of abstraction (the spirit) is useful as you point out.  This can be done in a relational db with entities such as gestures, poses, servo movements, etc.  Later, some other entity (like a verbal command) can reference a gesture, which effectively could relate a complex set of movements over time.

I found that my movements were not always precise, but often desriptions of key points where it was up to the robot to improvise over time while achieving key points / goal positions.  What you describe is very much more interesting though.

Subsumption may have its place as multiple confllicting goals need to be balanced.  Ava wants to move her arms when she talks to be more expressive, but as soon as she drives (other than rotating in place), she needs to get her arms in positions that do not affect her sonar array.   The ears have other issues to balance…basically her internal software is generating lots of goals/options all the time that need to be arbitrated.    …a tangent from memory.

I agree with you generally, especially so on ontologies.  They are expressive, but difficult for a human to understand on the whole.  I think the AIs can help us do that though, by explaining their reasoning.  I find the best way to understand an ontology with respect to a given question or concept is to write an algorithm to “surf” the ontology, and then get the algorithm to come up with an answer and output its facts/evidence.  A question like “Can penguins fly?” or “Are ants small?” has a lot of evidence in an ontology/memory base that leads to a conclusion.  Effectively, I think robots should have their “reasons” for a conclusion travel in the data along with their conclusion and “action”.  A given question can have multiple algos surf the ontology, each with its own reasoning.  This just means yet more data in the output stream.  Name/Value pair dictionaries/hashtables can help hold all this output.  The output can be transported over soap, xml, json, etc. if needed.

In reference to your final paragraph about translating to various representations, I have a proposal which I am finding to be very useful in many contexts both in and out of robotics.  What if all things can be translated to NVPs - Name Value Pairs, stored in a container together (A Dict) ???   This can be done generically with a row of any database table.  I have written an algo that can convert any xml into a dict.  The same can be done with JSON.  I am doing this for a commercial cyber product that has to deal with lots and lots of diverse external data feeds…the dict seems so much easier to deal with than complex customized structures.  The ability to normalize all information that moves around into a common container is useful.  You then need metadata about each unique key (name) in this global information namespace.    …we have arrived at the “context” I have wrtten about elsewhere.

I think the common container / context or something like it becomes the pre-requisite for a generic pattern recognition, temporal pattern matching, or any other universally desirable brain capability…if it can process a NVP dict and add value (literally ADD more name/VALUE pairs), it can move the brain forward.  This is also a way to tackle the code maintenence problem, as techniques like this can be done with a thousand times less code than conventional techniques.  The multiplier only grows as the technique is used more.

Then again, I could be a dinosaur…we could just wait for the deep learning people to figure out everything and render code obsolete.

Great Subject

Martin,

I wanted to be the first to thank you for bringing up this subject.

It has enlightened me on what other ways are out there and brought me out of the box so to say.

We need more forum topics like this that bring attention to the other ways of accomplishing goals in software.

Thank you

re: Jeff

Thanks Jeff!  Most welcome!  I am glad some of our community are interested in this area.

If we could only get more people to share what they do and how they do it here.  I got a chance to meet and speak with Pieter Abbeel one day at a workshop where we both spoke… what I wouldn’t give to get him and others like him to share here on LMR. 

Motors and wires are talked about a lot here.  Software has infinitely more “moving parts” and creativity, and so deserves a share of the attention in my opinion.