MDIBs - Metadata Driven Internet Brains

Community General Discussions
5

You could try PostgreSQL.
And yes, it does run under Windows and runs very quickly with large databases.

It sounds like you’ve taken something from OpenCog. I think I would use a bit more structure, with other tables for data of different atom types. On the other hand, I have a button that says “Why make something simple and elegant when you can make it complex and wonderful!” (Seriously, I generally try for simple code, but during the middle part of a project when creature feep it at its maximum things get complex before I can fold them nicely and refactor the code.)

I look forward to seeing this.

6

OpenCog: AtomSpace as a language
One of the OpenCog bloggers had an interesting post that claimed their AtomSpace had all that was necessary to be a computer language. He then went on to prove it quickly, without an example though. He compared it to Prolog.

If this is true, it would be easy to write sections of high-level code fairly easily. It would even be reasonable for a robot to write code of its own, because the distinction between a memory and running code is fairly small.

Just another thought.

EDIT: The blog title is “Why Hypergraphs?”

7

Have you tried sic’ing your text to speech service on Wikipedia?
Martin,

I’m wondering if a specialized version of your language-understanding-service would work on Wikipkedia articles.

It would have to be a specialized version in order to either skip or use the metadata available (ToC and the various strings from the Wiki managers such as “need more citations” etc.) Also, most Wikipedia articles are written in the third person and such.

Also, it would be an interesting experiment to run a natural language processor (NLP) with inference and error recovery on the “blogosphere.” I think I’d find it interesting from a linguistic standpoint. However, I think you’d almost have to entirely redo the OpenCog project in order to do this. (They have several interesting ways of improving the strength if a given atom.) There would be too many statements of “fact” that are merely opinions. Though, as long as you were working on a copy of your brain it wouldn’t matter if it got too messed up. :slight_smile:

What I would look for in the above experiment is to see if the NLP’s vocabulary could improve and how, and also how various opinions/facts are entered and change during the course of the experiment.

This would require an inference engine be built into your MDIB and that language (words, sentence fragments, etc.) be atoms. Unfortunately OpenCog seems to have written a parallel version of AtomSpace just for NLP, which is a huge duplication of effort, and also creates two versions of the code - one for normal atoms, one for NLP.

There are many things that OpenCog did right, though - truth values from 0-1.0 along with a strength value (and this is the simple model, there are more complex models), inference, the hypergraph, and the attention currency model. Unfortunately it isn’t finished now, but it is still being used and there are many full-time developers on this for an open source project.

I’m still up in the air as to whether I will use OpenCog, a derivative, your eventual service, or something totally different. I expect that I’ll try your service, at least, because it has the most immediate promise and you have a version that actually works and isn’t just academic farting around.

8

**re: Wikipedia **

Jay, I have tried a few variations of your query about Wikipedia.

I map words in my dictionary to terms in wikipedia.  This allows some questions to be answered by going out to wikipedia and grabbing particular bits of the xml.

Question Answering:  An example in the videos is when I ask questions like “What is the population of germany?” or some such.  There are a great many xml tags in the wikipedia data.  I have also tried tags about people so I can do things like ask how much a given athelete weighs or what team they play for.

Reading and Learning:  I tried writing a routine that grabs given articles, removes html/xml and other markup, and then attempt to find sentences in the remaining data that contain a given word.  Of the results, this routine currently generates about 30% garbage.  The rest are decent knowledge about the world that a bot could use in conversation.  Right now I don’t set it loose because I don’t want the garbage and the routines coule run until my harddrive filled up with excess trivia.

Blogosphere

Sounds interesting.  I’m not really a blogger, so I have trouble imagining the possibilities.  I suppose “infering/learning” from many different sources (Twitter?) could lead to a lot of robot knowledge or a bunch of fluff, depending on the source I suppose.  I would need a good way to filter out the garbage, like the wiki issues, or is some probabilistic inference approach possible?

NLP

I believe words, phrases, etc should be atoms.  Last night I converted all of Anna’s known words, phrases, and associations into Atoms.  This is paramount for word/phrase associations to occur.  Some people call these “triples”.  “Birds can fly”, “Humans are primates”, etc.  The next level would be triple+, something like “Birds have 2 wings”.  This might sound like a weird exception, but the one thing I am leaving out of atoms currently is the Princeton WordNet data, as I mainly just use it to look up the definition or part of speech of a word.  It has 210000 rows and I don’t think all that is particularly needed in the atomspace.  I could change my mind and import them if I decide to decipher their synonymn/antonym data, but I can get that sort of stuff from a web service.  I have been teaching those to Anna manually in speech.  Besides, I create a new “word” atom the first time the bot encounters it automatically, so I think I could do without around 190000 atoms that I will likely never use and can look in up in the other table when needed.  My next goal is to convert sentences, this is probably my most complex conversion as I have a lot of sub structure and sub types of sentences.

It looks like a good bit of OpenCogs triples and other brain data is available.  I could load these as Atoms.  That would be quite a knowledge set.

I include a “Usage” on every atom that tallies the number of times that word has been used.  That alone is probably not enough, I will probably also need a “probabillity” or “strength” for an atom.  Not worried there, easy stuff to create, just how to use it?

Attachment Theory and Trust Level

I am considering some type of robot/human “attachment theory” and “trust level” where the robot is inherently trusting of its creator and close friends (which must be earned), and inherently skeptical of others without multiple sources.  This means I will need to save sources for everything.  While that means a lot of “source” atoms, which are simple associative atoms to people, websites, etc, which are simple more atoms, it is useful to be able to interact socially and say things like “I heard from Jenn that your mother was sick.”, or “Wikipedia says birds are modern day dinosaurs.”  I think this is necessary.  Also inherent in this model would be “trust level” atoms to represent the level of trust that the robot has in a given person/source/website/etc.  

B.S. Detector

Ideally I’d like the robot to evaluate a new statement like “Penguins can fly” against prior statements from other sources like “Penguins can not fly” and determine which statements are credible and which should be forgotten, and which sources "Might not know what they are talking about because they seem to give me a lot of B.S. and lower their trust level.

Brain Viewing

I would like to make the brain “viewable” as soon as I can.  The sooner I can make it transparent, the sooner I can get help.  I have to convert all her existing memories to atoms, and write a webapp to view and edit the atomspace.  At this point I could make the memory “viewable” and editable.  The brain would not be usable though until I rewrite the various services and agents to use the new memories, and re-test everything  Thats before I make any improvements.

Its a lot of work, but I know it can be done, I know it will be powerful, and I know I could do it if I could sit down for even 2 weeks of straight time and crank it out.  Unfortunately, my days are filled right now with other summer labors.  It will happen though.  I would like to pull you and a few others in at that point and see if I can get a few to try it out and keep contributing ideas.  I very much value your ongoing thought contribution/leadership on this.

I’ve mellowed in my enthusiasm for OpenCog.  I’m no longer frothing at the mouth, I’m merely a big fan and student of their work. I still lack the computer science and AI background to grok a lot of techniques they are using.  Perhaps others can help on that.  I like Minsky’s ideas too and they seem similar (although I haven’t read his stuff, just wiki).  Minsky makes me think that my memory ideas and having a bunch a fairly simple agents for different purposes is a good foundation to build on.  We’ll see.

A final thought…“Gesture” atoms.  A gesture would represent a pose or a set of synchronized movements.  If robot responses can be associated with gesture nodes, then this facilitates more expressive behavior beyond just facial emotion expression, if a robot has arms, head movement, or other means of movement/expression.  Figuring out how to map that to servo movements will be on wish list for next year.  More atoms.

9

The blogosphere thing was mostly a thought experiment
I was thinking about the more prose-like areas of the web, and blogs along with some forums came to the top of my brain. This is something I wouldn’t try on the default brain, but a copy. There are way too many untrustable facts in blogs and too many opinions masquerading as facts.

I will help you with your brain as much as possible. I can test, I can code, I can build. As long as Lee doesn’t need the time first. I look forward to seeing it.

Be careful about having too many types of atoms. It’s probably very easy to do now. On the other hand, most of the math behind AI assumes typing, so you need different types of atoms. I wish I knew how many were ideal.

I also feel strongly that you should look at the OpenCog TruthValues. They have three different types (though one is just a list of other TruthValues). Reading their glossary and blogs (even though the math/philosophy is beyond me) I’m getting an idea of some of the ways these can be used. The TruthValues are very useful in filtering out crap. And they are essential to making inferences from old data to new data.

There are always going to be problem areas when you’re dealing with the real world and natural language.

The whole “birds can fly; penguins are birds; therefore penguins can fly”.

A real artificial general intelligence (AGI) has to deal with exceptions and also must have some non-Boolean TruthValues in it. For example, the statement “birds can fly” is a bad statement if told as 100% truth. Think of baby birds, think of birds with broken wings, and yes, think of penguins.

As for hard drives, they are inexpensive nowadays. I have a 3 TB USB 3 drive that I have free at the moment and may use for something like this.

10

re: Truth Values

I love thought experiments.  Einstein thought they were useful, and he turned out ok.

I will look into the truth values and see what they are doing.  I haven’t gotten that far.  If you figure it out and don’t mind summarizing, that would be great too. 

I dealt with the exceptions thing when writing the initial algos for Anna.  Anna doesn’t assume something is true just because she has one truth statement saying all.  She does look for exceptions if it seems relevant.   I wouldn’t be surprised if she could be tripped up though.  “Can birds fly?” can be answered by a bot in the general sense saying “Yes” or “Most can”, even though the bot knows of specific exceptions.  If you asked “Can all birds fly?”, or “Can penguins fly?” that’s getting a bit more specific, and deserves a more precise answer.  Anna can handle some of these currently, the others are fairly easily doable with a little more coding.

I had a concept of quantifiers…where words implied a value or probability.  It seemed useful at the start, but I haven’t touched it since I wrote it (maybe it is working).  The words mostly, some, a few, probably, seldom, never, always, etc. strongly imply probabilities.  Ordinals were also quantifiers, so “First” implied 1, and so on.  These could be used to sort things or to predict a truth confidence.

The number of atom types might seem artificially large as anything that has a limited set of values (something that would have a table with an int key and a varchar description), basically a list, could be an atom type.  Not all lists, just lists relevant to the brains internal functions, like my list of software agents, or a list of emotions.  Each of these could have additional data added to them in the future though.  Setting these up in the meta will allow the UI to show english readable descriptions of the info that is in each atom, and allow data to be changed using dropdown lists.  Otherwise, looking at the atom would mostly look like a bunch of integers and a few strings…a confusing matrix of gibberish.

The biggest foundational “weakness” between what I have thus far and opencog and other natural language systems is that I’m not truly using any of the academic theory on NLP with the parsing of sentences into NP, VP (noun phrase, verb phrase), and all the other symbols they use into a tree using probabilities.  It may be possible to add this later but it could be a real mess to not figure out how to fit it in now.  I mostly use a set of home grown techniques (as I have previously described) that amount to “feature extraction” or “question-answer translation” where some degree of implied meaning and structure are derived at the same time through regular expressions or data.  A regular expression of “Where is _____?” (and other similar expressions) can be mapped to an agent that determines location.  Normalizing the sentences first allows the system to handle some variations in the text, like “please” being in the sentence.  The system is very practical and easy to understand, but it is NOT NLP.  The huge weakness of this is exposed by trying to use anything other that simple sentence structures that the bot understands.  Anything with multiple “phrases” or several adjectives or adverbs will not be understood by this system.  I have read about NLP, but they get so into the trees and symbols, that I never get how to get practical “meaning” from the tree so I can map it to an agent that can do something.  If I could get how this might be done, it would be a huge leap forward.  

A note on gender differences…Sadly, as soon as humans start forming sentences with multiple phrases, I often lose the point of what they are saying.  Sometimes it is because the speaker lost the point and forgot to include it in the words, hinting, implying, etc., and telling me later that they told me something that was not in the parse.  Some would say its a mars venus thing.  As a guy I get the mars thing, but I’m not sure how a robot is ever going to parse through the venus thing.  Will I need a separate set of “Venusian Parse Algorithms”?   Food for thought.  Maybe they figured it out with Jibo.

11

Gesture atoms
I really like this idea, though there should be two types of them: robot gestures and human gestures (or meatbeing gestures, because cats and dogs make such things also).

Location and approximate location could be two more types of atoms or one atom with appropriate truth values. And then there is time, which might be an atom, but I think it fits better as an attribute of atoms. Arrrgghhh! Now I’m seeing atoms everywhere!

Thinking about human gestures makes me wonder how best a robot should think abstractly about humans. I think some sort of tree structure that could be easily assembled into a stick figure might be best. I remember reading a long time ago that stick figures are one of the better candidates for how humans think abstractly of humans.

Allowing gestures, which are collections of atoms, as a single atom, means that AtomSpace needs to be a hypergraph.

12

re: Your thought…Google or Amazon Vision Services Needed

A Google or Amazon Vision Service is badly needed…

Input:  An Image

Output:  An array of objects in the image, and their projected positions in 2D (in the image) and 3D space (relative to camera).

If this service existed, then we might be able to do the stuff you are talking about.

Anna already knows things like “A kitchen has a faucet”, “A bathroom has a toilet” so mapping words like “faucet” to whatever words google gave to things should be doable if the service existed.  Infering location should then be possible.

I would think if it isn’t there already in the new Amazon Phone or Google’s Project Tango or others, that there are teams somewhere working on this right now.  It is just too logical and too important not to do.

13

NLP
I just installed Julius on Groucho. Julius is a speech-to-text processor for Linux and also Windows. There is a partial English word model that can be used as as soon as I can find a microphone I’ll give it a whirl. There is also sphinx.

14

There was a Kickstarter that
There was a Kickstarter that was trying to produce this service, but I think they didn’t get funded.

15

It doesn’t have to be from
It doesn’t have to be from Amazon or Google. It could be from one of us if we had the knowledge to do this.

I could attempt it. In fact I’ll have to attempt something like this if Groucho is to get to where I want him to be.

The trouble is that vision is very much like NLP in that it involves some really deep brain structures and we only know the basic ones. I know that I can do it, but the algorithm I’d have to use is slow, and it wouldn’t work in all cases.

  1. Break the image down into concrete objects.

  2. Kill the background from each object, such that each object exists in an image all its own.

  3. For each image, compare to a database of generic objects in various rotations.

It’s step 3 that is so slow. Comparing images is slow when you’re not expecting an exact match. I’ll have to check on this. Maybe there has been progress. I’ll do some checking around.

16

Truth values and thinking
One of the things that cognition algorithms are supposed to do in OpenCog is to reward the atoms that “helped” the algorithm come to its conclusion. It does this by adding to some attention value associated with each atom. A cognition algorithm has a limited amount of “attention” it can give to atoms, so it’s a form of currency.

Truth values seem to get stronger the more they are used. There are different ideas on how TruthValues should be used.

Both of these are in the BrainWave blog that can be accessed from the main OpenCog page. Unfortunately without more experience with the system I’m as far as I can get. The programming I understand, but once it starts towards philosophy I can barely follow along, but if I lose my guide I certainly can’t find my way back. :frowning:

17

Some Progress to Address the NLP gap

Made some progress on NLP, someone ported OpenNLP to C#.  I have the source for it and its version of WordNet installed and running.

It has a lot of basic function that are far better than mine that plan on incorporating.  I am really hoping that none of these pieces will slow the overall brain down too much.

Split paragraph into Sentences

Split sentence into tokens (words and symbols)

Determine Part of Speech of each word using stats and maximum entrophy algo.

Finding people, places, dates in the sentences.  (I’ll still need to do mine I think, so I’ll do both)

Determine structure of sentence(s), full NLP parse into a Tree.  Even if I don’t use the full parse for meaning or mapping to agents/actions, this parse is a very useful annotator for determining tense, plural, etc.

It would also appear that it can interpret the WordNet data and get all kinds of “Is A” and “Has A”, many thousands of them, that I had taught Anna previously through speech or web page.

There are a lot of other features here, just getting going on real NLP.

The full parse trees are impressive, now trying to figure out how to use them to do something useful.  My regular expression stuff was simple and worked, determining structure and meaning in one simple step.  Should I start doing a new type of pattern recognition based on NLP Parse Trees?  I would think yes, seems like it would be complex and slow though, so my thought was to do it only if I didn’t get a sufficient match first using my existing techniques.  I am trying to figure out the types of statements that would be useful to a robot that would lend themselves to recognition with “Trees” rather than direct sentence matches or regular expressions.  I would have to come up with a way to match parse trees based on search criteria…which might circle back around to regex.

I’ll worry about that later I suppose, uses for this will emerge.  Happy to have some new tools.

18

Stick Figures w/ Kinect

I can see the recognizing human gestures with stick figures using Kinect.  One of the middleware libraries for it outputs stick figure representations of some kind.  We do some of the XBox dance stuff for fun in our house during family gatherings, the newer ones can recognize at least 4 people/stick figures at the same time.

It will probably be a long time before I can take that one on, like to crawl before I run.  My thought of crawling or walking would be to tackle “Robot gestures”.  I would start mapping emotional context and gestures to responses.  An example would be, when the robot says “No”, a simple less happy emotion and facial expression, a looking slightly down, and a slight horizontal head shake would seem good to synchronize.

19

Good!
Anna was already impressive. Now she will be more impressive. I can’t wait to see what you develop from all these pieces.

20

Different Sensors
I know that somebody is doing a database of images from the Kinect sensor, but what about other sensors that are more diffuse such as the various ultrasonic sensors available?

I can picture identifying a room by its sonar fingerprint of s room, but this is something that has to be up to the individual Builder. Each robot will have its own set of sensors.

Just some thoughts.

21

re: Diff Sensors

I heard about the Kinect data from another member.  Interesting.  I wouldn’t know how to do the math on the pattern matching for that.  It could be very useful for human gesture / stickfigure recognition, which I wrote a little about elsewhere in this forum.

I hear there is some new thermopile camera attachment that hooks up to a phone for about $300-$400.  A db of images and pattern matching based on heat signatures could be useful.  Once again, the math is beyond me right now.  Anna’s thermopile is only 16x4 pixels, probably not enough to do anything all that great.

I have long wanted some sort of visual landmark memory.  One that might be simple enough for me to make work…I speculated that if you pointed a robot due west, and recorded the rarest color every say 5 degrees, swinging around to the north and around to due east, adding each “rarest color” into a color “barcode” of sorts.  (Instead of rarest color, the change in color (up or down) might work better so as to remove the significance of the actual color number in HSV color space.)  You could do it south as well, or in a full 360.  Then, if you could pattern match this “barcode” against a memory of barcode scans from around a house.  This might be able to tell you where you are in a house.  If there were a couple of rarer patterns then in the barcode, the bearing could be found to those, and a position could be estimated within the room.   This is just a theory.  Its really localization and not a general purpose visual memory.  There are some OpenCV techniques that are probably more sound, I was just trying to come up with something new.  

I am doubtful on using sonars for any kind of pattern matching, although I do get a bit of some of the localization techniques based on them.  At the least, my thought would be to allow the developer to “Add Sensors and Services” to their bot using the website.  This would include sonars and algorithms, specifying the orientation of the sensor.  Maybe a set of algorithms (like Force Field) could be present that could be turned on and configured for the specifics of the given bot.

On a related “pattern matching” front, I am still trying to wrap my head around what a hypergraph really is and how I might pattern match hypergraphs.  Is that is even a valid concept?  I’d hate to build something that it conceptually similar to hypergraphs without realizing it, and not be able to benefit from the mathematical work that has been done on them.  Also, how to pattern match natural language parses?  So much of this is new to me.  You implied somewhere that you understood the math behind some of these things.  I don’t yet.  Calc and Stats was as far as I got.

22

Hypergraphs
Hypergraphs are, as I understand it from the OpenCog blog, merely a form of graph that breaks 2 rules:

  1. A node can be a graph or hypergraph.

  2. A link can link many nodes to many other nodes, instead of just one node to one node.

Both of these things have been done in relational databases for a long time, so I think you’ve been using hypergraphs without realizing it.

You can convert any graph into a hypergraph; in fact every graph already is a hypergraph. I believe that you can contpvert any hypergraph into a graph, but I can’t prove it.

The advantage of a hypergraph is that seems a better model to store information with for cognition. In other words, our data storage is more than a bit chaosy. Another advantage is that it seems to be easier to do pattern matching on hypergraphs.

And I may have had a couple of math classes more (probability, proofs, and differential equations – I remember very little of the last two), but I do read a lot.

23

I’d try OpenNLP FIRST
I don’t know how fast OpenNLP is compared to your algorithm. I would try getting a parse tree to see if that can get you to the core of the sentence. It might be more accurate than your routines, especially if you can find a way to keep the modifiers (numbers, etc) as annotations or whatnot.

Perhaps each word could be an atom, or maybe just the core of the word. A sentence would be a graph that connects the words in their proper order (and also be an atom). Perhaps the proper order is as a parsed tree, I don’t know.

24

Mixing of OpenNLP and Anna’s Brain

Thanks for thinking about this Jay,  I believe we are on the same page generally.

This is my latest over-simplified pseudo-code of how to fit my prior code and OpenNLP together:

Sentence Comes In as Input:

1)  Calculate “Normal Sentence” with “Annotations hanging off of it”.  There are a few versions of the sentence held in different structures for various agents to use as they best see fit.

1.1)  Determine Tokens - Open NLP

1.2)  Do OpenNLP Parse - The first sentence takes a second to warm up OpenNLP and WordNet but after that I barely notice a speed difference if at all.

1.3)  Group Words Into Phrases - My stuff

the OpenNLP tokenizes into words, but my own routines recognize more multi-word phrases like private people, learned phrases, countries, states (New Jersey) as single concepts, which is good.  Maybe the OpenNLP tokenizing could be rectified by modifying the WordNet db that OpenNLP uses when tokenizing.

1.4)  Annotate Sentence with singulars, plurals, tense, adjectives and adverbs from NLP parse, and REMOVE all these words from the “Normalized Version” of the sentence that will be used for pattern matching.  This leaves me with the “Core of the Sentence” as you said.  This removal will have the HUGE initial benefit of making all my existing processes “match” when the input sentence contains adjectives, adverbs, etc. that the regular expression didn’t expect.  I can make the agents where this “additional color” is relevant look for this color in the annotations when they do their individual processing functions.  i.e.  The difference between “Go Forward” and “Go Forward Slowly” is an adverb that would be an annotation.

2)  Find Agents that Match “Normal Sentence” in Some Way.  The following basically represents a prioritizing from “Exact Matches” that take priority over less exact “Pattern Matches”.

2.1)  Look for Sentence in List of Commands (AtomList) and return commands to caller.  'Drive Forward"

2.2)  Look for Sentence in List of Chat Requests (AtomList) and return one of specified chat responses to caller - this is a random routine that checks responses against recent history so as not to repeat.  “Whats Up?”

2.3)  Look for 2nd Person Sentence in List of Questions (AtomList) and process accordingly if found.  (this is a topic unto itself)  “How old is Fred?”, translates to “How old be you?”, question is found and Fred’s response to this question is found.

2.4)  Find Regular Expressions that Match Sentence and execute corresponding agents.  “Where is London?”

2.5)  Find “NLP Patterns” that match Sentence - must invent this if necessary.  I need a way to define a bunch of NLP parse expressions with wildcards, and then a way to determine which of those “NLP patterns” match the input sentence’s NLP Parse Tree.  “How would you describe the fox?” - from the prior sentence “The quick brown fox jumped over the lazy dog” …the answer "The fox was brown and quick."

A bit on atom storage.  I am proceeding with data conversion.  Words, Phrases, and Sentences are atoms.  The Keep It Simple Stupid part of me stores sentences as strings rather than some ordered list of IDs to other Word atoms.  Maybe when I finally get the Graph/Hypergraph concept in my gut I will change my mind.  Is a string/word/phrase/sentence just a graph of ASCII characters anyway?  I don’t see storing sentences as strings as a problem right now and there are many advantages, as I can always lookup a word atom based on its string key or its int key.  Its a whole lot easier to comprehend what’s going on in the system if I can see words, phrases, sentences, etc. as strings that I can personally relate to.  I have been thinking about storing a full NLP parse of each sentence in another column, thinking that maybe I could then use SQL and LIKE expressions to do the “NLP Pattern Matching” process described in 2.5 in some way.

Try as I might, I can never write short posts on this topic.  I hope this though train helps some future hobbyists out there.