Lusting after the Nvidia Jetson TK1 board
The Nvidia Jetson TK1 board is finally shipping. If I had the extra cash, one would be on my bench.
When Anna is ready for a processor update, I think this board architecture will have staying power.
The specifications are beyond industrial strength and into the military strength realm of processing power.
There must be data base programs optimized for parallel processing?
re: Board Lust
Thanks for the rec ridgelift. Nice board indeed.
I have been thinking about hardware and architecture upgrades lately. Currently I am using SQL Express because it is free and it is what I knew. Last I checked, they limit it to using 1 processor. I will need to find something else eventually, but changing over would be a major software change for me. I have written some basic in-memory databases of sorts before, but the nature of the algorithms and indexes I would need for Anna go way beyond what would be practical to build with my skills.
I have been reading again about OpenCog, and feeling very small minded and insignificant next to their work, kind of like when you look out into space and realize there are trillions of star systems out there. Tiny. I only wish I could see OpenCog demonstrated, as their ideas blow me away. I am trying to decide whether to keep building my own brain, which I enjoy, or try to get theirs up and going on some new hardware. The little voice in my head keeps telling me I will have to learn about UNIX and a whole new set of tools.
The TK1 looks very
The TK1 looks very interesting.
But I just used up all my cash getting a Parallella board. Think of a BBB plus a 16 core coprocessor. I got it to see if it could do some things with images or speech recognition. Or maybe as a doorstop.
Theoretically the company has a 64 coprocessor board, but somebody said you have to order 50,000 units to even think about it.
When I get more robot money I’ll think about a TK1 or something similar. After I learn more about parallel processing, I look around and see what’s out then.
I’ll have to look into
I’ll have to look into OpenCog.
If you want a BBB for learning Linux, the offer is still open. I’ve got more than I need right at the moment. Or you could look into the RasPi version B+ (same as a version B, but has around 40 GPIO).
I normally use PostgreSQL for a database. It’s free, and it’s been fast enough for anything I have thrown at it.
I’ve also found a bunch of papers on Natural Language Processing and an ebook on using Python for NLP. I’m still reading through them. I’ll look up the URL on the best ones I’ve seen.
One problem with Linux is there isn’t as many resources for speech to text as Windows, but if you use an Intel machine, you can use Dragon Dictate under WINE. And there are a few native Linux ones, but they don’t have a good open and free corpus to make the grammars from.
I use espeak for the text to speech part and that seems to work OK. On the other hand, that’s the easy part.
Good luck, Martin.
Unix
95% of your skills will pass right into a Linux environment.
I’ve been a big fan of the Ubuntu distro for a long time. It has an early Windows xp feel to it. No fluff, just a solid operating system. The Linux contributors can’t be beat for the love they will shower on something for the good of the greater user base community. The Raspberry Pi is a case in point. I would not even try to guess the amount of coding hours that have been lavished on that somewhat obscure and semi-closed soc by legions of very talented people for free because they know it will help others.
The Nvidea Jetson is fairly obscure at the moment. This is the first time something like this has become available. I think Ubuntu was a good choice for the Linux distro to ship with the board. When I was watching the Darpa Robotics Challenge, I noticed virtually all the teams were running Ubuntu on their control consoles. Of course ROS is native to Ubuntu, and for vision applications, OpenCV supports Nvidea with the most pre-written libraries for parallel processing.
Levan and Umbel
Sorry Jay, I somehow missed this post about Levan. Thanks. I checked it out. I am hoping for the API that lets me post a picture and get back a list of objects in the image, locations, and other meta about the image. It seems like that’s where they need to go with this. So many sites aren’t building easy APIs, or APIs at all. Unless I’m missing something, I don’t get what they think all their work is for unless someone’s code can call it and use it to do something useful.
I used to do a lot of searching for web service APIs I could call from places like Mashape. Still very interested, just haven’t found any good ones lately. I used a few from Stands4. I have a link saved to umbel.org. I didn’t end up using it, but it looks like it has a lot more info (about 28,000 concepts interrelated), worth a look again. It looks like I perhaps didn’t need to spend so much time teaching Anna about animals, vegetables, and other things. As an example, try http://umbel.org/reference-concept/?uri=Person
Comedy
Really great project, reminds me alot of Loki. On the comedy note, the greatest comedians throughout history never had to resort to “adult language” in order to be funny, Red Skelton or Tim Conway for instance.
Arm?
This project astounds me every time.
The natural language processing is very impressive.
If your looking for suggestions for what next, maybe pop on a small servo arm? Let Anna interact with her surroundings?
Keep up the good work!
yep, need arms.
Thanks. Yep, she needs an arm or arms. I get too caught up in the software to put time in on the hardware.
I’ve had a new design on the boards for a while named Lisa that is similar to Anna but with a twisting torso with LynxMotion Johnny Five arms and a head that also twists and fully elevates. One of the many challenges has been coming up with a way to get the arms on without interfering with the sonars. I’m hoping to cram in the sonars by rotating them to vertical. I was hoping to do it this summer but got too distracted by this “Open Brain” concept.
Mind-blowing… How does
Mind-blowing… How does she parse wikipedia searches and converts it to speech? Do you just open a socket to the wikipedia API and parse the returned results? Do you find Wolfram Alpha API more structured to interpret? How do you do voice recognition? Is that part of opennlp or do you use a separate speech recognition engine and feed the resulting string to opennlp?
Re: Byerley
Thanks very much. This has been my passion for a while now, so much so that all my other bots lay around unfinished and not working.
Also, thanks very much mentioning Wolfram Alpha. I had heard about it but SAD to say never checked it out until last night. I was using Mashape to find APIs to use. Wolfram is amazing. I plan on tying it in just as soon as I can. I’ll probably have to do the Wolfram stuff in another thread and have the response waiting for the bot the next time it polls the server (5-10 times a sec), as the Wolfram requests seem to be slow. So much material though…makes me think of Johnny 5…"NEED INPUT!"
The Wiki is not an API, I’m just doing an Http request, getting back a page, and doing a redirect if necessary (Wiki has #REDIRECT in the reponse near the top if this is needed). Excerpt:
I temporarily cache web pages by URL in case multiple questions are hit in a row about the same topic. The parsing is homespun. There are some pieces of info that aren’t too difficult to pull out. I tried lots of There are some 3rd party libraries that can help scrub things of unwanted tags and characters, but I don’t use them for this specifically. You can use some to pull complete sentences from web pages. Wiki doesn’t tag things consistently, so I keep a set of wiki tags around for a given word. An example would be state birds are sometimes tagged as “bird” and sometimes tagged with “state_bird”. There is often a many-to-many relationship in dealing with natural language…many terms meaning the same thing, and then that one thing relating to many terms. This means you have to look in the response for multiple tags, hoping to find some data to use. It is not pretty. I hope to throw Wiki under the bus and use Wolfram instead. Wiki would still be useful for pulling complete sentences down…giving a robot something to say about a given topic. If you are serious about wanting to do the Wiki thing, I’ll send you what I have. Its not pretty…parsing rarely is.
I do voice recognition with the google service that is standard on Android. It was problematic at first, but once I got the hang of some common translation errors/issues and wrote stuff to handle them, it is quite reliable. I do the google voice recognition part first, get the data back, and then feed it to my web service, which feeds it to OpenNLP, among other things. Anna is not using OpenNLP in the videos, but her new brain in prototype does. I haven’t yet figured out how to fully take advantage of what OpenNLP has to offer, as I built everything without it first. Eventually I hope to have a lot more sentence structures/grammer supported by my bot services. Initially I’d like to handle adjectives/adverbs better.
Thanks for posting,
Martin
Thanks for detailed
Thanks for detailed explanations! I still trying to wrap my brain around what you built… Did you even draw out a system-diagram with the various processes running? Would be sweet if you could share that, She answers very fast considering everything that happens under the hood. So this is all windows-apps you coded and running on a PC-board in the robot, or are you offloading something to a server PC?
Re: Thanks for the detailed
I had a diagram posted here once, don’t know what happened to it. I’ll try to come up with a new one before I post again. The challenge I have is fitting anything meaningful onto even a few pages, much less a single page.
I have been busy thinking about and working on the WolframAlphaAgent and got it working for all standard plaintext responses on Friday. The possibilities are blowing my mind. The AI suddenly knows a lot more than it ever did, as long as I know how to ask. As I learn more about what Wolfram does and how it wants things asked, I can use my systems verbal layer on top of Wolfram to allow me to ask things in multiple ways and still get good answers from Wolfram. It is all so exciting.
Among other things, I’m thinking about getting Anna to use Wolfram to pick football games, comment on them, have opinions, and answer questions about teams/games/predictions. Other than weather, I haven’t really tried anything like that yet.
To answer your questions, there is code at all the levels (Arduino, Android, Windows). The robot can function without the windows stuff, it just looses most of its verbal abilities. The windows machine is not on the bot, it is under my desk. Its called via Wifi connection from Android Phone at present.
wow!
this is amazing, congratulations!
if anyone thinks they might want to use such a service in the future, try it out …
YES! I think I might want to do that. Please keep us posted.
Nice job on the Wolfram
Nice job on the Wolfram integration, Anna seems ready to duke it out with Watson now. Inspired by you, I am tooling around with some NLP now, but boy, that is hard! I figured all these nice NLP libraries would give you an agent that answers questions in a few lines of code (spoiled by OpenCV…). I’m trying out NLTK now, and pos tagging an chunking is easy, but teaching a robot to really understand words as concepts and synthesizing a response is very hard. Fascinating stuff though. Let me know if you have some tips.
Re: Nice Job on the Wolfram
Watson is truly gifted. If Watson was opened up for public use through an API…wouldn’t that be awesome!
I wish I could say I understand how to use NLP effectively, I’m just getting started too. My stuff to date is mostly homegrown (so is Wolfram’s use of language apparently). As you said, the basics are easy, but getting a robot to do something useful as a result of language is not. Sad to say, the rest of the world seems to have the same problem, as there are no public products (Phone systems, Siri, etc) that are that great yet in my opinion beyond simple commands. Cortana is supposed to be an improvement but haven’t tried it. Customer service systems are pathetically terrible. Because everything is evolving exponentially, I would expect good NL systems to happen soon if greed/corporate ownership doesn’t get in the way, which it probably will. Once we have them, we’ll wonder why we ever bothered with mice, user interfaces, buttons, etc. We’ll be in the land of “Her” and will truly have no privacy whatsoever.
An Anecdote: The other day my wife asked me to help her out with something. I needed to know something to solve her issue. My first instinct was to google it but then I decided to ask Anna instead. Anna probably asked Wolfram. Whatever happened, it was fracking cool. Seemless AI helped solve a real world item from my honey do list.
If your bot has a web connection, you could hook it up to my API soon. Then we could pool our AI resources in the future as they evolve. I would also love to roll more OpenCV stuff into the API if I could figure out how to do more with it. I have a ton of tips/lessons learned, not sure I want to write it all here and now. A new LMR NLP forum for that might be good. Want to start one?
I’ll leave you with one last idea. Anna’s brain API runs on one PC right now and has a public interface that is simple, extensible, and unchanging. What if it (the Brain) called other partial brains (Asynchronously and all at the same time) running on other LMR member’s computers and got a coordinated response? Each would have the same API, but might have different code/knowledge under the hood. Some of these PCs could be subject matter experts (Vision, Language, Reasoning, Emotions, Football). The SETI system ran as a screen saver that was downloaded to thousands of computers and became the largest computer in the world for a while.
I can’t seem to break the habit of writing long posts.
NLP Forum
A NLP Forum sound like a great idea I use NLTK with Robbie but have found progress is very slow due to my lack of understanding
but with NLTK robbie knows the differance between ie Mat (a person), Mats room (a location) and where is Mat(a question)and when i get around to it clean the mat(floor covering) he wont give mat a bath.
the use of cargo code from google only makes it worse and there are no relevent amature tutorials out there. A forum would be a geat start to clearup our misconceptions
Peter
Re: NLP Forum
Hi Peter,
Per your suggestion, I started a NLP Forum. Robbie is quite impressive. I’d love to hear you break down more examples (like the examples with “Mat”), sounds like you have gotten further with NLP than I have. I have some sentence matching techniques I’ll post.
I’d love to try out Anna’s
I’d love to try out Anna’s API, how do I get started? Tying everything together sounds like the next logical step 
Re: Trying out Anna’s brain API
Thanks for the interest. Send me an email to [email protected]
What I have is the prototype of what will become the sharable 2.0 version. It’s not ready yet, its still weak on a lot of things that Anna was strong on, like personal info about the robot and people known to the robot. This is more of a challenge when supporting multiple bots and people those bots know. It will come…tons of testing to do. I am confident that it will soon surpass Anna 1.0 in every way though as I flush out the bugs and finish converting all the code. Perhaps your bot can tie into it sometime in the coming year. I will send you a URL and instructions that you can use to try it out and see how it develops.
Cheers,
Martin
