I incorporated the following off-the-shelf neural nets into the brain of this robot. Most of these models are running in python on a separate laptop with a graphics card, and accessed through a custom built flask api.
- NLP DialoGPT Model (Microsoft)
- NLP Text Generation Model
- NLP Sentiment Detection Model
- NLP Masking Model (Transformer)
- NLP Question Answering Model (Transformer)
- NLP Entity Recognition Model
- YOLO v3 DarkNet Model
- AlexNet Model
- Gender/Age Detection Model
- Face Detection Model
- Emotion Detection Model
I incorporated the following libraries and algorithms into the software. This list is only the major ones.
- Spacy NLP Library
- Open CV Vision - Various algos for color, shape detection, etc.
- A Star Pathfinding (in Python)
- 2D Occupancy Grid Map (in Python)
- 3D Memory System - (in Python)
- Various custom built NLP Algos
- Various Graph Algos
- Fast Fourier Transforms (for audio spectrum analyzer)
This bot learns on its own any time any new word is encountered. For this, I use the following data sources.
- Word/Thesaurus API
- Wikipedia Text and API
- RDF Triple Sources (Dbpedia and others on Linked Open Data Web)
- Wolfram Alpha API
- GeoNames database
- Some custom built SQL Server databases
- Weather API
A black and white image of a depth stream:
A 2D map created from the depth info but with a force field applied around all objects. This is used for pathfinding.
A portion of a 2D map...before the force field is applied.
A representation of the sonar data input.