to mark a step as In order to build my simulator, I had to build the following elements:
- A Virtual World Environment with simulated physics.
- Some Virtual Robots with motors, arms, and lasers.
- Simulated Camera Sensors and Sonar Sensors
- An ability to control the robots individually, in groups, or make them autonomous.
- An ability to oganize the robots into multiple teams and roles within those team.
- An ability to organize robots into dynamically moving formations and have some concept of strategy, tactics, fields of fire, cover and concealment, etc.
- An ability for the robots to inflict damage on be blown apart into one or more pieces.
- An ability for the robots to reorganize in the midst of battle, appoint new leaders, medics, take on new formations, etc., as casualties happen in real time.
- An ability once a battle is over, for the winning teams (one with survivors) to put themselves and their former opponents back together, while balancing other goals at the same time.
Creating an Environment
The first step was to create a virtual environment for the robots. I did all this using a Unity frontend in C#. I started by creating a small piece of land with some hills around the edges and a small cabin and some trees in the middle. I made some areas of the terrain flat and others very bumpy to provide cover and challenges when driving over them. I also put a few other objects of interest in this world, like the gray wall in the pic below.
Here is the inside of the cabin. The point of this is multi-fold. The mission of the bots is to protect the cabin. At times, I wanted them to navigate up some small ramps, through the doors and cabin, and out the other side. This involved careful pathfinding and obstacle avoidance.
Creating a Robot
The robots were created by importing 3D models I had for one of my actual robots (AVA, here on LMR). Once all the parts for Ava, I had to carefully align all the axes to where the axes of rotation were on the servos in the bot. This took some hours, but eventually I got it.
Next, I needed to build a UI for driving the robot, moving all the "virtual" servos, and firing the lasers.
I used a combination of buttons, sliders, and code to implement all the capabilities of the robot. In the shot above, the robot is reattaching on of its lasers that had fallen off. It is also missing its right ear and left hand, which are lying around somewhere in the scene.
Simulating Vision
I implemented simulated vision by using ray tracing algos built into Unity. By finding the 8 "corners" and/or the center points of the bounding boxes around other objects and doing some ray tracing, it can be determined whether there is a line of sight to something or not. There are other ways to do this, I could have gotten the camera view from the actual position of the cam on the bot and sent that image to some vision processing algos. I went with something that was faster, as I knew I would have lots of bots running at the same time.
Simulating Sonars
I implemented sonars by creating invisible 3D objects that extend out from the bot. These invisible object represent the FoV for the sonars. Algos in Unity can then be used to detect "collisions" with these objects...these collisions represent sonar detections.
Simulating Physics
This was easy to implement as Unity has all this built in. You just have to configure gravity, mass for all the objects, and make it objects respect their boundaries and don't pass through each other. When an object falls on the ground, it needs to not pass through the ground.
Pathfinding
This was another relatively easy one, configurable through Unity.
Sides, Teams, and Individual Roles on a Team
This took a bit of programming, but I eventually got it so I could organize many bots into multiple sides, with multiple teams on each side, and multiple bots on each team.
A single bot could be a leader, a follower, a scout, a medic, or a casualty.
Formation Control
The leaders decide on strategy and formations and communicate that to their teams. There was a lot of work involved to decide what formation to adopt based on an opposing teams position and strength. Once a formation is picked, there are the particulars of how each bot should navigate to its position within the formation, hold formation, etc. A circle formation (or any other) is different depending on how many bots are on the team. A wedge formation is tricky if all the sudden the whole wedge needs to pivot.
Strategy
The teams had not only to pick formations, but whether to move, where to move, whether to wait for the other side to come, take cover, etc. A variety of options are available for aggressive or more conservative defensive postures. I knew I eventually wanted to use a neural net of some kind so what I didn't simply program my own tactical biases into the robots. I wanted them to figure it out for themselves.
Roles & Behaviors
One way I tried to stay out of deciding the strategy was through roles. I concentrated more on programming lower level behaviors into a "set of options" from which an AI could choose. Thus, if a bot was asked to be a medic and save its teammates, it would have some decent behaviors to follow.
Laser Battles
When a battle starts, the two sides assess the other and begin to maneuver into a formation relative to the enemy. Usually this involves one side perceiving an advantage and taking on a more aggressive formation and movement, taking on the role of an attacker.
Once some of the bots are in range of each other...lasers start firing and all hell breaks loose. Bots follow orders but they also take life and death into account as they try to survive. In my model, each robot could fire its lasers a couple times a second.
The explosions create confusion and the "fog of war". A typical battle is over in less that 30 seconds. Often one bot dies bot other bots survive taking cover behind their dead teammate, firing on the enemy with the enemy being able to fire on them. Then another body part can blow off and change the situation.
Damage & Body Parts
Each bot can take damage. Damage is represented as an overall health score but also by missing "body parts" that render the robot ineffective. Every time a bot gets hit, there is a chance that one or more parts can "explode off" the bot. A bot without a head can't aim its head mounted lasers. A bot without tracks can't move.
After the Battle
Battles end when one side gives up. These bots don't give up, so battles end when one side has no survivors.
Most often, one side has at least one semi-functional robot left. This bot can then repair itself and all the other robots. The way this happens is through a lot of prioritization and planning to decide what part to look for and repair first, or what teammate to save, etc. Each bot could have been blown into 20 pieces or so, so gathering all the parts from the battlefield can take a several minutes.
Former Enemies Become Friends
The former enemies get saved and reconstructed too. However, when they are saved, the winning side reprograms them to be on their side. This means a winning side gets larger and larger. Formations get more complex as a result.
Sorry I only seem to have pics of circle and wedge formations...Typically, they are a lot more interesting when a team gets larger it usually has a core team and multiple scouts.