A simple emotional agent

Recently I came across the so called circumplex model of affect. When seeing the plot, I immediately associated it with the kNN algorithm. I have re-drawn and simplified the model by creating an Microsoft Excel 2010 scatter chart:

More formally: let the sample space be a 2-dimensional Euclidean space

emo_5.jpg

, the centre coordinates (0,0) and the radius

emo_e_1.jpg

, then our classes are located at

emo_e_2.jpg

chem_eq_8.jpg

emo_e_4.jpg

emo_e_5.jpg

emo_e_6.jpg

em_e_17.jpg

emo_e_8.jpg

emo_e_9.jpg

emo_e_10.jpg

Next we define

emo_e_11.jpg

chem_eq_7.jpg

and setting its boundaries

emo_e_14.jpg

emo_e_15.jpg

Maybe a few words about Pleasure and Activation before we continue. Activation could be seen as a kind of entropy of the robot in a physical sense, depending for instance on the ambient light level and temperature. Pleasure could be seen as a success rate of a given task the robot has to fulfill or to learn. Just to give you some ideas.

The distance between two points in a 2-dimensional Euclidean space are given by

emo_e_13.jpg

and follows directly from the Pythagorean theorem.

As we have set the boundaries

emo_e_14.jpg

and

emo_e_15.jpg

, so what is the maximum distance 

emo_e_16.jpg

between a class point and a new sample point that can occur?

Let me illustrate this with the following little sketch:

Obviously

emo_e_17.jpg

Before we write down the function, we declare the radius as a constant and assign a value to it:

const float r = 5.0; 

The function might then look as follows:

void Emotional_agent(float x_P, float y_A) {
  if(x_P < - r) x_P = - r; // limit the range of x_P and y_A
  else if(x_P > r) x_P = r;
  if(y_A < - r) y_A = - r;
  else if(y_A > r) y_A = r;
  float emotion_coordinates[2][9] = {
    {0.0,  0.0, r, - r, 0.0, r/sqrt(2.0), - r/sqrt(2.0), r/sqrt(2.0), - r/sqrt(2.0)}, // x-coordinate of training samples
    {0.0,  r, 0.0, 0.0, -r, r/sqrt(2.0), r/sqrt(2.0), - r/sqrt(2.0), -r/sqrt(2.0)} // y- coordinate of training samples
  };
  char *emotions[] = {“neutral”, “vigilant”, “happy”, “sad”, “bored”, “excited”, “angry”, “relaxed”, “depressed”};
  byte i_emotion;
  byte closestEmotion;
  float MaxDiff;
  float MinDiff = sqrt(2.0) * r + r; //set MinDiff initially to maximum distance that can occure
  for (i_emotion = 0; i_emotion < 9; i_emotion ++) {
    // compute Euclidean distances
    float Euclidian_distance = sqrt(pow((emotion_coordinates[0][i_emotion] - x_P),2.0) + pow((emotion_coordinates[1][i_emotion] - y_A),2.0));
    MaxDiff = Euclidian_distance;
    // find minimum distance
    if (MaxDiff < MinDiff) {
      MinDiff = MaxDiff;
      closestEmotion = i_emotion;
    }
  }
  Serial.println(emotions[closestEmotion]);
  Serial.println("");
}


The above model is very simple. We haven’t yet considered that emotions aren’t static. If someone isn’t suffering from mental illness, emotions aren’t static. They are dynamic. They depend on time. Usually nobody is always angry or happy or sad. These states are just moments in time. Complex biochemical reactions are responsible to stimulate an emotion. The human body synthesizes  chemicals, distributes the chemicals and then decomposes them. Epinephrine might be an example. The distribution takes a split of a second, but the decomposition takes a few minutes or longer. Hence we can conclude that the emotional agent will always aim to reach equilibrium; means it is always aiming to reach the center point.

A simple biochemical reaction or distribution-decomposition model is the following:

chem_eq_1.jpg

where

a: Decomposition rate of the chemicals in the human body, which are responsible for the emotion
t: Time
b: Base


Extrema:

chem_equ_2.jpg

Plot of f(t) with a > 0 (black) and a < 0 (red):

chem_plot.jpg

Now we equal the maximum function value to r:

chem_eq_3.jpg

Substituting a in

chem_eq_4.jpg

yields to

chem_eq_5.jpg

The limits for t→∞ are given by

chem_eq_6.jpg

How fast the function value approaches 0 depends only on b.  We can set the function value to 0 by an according if-statement if the function value has reached a defined constant close to 0, as it will never reach 0 for t<∞.

Example code for one axis (Activation). The distribution-decomposition is triggered by submitting 'a+' or 'a-' in the serial monitor.

const unsigned int interval = 100; // interval in ms
const float base = 1.2;
const float r = 5.0; 
long previousMillis = 0;
float Activation = 0.0;
float time_Activation = 0.0;
boolean Positive_Activation = false;
boolean Negative_Activation = false;

void setup() {
  Serial.begin(9600);
}

void loop() {
  String InputBuffer = “”;
  while (Serial.available() > 0) {
    char ch = Serial.read();
    InputBuffer += ch;
    if(InputBuffer == “a+”) {
      time_Activation = 0.0; 
      Positive_Activation = true;
      Negative_Activation = false;
    }
    if(InputBuffer == “a-”) {
      time_Activation = 0.0;
      Positive_Activation = false;
      Negative_Activation = true;
    }
    delay (10);
  }
  unsigned long currentMillis = millis();
  if(currentMillis - previousMillis > interval) {
    previousMillis = currentMillis; 
    time_Activation += float(interval) / 1000.0;
  }
  if(Positive_Activation == true) {
    Activation = log(base) * exp(1.0) * r * time_Activation / pow(base, time_Activation);
    if(time_Activation > (1/log(base)) && Activation <= 0.02) {
      Activation = 0.0;
      Positive_Activation = false;
    }
  }
  if(Negative_Activation == true) {
    Activation = (log(base) * exp(1.0) * r * time_Activation / pow(base, time_Activation)) * - 1.0;
    if(time_Activation > (1/log(base)) && Activation >= - 0.02) {
      Activation = 0.0;
      Negative_Activation = false;
    }
  } 
  Serial.println(Activation);
}


We don’t need to set the maximum/minimum function value necessarily to r. We can make the maximum/minimum function value depending on other parameters. Same for the base b. Therefore we can model the distribution of different chemical substances in the human body.

wims_3.gif

Combining the distribution-decomposition function with the circumplex model of affect and the nearest neighbour algorithm:

const float interval = 100.0; 
const float base = 1.2;
const float r = 5.0; 
const float cutoff = 0.02; 
byte buffer_emotion = 10;
long previousMillis = 0;
float Activation = 0.0;
float Pleasure = 0.0;
float time_Activation = 0.0;
float time_Pleasure = 0.0;
boolean Positive_Activation = false;
boolean Negative_Activation = false;
boolean Positive_Pleasure = false;
boolean Negative_Pleasure = false;

void setup() {
  Serial.begin(9600);
}

void loop() {
  String InputBuffer = “”;
  while (Serial.available() > 0) {
    char ch = Serial.read();
    InputBuffer += ch;
    if(InputBuffer == “a+”) {
      time_Activation = 0.0; 
      Positive_Activation = true;
      Negative_Activation = false;
    }
    if(InputBuffer == “a-”) {
      time_Activation = 0.0;
      Positive_Activation = false;
      Negative_Activation = true;
    }
    if(InputBuffer == “p+”) {
      time_Pleasure = 0.0; 
      Positive_Pleasure = true;
      Negative_Pleasure = false;
    }
    if(InputBuffer == “p-”) {
      time_Pleasure = 0.0;
      Positive_Pleasure = false;
      Negative_Pleasure = true;
    }
    delay (10);
  }
  unsigned long currentMillis = millis();
  if(currentMillis - previousMillis > interval) {
    previousMillis = currentMillis; 
    time_Activation += interval / 1000.0;  
    time_Pleasure += interval / 1000.0; 
  }
  if(Positive_Activation == true) {
    Activation = log(base) * exp(1.0) * r * time_Activation / pow(base, time_Activation);
    if(time_Activation > (1/log(base)) && Activation <= cutoff) {
      Activation = 0.0;
      Positive_Activation = false;
    }
  }
  if(Negative_Activation == true) {
    Activation = (log(base) * exp(1.0) * r * time_Activation / pow(base, time_Activation)) * - 1.0;
    if(time_Activation > (1/log(base)) && Activation >= - cutoff) {
      Activation = 0.0;
      Negative_Activation = false;
    }
  }
  if(Positive_Pleasure == true) {
    Pleasure = log(base) * exp(1.0) * r * time_Pleasure / pow(base, time_Pleasure);
    if(time_Pleasure > (1/log(base)) && Pleasure <= cutoff) {
      Pleasure = 0.0;
      Positive_Pleasure = false;
    }
  }
  if(Negative_Pleasure == true) {
    Pleasure = (log(base) * exp(1.0) * r * time_Pleasure / pow(base, time_Pleasure)) * - 1.0;
    if(time_Pleasure > (1/log(base)) && Pleasure >= - cutoff) {
      Pleasure = 0.0;
      Negative_Pleasure = false;
    }
  }
  Emotional_agent(Pleasure, Activation);
}

void Emotional_agent(float x_P, float y_A) {
  if(x_P < - r) x_P = - r; // limit the range of x_P and y_A
  else if(x_P > r) x_P = r;
  if(y_A < - r) y_A = - r;
  else if(y_A > r) y_A = r;
  float emotion_coordinates[2][9] = {
    {0.0,  0.0, r, - r, 0.0, r/sqrt(2.0), - r/sqrt(2.0), r/sqrt(2.0), - r/sqrt(2.0)}, // x-coordinate of training samples
    {0.0,  r, 0.0, 0.0, -r, r/sqrt(2.0), r/sqrt(2.0), - r/sqrt(2.0), -r/sqrt(2.0)} // y- coordinate of training samples
  };
  char *emotions[] = {“neutral”, “vigilant”, “happy”, “sad”, “bored”, “exited”, “angry”, “relaxed”, “depressed”};
  byte i_emotion;
  byte closestEmotion;
  float MaxDiff;
  float MinDiff = sqrt(2.0) * r + r; //set MinDiff initially to maximum distance that can occure
  for (i_emotion = 0; i_emotion < 9; i_emotion ++) {
    // compute Euclidean distances
    float Euclidian_distance = sqrt(pow((emotion_coordinates[0][i_emotion] - x_P),2.0) + pow((emotion_coordinates[1][i_emotion] - y_A),2.0));
    MaxDiff = Euclidian_distance;
    // find minimum distance
    if (MaxDiff < MinDiff) {
      MinDiff = MaxDiff;
      closestEmotion = i_emotion;
    }
  }
  if(buffer_emotion != closestEmotion) {
    Serial.println(emotions[closestEmotion]);
    Serial.println("");
  }
  buffer_emotion = closestEmotion;
}

I like it. I have to read
I like it. I have to read the original paper, but it seams easier to keep detailed track of only two emotions and figuring out the rest from that. And the positions seem to make sense.

How to define pleasure in an AI robot?

I’d base that on a combination of things, such as can it complete goals, doing things that it likes to do, not being yelled at or scolded (“Stifle, Edith!!!”).

I’m still not totally sure of Arousal. Perhaps it would be environmental (how much stimulus is there in the world around me?) along with internal factors (are my battery levels good and am I physically OK?).

These are just suggestions. I’d love to eventually have a robot that seemed to care about its appearance.

Thanks much, MarkusB, for bring this to our attention and making such a great example.

Yep, interesting topic. This

Yep, interesting topic. This mapping function is very useful if the robot is asked about it’s state and needs to make discrete assessment about it’s state (e.g. ‘I’m bored’). But what else can it do with these emotions? Could a robot’s actions be driven by these emotions, for example by trying to maximize this pleasure/arousal function? Could we apply this function to a robot’s decision tree, such that, for example, it would decide to drive towards the light instead of away from it, because the former decision would anticipate more pleasure/arousal? Would love to hear idea’s!

The big thing is that for
The big thing is that for the robot we have to define two independent things, pleasure and arousal. For the circle to really work, they should be totally independent.

Also, we’d have to define what a robot might do with these emotions. Every robot is different, as is every builder, but suggestions would still be worthwhile.

I’d also suggest changing the “Aroused” emotion to something different.

The interesting part it’s to

The interesting part it’s to modifying actions based on the emotional state. Another thing it’s that paying 1 and then paying 1, makes sadder than paying 2 once. Emotions are kind of fuzzy because it’s hard to say how much you are happy, but it’s easy to say if you are happier than yesterday.

So if it 100% happy, max happyness it’s raised by half, so that will be always something that makes the AI happier than before, but not so much that a few sad events could bring it down to ground. It also prevent extreme sadness. This way if it loses 30 times, but scores a last goal it will be somewhat happy:)

Pleasure=survival Arousal=goal achieving

Pleasure can be a function of the operating state of the robot, all functioning and full battery gets it happy, as one would be after a good meal or sleeping. Arousal may be a function of goal statuses, lots of new goal gets it excited, no goals bored.

It makes sense as many goals and low battery gets Angry(how **** do you think i will make it!?) and no goals and low battery gets Sad(noo, i’m useless).

I have seen that the

I have seen that the circumplex model of affect is already in use in robotics: http://www.takanishi.mech.waseda.ac.jp/top/research/we/we-4rii/index.htm#mental

It will get a little bit more complex soon as I will make Arousal Activation and Pleasure depending on time. Therefore a simple biochemical reaction model is considered to be responsible to stimulate an emotion.

 

The interesting thing about
The interesting thing about their emotional calculations is that they use a 3d emotional space, based on pleasure, arousal, and certainty.

I’m also wondering if short-term mental states such as embarrassment could be used.

All the deep math is WAY

All the deep math is WAY over my head… but I really like the main image! Very inspiring, so simple as a concept, so many things one could do with this.

You just made my tiny brain grow! This will be implemented somewhere at some point in time! Thanks!!