We would like to give an overview of the software architecture we have designed. Figure 1 describes the different software modules and how they are distributed over the difference pieces of hardware.
We decided to distribute time & CPU consuming applications on dedicated hardware. This is why we prefer to a camera with its embedded CPU to process images (So far Pixy but JeVois on Kickstarter now is another good candidate) as well as an OPEN CM9 micro-controller to execute the different postures that have been processed elsewhere.
All other applications run on the main processor board.
The objective is to create scripts which are a combination of actions and condition to be met in order to trigger further actions. Moreover scripts can be combined together. Another point is that these scripts will be used for communication between robots as well.
The formalism is based on Petri net where transitions contain conditions. It will allow us to include probabilistic values (to trigger a “sub-net” based on the mood of the robot for instance).
We end up with a “classical” software architecture with one centralized module running the scripts which is connected to different modules that will do dedicated computations - such as compute a sequence of movements to go from A to B - and produce a result. Result that will be used as an input by other modules – execute a sequence of movements for instance - or as a condition to be checked by the scheduler (transitions of the Petri net).
Figure 1: description of the software architecture
The following modules have been identified:
IK engine:this module computes the position of each joint based on objectives and constraints;
VoiceDB: this module contains a list of pre-recorded voice sentences (different tones) along with time markers that will be used to synchronize body movements;
CreateSeq: this module is in charge of computing the sequence of commands (speed and position) for each joint. In order to compute the sequence we need to setup a timeframe and the timeframe may be related to a voice sentence (by using time markers);
Scheduler: this module is in charge of executing a script;
PlayVoice: this module is in charge of playing a voice file;
Speech Recognition: this module is in charge of recognizing voice. We decided to use the Mycroft open source software;
Camera Server: this module is in charge of interfacing the Pixy Cam or another Cam which is coming along with its own processor ;
Play Posture files: this module is in charge of executing a sequence of postures issued by CreateSeq.
Mood Mngt: this module manages the mood of the robot and based on the current mood, different scripts (behaviors) will be triggered.
Figure 2 gives an example of the “hide and seek” script. Conditions have been written near their associated transition. Actions for each node have been listed on the right part of the Figure.
The scenario is the following: the robot is seated and we ask if it wants to play this game, if it accepts, it asks the color of the object that will be hidden somewhere around him. Then the robot hides its cam with the arms, counts until 10 (for instance) and then will look at the object in front of him then on its left and finally if necessary on its right.
As soon as the object is found, it will stop searching, express some kind of “happiness” and will ask to play again.
Based on its mood, we added some alternative scenarios, represented by clouds:
The robot can be stubborn, even though we say “no the game is over” it will keep asking to play again with a stubborn posture, let’s say it will cross its arms and keep the head down (Transition from Node 6);
The robot can be sad or imploring the user to play again, even though we say “no the game is over” (Transition from Node 6);
The robot can cheat at the beginning, meaning that while it is counting, its head will move so that the camera can track the object (Transition from Node 1).
In case the robot did not find the object, it can be suspicious and ask us whether or not we cheated, i.e., we did not hide any object (Transition from Node 4).
Many other sub-scripts can be added in order to enrich the “main script” with the objective to make the experience more enjoyable and create surprises for the end-user.
Figure 2: “Hide and seek” script