Current projects:___Cynthea ___Quantum Logic Synthesis ___CRIDE___Emotional Robots ___HPC
Cynthea - Cybernetic Networked Humanoid Emotional Agent
News and timeline
January 2010
the cynthea project is being reactivated as it contains potentially many interesting future technologies and applications.
first step: the manual and installation notes are all updated.
April 2008
The integration is nearing end. The input sensory modules included by default are (some of them are still in progress):
- - Motion Detection - reflex and reaction generations (only motor level - working on speech level)
- - Face Detection - following/disconecting from followed face
- - Face Recognition - Standard speexh and behavior generation
- - Music parsing and dance generation
- - Network interaction with other robots for interactive behaviors
- - Internet data parsing - accessing standard set of current news for dialogue and interaction generation.
December 15th 2006
Cynthea, has been improved and some bugs have been removed. More versatile command parsing implemented, so as the final phase is to be started. That is, Cynthea now has all modules ready for the performance and emotional expression, i need just to interconnect them. This means that Cynthea now has these various elements:
- fully functional CRL parser for script parsing
- sensor interface with the environment containing, speech recognition, face tracking, motion detection, dancing behavior to a given music
- networking interface allowing it to be connected on a network with other Cynthea robots and control their behavior using multi-robot CRL scripts,
- speech interface is using Alice (as a temporary solution)
- emotional layer is built into the robot and contains homedynamic system for spontaneous and behavioral expression
- and more.
September 1st 2005
During the summer, the whole Cynthea/CRIDE project went through several transformations. New functionalities ahave been added so as more support for various actions added. The orientation of the project is now aimed toward the emulation and consequent study for artificial emotions, using various robotic heads expressing their emotions through various devices
Main Changes in this release are on the higher level so they can be easily ported to any system running the lowest modules and drivers. The new functionalities are Vison and Motion Detection, Speech Recognition directly linked to a Alice Bot so as to the Robot Control Center, Music Analyzer allowing to the robot to dance and to be interrupted by external noises. CRL Language provide a nice interface to create new commands. The Motor Calibrator allows to precisely tune the servos range so as the speed and acceleration, for the optimal face and behavior expression
Energetic Layer and Strategic Layer are both installed, however are not yet stable. Dynamic Moods provide a interface to link all elements and their parameters allowing to tune on the fly the various devices in use. Until the Energy / Strategy problem is not solved the Mood Expression does not evolve
May 15th 2005
- Both OS ports are finalized. CRIDE provides yet an interface to execute behavioral scripts in CRL
- All code is now ported to Bio Modules containing Energy and Strategy self adaptive modules
- Dynamics of the information processing in each component is directed and altered by the internal state
- CRL now executes in two modes: Continuous and Serial.
August 15th 2004
- Windows port was tested. Using the SAPI 5 from the makers of the
popular environment allowed to generate a partial speech/mouth
synchronization,
- Most of classes have the Bio-Extension (still the core has to be changed),
- Sync and Continue are now operational
- Alice is integrated and allows to generate dialogues between robots -- check the video section
August 8th 2004
- First partial port to windows was tested. Unlike my expectation this
is done in less than one hour of code tweaking. Th final version of
windows version will be available end of this week,
- MNS is clean, and the temporal control of the commands is now much
more transparent and allows to design much nicer and smoother behaviors,
- Network dialogue is already up but running only partially because the
cause-consequence speech-answer-speech must be controlled somehow which
is not done yet,
- The first skins designed in the Portland Robotics Laboratory are
already shown in the pictures section. We tried a whole bunch of
different approaches, finally the most plausible is to use Latex foam
(home made), sprayed over the mask and fast dried allows to design
almost perfect skins/masks!
July 31st 2004
- Alice integrated into CRIDE,
- New CRL commands added specifically for temporal control and synchronization
- GUI improved, Server Improved, minor glitches eliminated
July 4th 2004
- Newly updated and recoded the command processing,
- The synchronization is now dependent only on the Speech-Synthesizer --- Few tweaks required
- The sensor interface was temporarily Disabled
Used Software Technology:
- JAVA virtual machine (Sun SDk, IBM Sdk) - Best works with JDK 1.5. The JDK 1.6 has some issues with AWT so be patient and it will be solved fast
- C++, C compiler at leas gcc in order to compile the openCV interface if required. , if you are using the i386 architecture you do not have to recompile the openclinking libraries - but you still have to install openCV. If you compile it from source be sure to use
--enable-static option - SWIG - Necessary only if you plan to change the Java/C++ interface functions
- JMF - Java Media Framework
- JAI - Java Advaced Imaging API
- openCV - the opensource image processing library, works with all versions above 1.0
- Quasi - java quantum simulator
- The MP3 Player and Sound Analyzer
Parts in development:
-- Vision (face recognition)
-- Sensor System: Quantum Emotional Simulator for behavior generation -- Multi-Robot interaction, task sharing. -- Web Spider and Info diging
Latest add-ons and generalities:
-- Speech recognition - Input through the recognition used either for feedback and/or for the user defined sentences. The current version supports yet only the direct Alice/user dialogue. Tested only on linux with Sphinx4 but the interface shuold fit and synthesizer supporting JSAPI.
-- Speech synthesis - tested on Windows (CloudGarden, JSAPI) and Linux (JSAPI, FreeTTS). Both Os's have free implementations of Speech Synthesizers: Windows provide SAPI 5 (5 voices) and Mbrola with FreeTTS can bu used with 5 voices.
-- Sensor System: THe main hardware sensory hooks are Vision system, Speech Recognition, Music Analyzer, Network Interface. based on the input from sensors under the skin (-- Sensory and Energetic system)
-- Server to control a multiple robot environment (assuming each robot is connected to a unique machine)
-- Upgrade of the CRL parse
-- One test build head with 8 degrees of freedom.
Operational parts:
-- Motor API (entirely written in java; designed to interface a large set of servo microcontrollers
-- Speech Production system: based on SUN Java Speech API and JSML, using free voice generators such as mbrola
-- Speech Recognition system: using CMU SPhinx 4 system
-- Face detection and Face tracking: using OpenCV linbraries. By default a precompiled libfacedetect.so is provided (compiled on a Slackware 12, i686 machine with standard system install). If you have a non standard install use LD_LIBRARY_PATH to adjust your system.
-- Motion detection and reflex generation.
-- Music parsing and motion generation.
-- CRL language specifications allowing to have a common ground to describe dynamical behavior of a robot
-- CRL parser based on the DOM and SAX XMLparsers
-- interface to accomodate different devices
-- Server to control a multiple robot environment (assuming each robot is connected to a unique machine)
-- Graphical User interface to edit the robot with multiple functionality
Install requirements and directions:
All components required either gglobally or for individual sysm must be correctly installed and linked. The SWIG interface links JAVA to C++/C so a compiler is reuired such as g++. The example of system settings illustrate the requirements.
Global Requirements:
- ALICE implemented entirely in Java (ProgramD - sources).
- Java comm API, implementation of Serial and Parallel protocol in Java.
- Robot with a ASC16 controller (optionally a simple PIC can be used).
- openCV, the Intel Open Source Computer Vision Library. The latest version is is now supported for opencv version 1.0.
- SWIG
Linux/Unix based systems:
- FreeTTS implementation of JSAPI (or any other compatible with JSAPI).
- Mbrola, linux implementation of speech synthesis binary and voices.
- SPhinx4 - the CMU speech recognizer written in java
Windows based systems:
- Talking Java, Cloudgarden implemented the interface of windows SAPI in Java.
- Speech Synthesis and Recognition:
- software such as Fonix or Naturally Speaking from Scansoft can be used. Beside these commercial application there is a variety of speech synthesis tools a list of some of them is here
Hardware requirements:
The simple use of the robot in a directed manner (such as Communication based on ALICE and logic perception) does not require lot of ressources as the only inputs are through the network.
The uasge of SPhinx4 tested and of the image processing requires however a Petium4 or better.
Stnadard usb2 ports and/or Serial so as converters usb/Serial.
A standard sound card with microphone input. A microphone.
A web cam, with RGB mode and a standard mode of resolution 320x160.
A 5V power supply for the head.
Environmental sesttings: :
- Example:
- JAVA_FACE_DETECT = /CoL/devs/devs/visionpackage/roebi/facedetect
- JAVA_HOME = (required >= 1.5) /usr/lib/jdk1.5.0
- JMFHOME = /home/workspace/libCoL/JMF-2.1.1e
- JAIHOME = /home/workspace/libCoL/JAI-x.x/ - CLASSPATH = $JAVA_HOME/lib $JAVA_HOME/jre/lib $JMFHOME/lib/jmf.jar $JAIHOME/lib
- LD_LIBRARY_PATH =
----- $JMFHOME/lib $JAVA_HOME/jre/lib/i386:$JAVA_HOME/jre/lib/i386/client:$JAVA_HOME/jre/lib/i386/xawt:$JAVA_FACE_DETECT
- LD_PRELOAD = $JAVAHOME/jre/lib/i386/libjawt.so (only if you get errors "unable to open data source while initializing the usb cam.