An Overview of How Jeremiah
Works
Jeremiah is based around two
subsystems, a graphics system which constitutes the head and a vision system which allows
him to see. There is also a simple, built in emotion engine which allows him to respond to
visual stimulus via expressions or emotions. I am currently working on a new head model
called Saul (see fig 4,5) who is a deformable eigenHead models providing more lifelike
appearance and motion and the ability to speak.
The system is perfectly capable of running on
a single PC but for speed of operation each subsystem runs on a dedicated PC connected via
a network, built into a flight case for portability.
Jeremiah’s head is based upon the GeoFace
(DECface) articulated bone model with prescripted expressions for key emotions. Linear
Interpolation provides transitions between these expressions. He contains a simple
Newtonian model of motion with random elements of movement, random blinking and ambient
motion.
The vision system is based around a per pixel
Gaussian Mixture model of colour distributions using expectation maximisation within the
background segmentation framework proposed by Grimson et al with additional shadow
suppression and noise removal algorithms. This allows static background scenes to be
learnt dynamically, providing a robust segmentation of foreground objects. The system was
developed, and is used for visual surveillance (Link to appear, see publications). Jeremiahs attention is randomly distributed
between these objects weighted by their size and motion. Co-ordinates of objects within
the field of view are sent to the head model for animation.
The emotion engine probabilistically determines
the current state of emotions from simple parameters extracted from objects of interest
within the visual field. This simple set of rules allows chaotic behaviour in a similar
fashion to flocking simulations where a simple set of rules in a complex environment can
produce what appears to be emergent behaviour. Jeremiah likes visual stimulus - high rates
of movement make him happy. Jeremiah likes company - no stimulus makes him sad. Jeremiah
doesn’t like surprise - high rates of change in the size of objects make him
surprised. Jeremiah doesn’t like to be ignored - if objects exist but don’t move
then he assumes they are ignoring him and hence gets angry. Too much of a good thing
spoils the experience - if Jeremiah experiences too much of any particular stimulus he
will get bored with it and reduce its influence on himself.
Jeremiah in
Operation
Jeremiah was originally designed to work on a
single desktop computer where the act of waving at Jeremiah would stimulate a smile.
During either a large scale installation or performance, Jeremiah works on the same
principle and a typical set up can be seen in figure A. This figure demonstrates how a
camera mounted above the projected head provides a wide field of view for Jeremiah (up to
180’). The projector mounted above either the performance space or audience, projects
the head onto a black canvas. Within a few seconds Jeremiah is capable of learning a
background sufficiently accurately to segment moving foreground objects. The model is
constantly updated so subtle changes in scene structure or lighting are accommodated by
the model and do not lead to failure. If a chair is placed in his field of view then he
will see the chair. But if the chair remains static eventually he will get bored of the
chair and will incorporate the chair into his model of the world. However, if the chair is
then removed he will not get confused by the change in scene structure as he remembers
what it looked like before the chair was present. Therefore, any object which enters
Jeremiahs field of view can be seen by him. From all visible objects a specific object of
interest is randomly selected weighted by its size and motion i.e. large fast objects are
more likely to attract Jeremiahs attention. Once an object of interest has been selected
the eyes of Jeremiah look in that direction. Depending upon the physical difference
between the orientation of the head and eyes the head may follow to turn and face the
object. Given multiple objects of interest this results in Jeremiah sharing his attention
between objects. The simple emotions gained from this visual stimulus are reflected by the
current facial expression of Jeremiah i.e. the more happy he is, the more he will smile.
Figure 3 shows the basic expression of jeremiah (default, angry, happy, sad and
surprised). All this results in an artificial system which exhibits similar behaviour to
that of a child.
Figures 1 & 2 see Jeremiah being used as
part of a live performance ‘Blue Bloodshot Flowers’. Here he is one of the
performers opposite a live performer who interacts with him. As he has both random
elements and reacts directly to visual stimulus he never behaves in the same way twice so
the performer is forced to interact with him to try and get the desired responses. Once
the performance is over the audience is encouraged to enter his field of view and
experience him for themselves. 
However, the audience can be seen
by him throughout the performance. Normally the performer is the most interesting object
for Jeremiah to watch but should the performer remain inactive Jeremiah will respond to
audience presence and movement (he gets bored very easily).
Jeremiah was born from a security system
Jeremiahs visual system is actually based upon a system developed for visual surveillance,
a system which is designed to watch people and vehicles performing their every day tasks.
Waiting for something unusual to happen. Jeremiah has literally 'been born of this
technology' and is the emotional face or the automated security guard who watches
patiently for something to happen. Imagine now if we gave that system feelings. Boredom
when nothing is happening, anger when people leave and he cant, sadness, joy and surprise
when finally the event happens that he has been waiting for. This is Jeremiah, but his
emotions and interest in the world and people make him more like a child than a computer
system.
Download
Q: Can I have a copy of the source code ?
A: No
Q: Can I download Jeremiah and play with him
?
A: Yes (See below)
MSWindows Version6(build
c) 8435KB (.zip)
MSWindows Version6(build d) 8474KB (.zip)
Publications and further
information
Bowden
R, KaewTraKulPong P, Lewin M, Jeremiah: The Face of Computer Vision,
Smart Graphics'02, 2nd International Symposium on Smart Graphics, ACM Int. Conf. Proc.
Series, pp 124-128, Hawthorn NY USA, June 2002
Bowden R, Jeremiah:
Increasing Public Engagement with Science, Popular Science Poster at SET for
Science 2002, House of Commons Reception, UK. 18 March 2002.
Press and Media Coverage
Exhibit in Minnesota Science Museum (Current)
naturalinteraction.org 19/5/03
Article in "Super Interessante" April 2003
Exhibit in British Science Museum Feb 2002
Radio Interview for South African National Radio SAfm "Future Watch" 25/2/02
Interview for Greek National Press "TA NEA" 21/2/02
Coverage in Swedish National Press "AFTONBLADET" 4/3/02
Science
Photo Library
BBC news 8/11/04 (pdf article,
pdf front page)
kurzweilai.net
8/11/04
Tweakers.net 9/11/04 (Dutch News) (pdf article)
www.terra.com.mx 9/11/04
(Mexico news) (pdf article)
www.elmostrador.cl
9/11/04 (Colombia news) (pdf article)
www.oidossucious.com
9/11/04 (Latin American news) (pdf article)
BBC Southern Counties Radio Breakfast Show Surrey 10/11/04
BBC Southern Counties Radio Breakfast Show Sussex 10/11/04
BBC Southern Counties Radio Breakfast Show Brighton 10/11/04
BBC World Service Radio "Go Digital" (to appear)
BBC Radio 4 "All in the mind" (to appear)
NB: I know there is more, especially as I get
called by all sorts of people asking for details but nobody ever tells me when and where
these things appear. Ive got a few others I need to add but if youve seen Jeremiah appear
somewhere then please let me know and Ill add it.
