Digest of previous email discussions (part 2 of 7)

[hannes]

============================================================================
Date: Tue, 21 Mar 2006 09:22:15 +0100
From: Stefan Kopp <skopp_(at)_techfak.uni-bielefeld.de>
To: Hannes Vilhjalmsson <hannes_(at)_ISI.EDU>
Subject: Re: IVA  paper: new version
...
> - There is therefore no need for a special FML/BML sub-section in the
> Existing Representation section of the paper.  But I can add a
> sub-section describing representations in BEAT and Spark.

Yes, I realized that when I scanned the page, too. Looks cool.  I also
think we should stick to the names FML and BML.
...
> (V) What other attributes describe behaviors, defining certain common
> terms like "amount", "angle", "target", etc. so they can be used
> consistently. (the point here is that researchers may be interested in
> a different number of attributes that describe the behavior with
> different levels of detail - we need not agree on the number of
> attributes!  *but* we need to agree on the definition of some key
> attributes so that when we choose to process an attribute, we know
> what to do with it)

Yes!
...
best,
Stefan
============================================================================
From: Hannes Vilhjalmsson <hannes_(at)_ISI.EDU>
Date: 28. mars 2006 02:12:05 GMT+02:00
To: "Kristinn R. Thorisson" <thorisson_(at)_ru.is>
Subject: Re: comments and edits in paper
...
My responses to Kris's comments in the document:

- [KRTh2] I like the expanded Ymir/YTTM context and background.   
We'll need to re-word the introduction to section 2 to explain 
we're not just presenting alternative XML languages, but also 
approaches to generation (does that mean we need to include more 
background/references to communicative behavior generation in 
general?)

- [KRTh3] Real-time considerations are important, there are still 
some outstanding issues here of course.

- [KRTh4] I absolutely agree, there was some discussion over email 
about this, and I think Stefan also agrees that we should leave the 
Gesticon ouside of the core framework (but maybe suggest it as a 
possible interface).

- [KRTh5] I like the feedback idea.  I had not included the 
feedback messages in the BML specification, but we are in fact 
getting "ENDOFBEHAVIOR ID=x" messages back from the Unreal 
Environment.

- [KRTh6] Regardless of actual terms, we need the following classes 
of participants:
 (1) The one producing the content
 (2) The one or more people the content is directed at (sub-set of (3))
 (3) The one or more people that are ratified receivers of the content
 (4) The zero or more people that are un-ratified receivers of the 
content
Based on Kris's suggestion and on discussions we've had over here 
in the past, I propose the following terms:
 (1) Producer (was "speaker")
 (2) Addressee
 (3) Receiver (was "listener" or "hearer")
 (4) Overhearer
Although I understand the more precise term "content producer", I 
much prefer a single word for the API term, there are just too many 
different ways to represent two words ("ContentProducer", 
"content_producer", "content producer","contentProducer", etc..) 
and it gets a bit too long in a compact message.

- [KRTh8] Yes, "turn" is just a super-category and the various 
things that one can do with a turn ("give", "take", "request", 
"keep") are sub-types that can be described as an attribute.  The 
argument for a super-category like this is that we may not all 
agree on how many sub-types there are - this way we can keep the 
number of attributes (and thus the types of turn management types) 
open.  The "real-time" vs. "pre-planned" use of this message is 
something we can discuss more - the  text in the definition is 
geared towards the "pre-planned" use, but maybe we can make it more 
general.

- [KRTh9] "Performative" here really describes "Content", often in 
the form of a "Speech Act" type and a "Proposition" (i.e. "INFORM" 
and "(is sky blue)").  As for whether it also describes a 
multimodal "Communicative Act", it depends on whether you define 
"Communicative Act" as simply a multimodal version of Speech Act or 
whether it's any communication action, in which case *any* FML 
could be considered a "Communicative Act" (including interaction 
management).

- [KRTh10] Let's get Stacy's and Catherine's take on this, I don't 
use it myself (but I think they do).  The definition includes my 
interpretation of it, they may have a better one.

- [KRTh11] Yes, I like "Performance", though we can't confuse it 
with "Performative" - let's ask what others think.

Cheers,
 -= hannes
============================================================================
Date: Wed, 29 Mar 2006 00:45:57 +0200
From: Stefan Kopp <skopp_(at)_techfak.uni-bielefeld.de>
To: "K. R. Thorisson" <thorisson_(at)_ru.is>, Hannes Vilhjalmsson <hannes_(at)_ISI.EDU>,
Subject: representations paper
...I think we need some discussion about the role and necessity of
the "Gesticon". It is not even clear in the paper what it really is
(it's explained as a behavior lexicon, but also referred to as a
language for describing behaviors in a player-independent way, which is
actually BML supposed to be).  From the comments of Kris and Hannes, I
took that they have some concerns there too.  I also think there is no
agreement yet about how it is currently plugged into the architecture in
figure 1.

Catherine and Brigitte, my suggestions are:
- to state that a behavior lexicon, which may be called "Gesticon", can
of course be part of the architecture, but it should be drawn upon by
the behavior realization component. That is, in Fig. 1 it should be
hooked to the BR box and not the connection between BP and BR. It would
still enable the exchange of behavior definitions as you wanted it.
- to fuse sections 5 and 6. Both address the same problem, namely to
specify overt multimodal behavior, and they are partly redundant. Hannes
and I have made some suggestions in the last long emails as to what can
be part of BML, such that the Gesticon elements would be covered too.  I
think there is a lot of good stuff in Sect. 6 which I would like to see
being described as BML elements.
- to leave out the detailed gesture specification. Besides the fact that
we are probably going to have not enough space in the paper, what is
described there is basically yet another approach in addition to MURML
or other languages, and we didn't really have a discussion about how a
unified gesture spec language could look like (besides sparse email
exchange). Let's have such a discussion and leave it open as a future
topic. If you want, we can mention that there are some markup-languages
out there for describing gesture form, though.

What do think??
...

best,
Stefan
============================================================================
From: Stacy Marsella <marsella_(at)_ISI.EDU>
Subject: Re: representations paper
Date: Wed, 29 Mar 2006 07:38:16 -0800
To: skopp_(at)_techfak.uni-bielefeld.de
...
The lead in sentence threw me a tad. "Human communicative behavior" 
suggests a large scope. Should we limit it? Any perceived action is a 
communication (e.g., Cristiano's BICs). An impatient driver can start 
the car to communicate to a spouse to hurry up so they can leave.. 
Writing a letter or putting up a billboard are also communicative 
behaviors. Perhaps something like face-to-face interaction is more 
the focus here?

The SAIBA diagram and the Gesticon. I agree with whomever made the 
comment that the link from Gesticon to the link between Beh. Planning 
and Beh. Realization seems strange. In our system (SmartBody) we use 
something akin to a Gesticon (we call it a Gestuary) but it is linked 
to both the planning and generation boxes. Also  it is currently a 
repository of BML commands and BML macros, not a distinct 
representation.

The FML section is pretty weak obviously. It reads like what it is - 
we all put our favorite bits (psycho-socio-linguistic theories) in.
The various categories come off as a weird mix. Are we not simply 
saying that an utterance can have a range of socio-emotive goals, 
there are a range of theories that characterize these goals and FML 
is currently being very open in what it includes?

Coping is one way to characterize the socio-affective goal and 
strategy of the utterance - I could clarify that in the text. In   
essence it details (or at least is one way to detail) the 
illocutionary force of the utterance in both personal and social ways.

I don't know what is meant by certainty? Is certainty an annotation 
describing the speaker's confidence in the propositional content, the 
intent to convey that confidence, the imposition on negative face, is 
it the strength of what Searle calls the sincerity conditions, or 
something else?

The Cognitive annottation is also not clear. The text states it is 
the meta-cognitive activities but the example  (problems with recall) 
is strictly speaking not meta-cognitive though of course one could 
have a meta-cognitive strategy for facilitating recall problems - by 
meta-cognitive do people mean performance issues associated with 
cognitive function (cognitive load, recall problems, etc.)
============================================================================
From: "Kristinn R. Thorisson" <thorisson_(at)_ru.is>
Subject: Re: representations paper
Date: Wed, 29 Mar 2006 17:39:00 +0000
To: Stefan Kopp <skopp_(at)_techfak.uni-bielefeld.de>,

On 29.3.2006, at 15:38, Stacy Marsella wrote:
> The lead in sentence threw me a tad. "Human communicative behavior" 
> suggests a large scope. Should we limit it? Any perceived action is 
> a communication (e.g., Cristiano's BICs). An impatient driver can 
> start the car to communicate to a spouse to hurry up so they can 
> leave.. Writing a letter or putting up a billboard are also 
> communicative behaviors. Perhaps something like face-to-face 
> interaction is more the focus here?

I think we are hoping to be more general than simply face-to-face, so 
that the same mechanisms, if not implementations, could be used to 
communicate via starting the car. Of course nobody expects us to 
cover the whole range of human communication from the get-go, but at 
least we are clear about the goals if we say human communicative 
behavior.  In other words, there is no theoretical reason to 
restrict ourselves to f-t-f, but perhaps we could say that we are 
starting a bit more modestly...

> The SAIBA diagram and the Gesticon. I agree with whomever made the 
> comment that the link from Gesticon to the link between Beh. 
> Planning and Beh. Realization seems strange. In our system 
> (SmartBody) we use something akin to a Gesticon (we call it a 
> Gestuary) but it is linked to both the planning and generation 
> boxes. Also  it is currently a repository of BML commands and BML 
> macros, not a distinct representation.

That is a problem. This whole issue of the gesticon needs to be 
cleared up. Whenever we find such problems we can either (a) solve 
them before the paper goes out or (b) leave them out of the paper. I 
suggest Stefan takes the lead on how to handle these things.

> The FML section is pretty weak obviously. It reads like what it is 
> - we all put our favorite bits (psycho-socio-linguistic theories) in.
> The various categories  come off as a weird mix. Are we not simply 
> saying that an utterance can have a range of socio-emotive goals, 
> there are a range of theories that characterize these goals and FML 
> is currently being very open  in what it includes?

I think it's a good idea to pull back a bit on detail in the places 
where there are large divergences... so sure, we can say that it can 
have "a range of socio-emotive goals".

> I don't know what is meant by certainty? Is certainty an annotation 
> describing the speaker's confidence in the propositional content, 
> the intent to convey that confidence, the imposition on negative 
> face, is it the strength of what Searle calls the sincerity 
> conditions, or something else  ?

The way I understood it: assertiveness.

> The Cognitive annottation is also not clear. The text states it is 
> the meta-cognitive activities but the example  (problems with 
> recall) is strictly speaking not meta-cognitive though of course 
> one could have a meta-cognitive strategy for facilitating recall 
> problems - by meta-cognitive do people mean performance issues 
> associated with cognitive function (cognitive load, recall 
> problems, etc.)

Good point. I'm not sure what was meant there.

=K
============================================================================
Date: Thu, 30 Mar 2006 00:52:07 +0200
From: Catherine Pelachaud <c.pelachaud_(at)_iut.univ-paris8.fr>
To: "Kristinn R. Thorisson" <thorisson_(at)_ru.is>
Subject: Re: representations paper

A quick reaction to these last exchanges of emails.
>> The lead in sentence threw me a tad. "Human communicative behavior" 
>> suggests a large scope. Should we limit it? Any perceived action is 
>> a communication (e.g., Cristiano's BICs). An impatient driver can 
>> start the car to communicate to a spouse to hurry up so they can 
>> leave.. Writing a letter or putting up a billboard are also 
>> communicative behaviors. Perhaps something like face-to-face 
>> interaction is more the focus here?
>
> I think we are hoping to be more general than simply face-to-face, so 
> that the same mechanisms, if not implementations, could be used to 
> communicate via starting the car. Of course nobody expects us to 
> cover the whole range of human communication from the get-go, but at 
> least we are clear about the goals if we say human communicative 
> behavior.  In other words, there is no theoretical reason to 
> restrictu ourselves to f-t-f, but perhaps we could say that we are 
> starting a bit more modestly...

I agree with you. Several of us will be at ZIF next week. It has been
discussed at length at a first workshop at ZIF what communicative
gesture is? when does a gesture can be considered communicative? there
are much more gestures that are communicative than the gesture we
normally used for our ECA! so we do need to state that!

>> The SAIBA diagram and the Gesticon. I agree with whomever made the 
>> comment that the link from Gesticon to the link between Beh. 
>> Planning and Beh. Realization seems strange. In our system 
>> (SmartBody) we use something akin to a Gesticon (we call it a 
>> Gestuary) but it is linked to both the planning and generation 
>> boxes. Also  it is currently a repository of BML commands and BML 
>> macros, not a distinct representation.

if necessary, an arrow can be added from planning to gesticon.

> That is a problem. This whole issue of the gesticon needs to be 
> cleared up. Whenever we find such problems we can either (a) solve 
> them before the paper goes out or (b) leave them out of the paper. I 
> suggest Stefan takes the lead on how to handle these things.

It really depends what we aim to achieve. Gesticon (or Gestuary) is
important per se as it defines behaviors independently of graphics model
(or at least independently of most graphics models (eg, it may not be
true for models done in flash animation)). If an aim of the languages we
are working on is to allow modularity, exchange of modules, and the
like; independence from graphics model seems a very important premise.
If so Gesticon has its place.
Using a fix set of token behaviors (smile, head nod,...) does not allow
one to get a large variety of subtle expressions. Personally I am not
keen in this type of representation.

>> I don't know what is meant by certainty? Is certainty an annotation 
>> describing the speaker's confidence in the propositional content, 
>> the intent to convey that confidence, the imposition on negative 
>> face, is it the strength of what Searle calls the sincerity 
>> conditions, or something else  ?
>
> The way I understood it: assertiveness.
>
It looks like several tags of FML are very similar to APML
(meta-cognitive, certainty, contrast...). If it is so, we could use
their definition from APML, that is from Isabella's theory.

best,
Catherine