<target> and a SAIBA semantics layer (again)

[amarshal]

We've occasionally discussed the concept of a semantic block for BML, and I want to raise the issue again.

At the workshop we decided the locomotion target parameter needed to include a syntax for selecting the frame of reference.  For Core, we decided to limit this to world coordinates (e.g., walking to a specific place in the world) and in a character relative direction (e.g., walk forward).

Further, many behaviors need this notion of a target.  BML Core already includes a target for gaze and gesture.  Existing examples include <cadia:operate target=".."/>.  SmartBody uses it for application specific behaviors.

And we can easily consider these behaviors to act in unison: Look at where you are going or what you are operating.  In the current syntax, each specification requires a new <target> with its own reference id and frame of reference:
<!-- Current specification, roughly -->
<target>
  <frame>WORLD</frame><!-- frame of reference -->
  <vector>1 2 3</vector><!-- translational offset (replace with translation & orientation pair?) -->
</target>


And this structure gets repeated for each behavior if we continue to encapsulate the parameter:
<!-- Look and point -->
<bml><behaviors>
  <point>
    <target>
      <frame>WORLD</frame>
      <vector>1 2 3</vector>
    </target>
  </point>
  <gaze>
    <target>
      <frame>WORLD</frame>
      <vector>1 2 3</vector>
    </target>
    </target>
  </gaze>
</behaviors></bml>

I fear this will get worse if the target notation is extended further, which I see as likely.

I believe we're seeing duplication because the details of the target are not part of the behavior.  We are describing a single entity, and we should use a reference notation to a single XML tag to reflect that.  I propose (repropose?) a semantic block:
<!-- Look and point -->
<bml>
  <semantics>
    <location id="there">
      <frame>WORLD</frame>
      <vector>1 2 3</vector>
    </location>
  </semantics>
  <behaviors>
    <point>
      <target ref="there" />
    </point>
    <gaze>
      <target ref="there" />
    </gaze>
  </behaviors>
</bml>


Now we can distinguish the behavioral components of the target from the semantic components:
<!-- Walk to ActorBob, but don't chase him -->
<bml>
  <semantics>
    <!-- We can interact with objects defined in the world, including other characters -->
    <object id="bob">
      <!-- these objects persist between performance requests, and need a shared identifier -->
      <external-id>ActorBob<external-id/>
    </object>
    <!-- Alternatively, we may want to consider something like the FML participant tag,
       -  to distinguish SAIBA/BML aware objects (a.k.a., characters, actors).
       -->
  </semantics>
  <behaviors>
    <gaze>
      <target ref="bob" />
    </gaze>
    <locomotion>
      <target ref="bob">
        <dynamic>false</dynamic>
        <proximity>1m</proximity>
      </target>
    </locomotion>
  </behaviors>
</bml>


Note, while the dynamic and proximity parameters were originally defined for the locomotion behavior, I have grouped them with the target.  After all, these parameters only apply when using locomotion with a target.  Further, they have meaning when carried over to other target behaviors:
<!-- Look and point at Bob's current location, within a meter (even if he moves) -->
<bml>
  <semantics>
    <object id="bob">
      <external-id>ActorBob"<external-id/>
    </object>
  </semantics>
  <behaviors>
    <gaze>
      <target ref="bob">
        <dynamic>false</dynamic>
        <proximity>1m</proximity>
      </target>
    </gaze>
    <point>
      <target ref="bob">
        <dynamic>false</dynamic>
        <proximity>1m</proximity>
      </target>
    </point>
  </behaviors>
</bml>

Almost.  The term proximity may not make sense for angular behaviors like point and gaze, but the concept of accuracy does make sense.  I believe this is just a matter of finding the right word for the shared terminology.


Finally, the notions of objects and their reference is a general issue across all of SAIBA.  In fact, FML already has notions of participants and topics.  Perception, when considered, will also need to persist references between samples.  Without it, the transformation of concepts between layers/modules is ad hoc.  A transformation module, like an FML->BML behavior planner, should be capable of copying select identifier strings between layers without understanding the values (domain and system independent black box) but also without risk of loosing information.

If we define these foundational concepts simply and clearly, in a manner usable at all levels, we will reduce our overall effort and unify the SAIBA framework.

[herwin]

If I understand correctly, you want to put object specific information (world location, object type) in <semantics> and behavior information (proximity, precision, dynamic/not dynamic) in the target?

I'm not extremely enthusiastic about the semantic block. Maybe it is because it feels like you're stuffing world-object knowledge into BML now. I find the world position example not very convincing, do you have any other extensions of target in mind? I would prefer a design in which a realizer and planner share a world that they can query for object information based on object id, rather than directly providing object information in the BML itself. The latter might cause quite a bit of extra verbosity over different BML blocks too (you'd need to redefine bob in each new BML block that requires him). This is definitely an issue that requires a broader SAIBA discussion though.

I do like putting proximity and dynamic in the <target>.

[amarshal]

If I understand correctly, you want to put object specific information (world location, object type) in <semantics> and behavior information (proximity, precision, dynamic/not dynamic) in the target?

Perhaps <semantics> is overly broad.  I am trying to localize named locations so they can be effectively reused between behaviors of the same request.  I do believe that reuse benefits from separation from the behavior information.  I don't don't believe BML processors need to know object type information, and in my examples I intentionally referred to other actors (e.g. "bob") as objects.  Other aspects of the SAIBA pipeline, like the FML processor may need object type info (e.g., is a communicative intent about another person).  It is important that a BML processor understand some concepts of space and spatial relations.  Conversely, the upstream behavior planner should not need to understand as much about spatial relations, but may still need to share some set of object references external of any particular behavior request.

I'm not extremely enthusiastic about the semantic block. Maybe it is because it feels like you're stuffing world-object knowledge into BML now. I find the world position example not very convincing, do you have any other extensions of target in mind? I would prefer a design in which a realizer and planner share a world that they can query for object information based on object id, rather than directly providing object information in the BML itself. The latter might cause quite a bit of extra verbosity over different BML blocks too (you'd need to redefine bob in each new BML block that requires him). This is definitely an issue that requires a broader SAIBA discussion though.

I agree that much information must come from an shared ontology and knowledge base.  Nonetheless, such a query process requires another protocol if we are aiming for a truly modular SAIBA pipeline of replacable parts.  No doubt, we will need to address that at some point, but not before we finalize BML 1.0.  This proposed modification just attempts to address a thin slice of that functionality, with an emphasis on spatial relationships, so we can make useful applications.

Even so, I expect we will often need to refer to unnamed locations, where a string identifier is not enough:

<!-- To the left of Bob (example walk target)-->
<semantics>
  <object id="bob" />

  <location id="there">
    <frame ref="bob" />
    <vector>1 0 0</vector>
  </location>
</semantics>

I've also thought about defining locations in space for gestures relative to the characters own body or body parts:

<!-- Gesture space reference locations (e.g., for comparison gestures) -->
<semantics>
  <location id="left">
    <frame ref="self:chest" />
    <vector>0.2 -0.1 0.5</vector>
  </location>

  <location id="right">
    <frame ref="self:chest" />
    <vector>-0.2 -0.1 0.5</vector>
  </location>
</semantics>

This generalizes a concept we see in the <gesture> behavior in a way that allows it to be used by other behaviors.  For instance, it allows gazing at the space, even if empty, as a gestural reference.  However, it is also more precise than may be necessary, and looses the relaxed definitions in the current <gesture>.