Dialogue History

[Glenn Campbell]

I am currently concerned with using dialogue history in my decision making. Has anyone any advice on how I can access and use past messages from a whiteboard in Psyclone? For example, in a dialogue, can I find out what was the last object, person, place referred to. This would allow me to resolve questions such as "How do I get from his office to Paul's office?" . Can any one advice on how I could use previous messages (dialogue history) to resolve similiar issues in my project. Any comments or advice welcome.

All the best,

Glenn.

[thorisson]

You can use so-called retrieves to get old messages off a Whiteboard. The manual gives this example:

  <trigger from="WB1" type="Input.Sens.MultiM.Vision.Human.Found.True"/>
  <retrieve from="WBX" type="Input.Sens.UniM.Hear.Human.Voice"/>
  <retrieve from="WBX" type="Input.Sens.MultiM.Vision.Human">
    <latest>3</latest>
    <lastmsec>8000</lastmsec>
  </retrieve>

This is part of a module spec in the psySpec. It will result in both *.Human.Voice and *Vision.Human being returned whenever *Human.Found.True is posted.

In your case it sounds like the simplest thing to do would be to do an active retrieve whenever you want to (because you may not need the last sentence all the time). A module can at any time decide to retrieve old information. To do this from inside the module, randomly at runtime, you can use any one of these (Java):

ObjectCollection retrieveMessages(String xml)
ObjectCollection retrieveMessages(RetrieveSpec spec)
ObjectCollection retrieveMessages(ObjectCollection specs)

In the second one you could for example stick the XML code above. Hope this helps!

=K

p.s. I recommend browsing the Manual for some more good ideas related to this.

[Glenn Campbell]

Thanks for your advice Kris. I have a more few queries relating to the retrieval of past messages from the whiteboard. To put things in context, I am trying to retrieved from a whiteboard called ‘MediaHub_Whitboard’. I am trying to retrieve the last 5 messages of type *input* that have been posted to the MediaHub_Whiteboard. I am getting a Null Pointer Error when I try to call the retrieveMessages() method of the Messenger class.

Here is a snippet of my code:

   public Object firstObject;
   private ObjectCollection retrievedMessages;
   public Messenger mgr1;
            …
            …
         
        String strXML = "<retrieves><retrieve from=\"MediaHub_Whiteboard\" type=\"*input.*\"> <latest>5</latest> <lastmsec>80000</lastmsec> </retrieve> </retrieves>";
        String from = "MediaHub_Whiteboard";

Messenger mgr1 = new Messenger(“mgr1”, MediaHubGUI, “*input*”, “Psyclone.System.Ready”);

        // retrieveMessages method is contained in the Messenger class
        retrievedMessages = mgr1.retrieveMessages(from, strXML);              //Null Pointer Error Here

        firstObject = retrievedMessages.getFirst();
        String strObject = firstObject.toString();
            …
            …

Here area few queries I have relating to the parameters of Messenger() and retrieveMessages():

1. The constructor for the Messanger object is:

Messenger(String name, MessageSender sender, String wakeupMessageType, String phaseChangeType)

I could be passing the wrong parameter to Messenger(). For example:

Is sender the sender of the original messages (i.e. DialogueManager)?
or is it the sender of the request to receive messages (i.e. MediaHubGUI)?
What does the string name relate to?


2. It could also be that I am passing the wrong parameters to retrieveMessages method. The constructor for the method is:

retrieveMessages(String from, String retrieveSpecXML)

I am not sure what from refers to. For example:

Is it the name of the module doing the retrieve (i.e. MediaHubGUI)?
Is it the module that posted the messages that are to be retrieved (i.e. DialogueManager)?
Is it the name of the Whiteboard where the messages will be retrieved from (i.e. MediaHub_Whiteboard)?

 
3. It could also be that the xml string being passed is incorrect. I am creating the xml string using:

String strXML = "<retrieves><retrieve from=\"MediaHub_Whiteboard\" type=\"*input.*\"> <latest>5</latest> <lastmsec>80000</lastmsec> </retrieve> </retrieves>";

Is this syntax ok?

If anyone can clarify any of these issues, I would greatly appreciate it.

All the best,

Glenn.

[thorlist]

Sorry that we didn't reply to this sooner - have both been travelling. Thanks for giving us a kick :-)

First of all, you should never use the Messenger object directly, this is for internal use in the JavaAIRPlug object only.

You should always use the JavaAIRPlug object as this will ensure that everything is initialised properly before you start using it.

You do that like this:

    private JavaAIRPlug plug;

    plug = new JavaAIRPlug(name, host, port);
    if (!plug.init()) {
        System.out.println("Could not connect to the Server on " + host +
                " on port " + port + "...");
        System.exit(0);
    }

Now the plug is registered with Psyclone, but you have not yet asked for any messages (triggers or retrieves).

If you merely want to retrieve data yourself from Whiteboards and never be triggered by postings from other modules, you can use the following API:

    ObjectCollection coll = plug.retrieveMessages(String retrieveSpecXML);

with your xml ("<retrieves><retrieve from...") as the argument. You can call this as often as you like, and you will never be pushed any messages. I would strongly recommend that you always have this main loop in your code:

    Message message;
    while (true) {
      if ( (message = plug.waitForNewMessage(100)) != null) {
        // ignore it
      }
    }

or at least regularly call the waitForNewMessage(ms) function as it makes sure that the channel to Psyclone is managed properly.

You can, however, also do as Kris suggests, and use a trigger to be pushed the information when something happens. Say, for example, that you wish to do your retrieve every time message x.y.z is posted on Whiteboard WB1. You can then register your plug module with the following spec:

<module name="mymodule" type="external">
  <trigger from="WB1" type="x.y.z"/>
  <retrieve from="WBX" type="Input.Sens.MultiM.Vision.Human">
    <latest>3</latest>
    <lastmsec>8000</lastmsec>
  </retrieve>
</module>

Now, in your loop do this:

    Message message;
    while (true) {
      if ( (message = plug.waitForNewMessage(100)) != null) {
        // got the trigger for x.y.z
        // Get the current number of retrieved messages
        n = plug.getRetrievedMessageCount();
        // get all retrieved messages
        ObjectCollection coll = getAllRetrievedMessages();
        // or get only the second retrieved message
        Message rmsg = getRetrievedMessage(2);
      }
    }

You can choose to either enter your module registration XML in the PsySpec XML file or as a parameter to the init() function just after creating the plug:

    if (!plug.init(xml)) {  // xml = "<module name...>"
    }

By the way, which version of Psyclone are you using? If you are using anything older than version 1.0.6, please download the latest version from

    http://www.mindmakers.org/projects/Psyclone

This contains the manual and tutorials, which you can also see at

    http://www.cmlabs.com/psyclone/manual

This contains lots of examples of how to do many different things. See especially

    Part 4: USE CASES

I hope that this helps. We have made things much better and simpler in the past year, so I hope that you don't get too confused if you are used to a much older version of Psyclone.

Please keep posting problems - we need to hear these as much as you need the help.

[Glenn Campbell]

Thor,

Thanks for getting back. I am using version 1.0.6 of Psyclone.
 
 I have already got the following in my code:
 
 private JavaAIRPlug plug;
 
 plug = new JavaAIRPlug(name, host, port);
 if (!plug.init()) {
 System.out.println("Could not connect to the Server on " + host +
 " on port " + port + "...");
 System.exit(0);
 }
 
 and this works fine. I have also registered for messages in the PsySpec and
 can receive them automatically using:
 
 message = plug.waitForNewMessage(100)
 
 I just need to be able to request past messages as and when they are needed
 so what you have suggested is just what I need. I was wrong to try to
 access the Messenger object directly - thanks for pointing this out.
 
 You have given me plenty to work on, so I will try what you suggested and
 let you know how it goes.
 
 Best wishes,
 
 Glenn.

[Glenn Campbell]

Hello again,

I have a class called History. I want to access past messages of type input.language.confidece as and when I need to. Am I right in saying that

retrieveMessages(strXML);

will return an ObjectCollection containing messages automatically when it is called? That is, do I not need to use a waitForNewMessages while loop?

I am calling this method but when I try to count the number of objects in the collection I get a null pointer exception error. Here is the code:

public History() {
       
                plugHistory = new JavaAIRPlug("History", host, port);
        if (!plugHistory.init()) {
            System.out.println("Could not connect to the Server on " + host + " on port " + port + "...");
            System.exit(0);
        }
       
        System.out.println("Connected to the Server on " + host +
                " on port " + port + "...");
    }

   
    public static void main(String args[]){

        new History();
       
       
       strXML = "<retrieves> <retrieve from=\"MediaHub_Whiteboard\" type=\"input.language.confidence\"> <latest>3</latest> <lastmsec>80000</lastmsec> </retrieve> </retrieves>";
       
       
        ObjectCollection coll = new ObjectCollection();

       
        coll = plugHistory.retrieveMessages(strXML);
       
    int count = 0;   


       count = coll.getCount();
 
         
       System.out.println("The number of messages retrieved is : " + count);

    }


  Of course I will want to do more than just count the number of retrieved messages - I am just doing this as an initial step to check if my retrieve is working ok.

 Perhaps my call to the method is returning no messages due to my xml string being incorrect. Does the following syntax look ok?

  strXML = "<retrieves> <retrieve from=\"MediaHub_Whiteboard\" type=\"input.language.confidence\"> <latest>3</latest> <lastmsec>80000</lastmsec> </retrieve> </retrieves>";

All the best,

Glenn.

[thorlist]

Hi Glenn,

It appears to be correct, but could you send me your History class file by email so I can test it on my end?

I will then get back to you and the forum as soon as I can...

Best,

Thor

[Glenn Campbell]

Thor,

That would be great. I have forwarded you the file.

Thanks,

Glenn.

[thorlist]

Hi Glenn,

OK, I have found a bug in Psyclone which we will need to fix, and I have found a way for you to get around this bug until the next release.

The bug is that the <retrieves><retrieve></retrieve></retrieves> mechanism has a bug, which causes Psyclone to crash. This we will fix as soon as we can.

In the meantime you can use <retrieve></retrieve>, ie just a single retrieve statement which in your case is equivalent. Strictly speaking, you only need to use <retrieves><retrieve></retrieve></retrieves> when you wish to put multiple retrieves into one request (<retrieves><retrieve></retrieve><retrieve></retrieve></retrieves>)

So you should use:

       strXML = "<retrieve from=\"MediaHub_Whiteboard\" type=\"input.language.confidence\"> <latest>3</latest> <lastmsec>80000</lastmsec> </retrieve>";


Secondly, in the Java AIR implementation it is perfectly valid to return a NULL pointer from the retrieveMessages(strXML) call. This merely indicates that no messages could be retrieved because none matched the criteria. So you should check for (coll == null) and if this is false for (coll.getCount() == 0) to be sure.

In general, when you request an object it is always healthy to check for NULL anyway - you avoid problems later on and can detect it without crashing, which makes the code more stable.


Thank you very much for helping us find this bug. I hope the temporary fix works for you...

Best,

Thor


 

[Glenn Campbell]

Thor,

Thanks for that. I have tried what you suggested and it is now working fine. I was checking for (coll == null) in a previous version of the code but removed it for some reason during testing. Thanks for pointing this out, I have included the code again. Now that this issue is resolved, I can work on developing a decision-making mechanism that will use present and past messages to resolve uncertainty in multimodal input/output.

Best wishes,

Glenn.

[thorlist]

I am happy that it now works for you.

Your project sounds very interesting indeed. Would you consider putting it up as a Mindmakers project (on http://mindmakers.org/) to share it with fellow OpenAIR users? Don't worry about the code not looking perfect or things not working exactly as it should - the projects on there are usually ongoing research and by sharing we give the community better resources to work with. And you might also get feedback, suggestions and fixes back...

Thor

[Glenn Campbell]

Thor,

That sounds like a good idea. I have created an account at mindmakers.org and will now create a project page where I can share more information on my work.

Thanks again for your help,

Glenn.