Welcome to the MediaHub Forum

[Glenn Campbell]

Welcome to the MediaHub Forum, kindly being hosted by Mindmakers.org.

Here you are invited to post comments or queries in relation to our work on developing an intelligent multimedia distributed platform hub (MediaHub). MediaHub will integrate and synchronise language and vision data in such a way that the two modalities are complementary to each other.  The main objectives of MediaHub are to interpret/generate semantic representations of multimodal input/output and perform decision-making over multimodal data. More information on MediaHub is available at:

http://www.infm.ulst.ac.uk/~glenn/phd.html

Please register with a username and password and start participating today with any comments and queries relating to MediaHub or intelligent multimedia in general.

All the best,

Glenn.

[thorisson]

Hey Glenn,
what is the status on MediaHub these days? Are you actively working on it?

=K

[Glenn Campbell]

Hello again Kris,

Thanks for your email. How are things? Yes I am still working on MediaHub. Spent a while focusing on the Bayesian decision-making side of things (using Hugin) but now back focusing on the role of Psyclone.

I am now trying to develop a strategy for accessing dialogue history on MediaHub's whiteboard. As an example, consider the following scenario:

1. U: whose office is this [-->]? 
2. S: That is Paul’s office.
3. U: Ok. Whose office is that [-->]? 
4. S: That’s Sheila’s office.
5. U: Show me the route from her office to this [-->] office.

All turns are marked up in XML which is passed as content in Psyclone messages. I am passing the marked up data as strings, then converting it into XML documents for parsing. I am trying to use the whiteboard to determine which office 'her' refers to. I am using an XML-based domain model to determine whose office the user is pointing to on a 2D office plan. Of course more complicated scenarios will be considered, e.g.:

1. U: whose office is this [-->]? 
2. S: That is Sean’s office.
3. U: Ok. Whose office is that [-->]? 
4. S: That’s Paul’s office.
5. U: Show me the route from her office to this [-->] office.

Here, the user has referred to a female occupant, but the dialogue history only has a male occupant. It is possible that turn 4 should be ‘That’s Paula's office’. The domain model could tell us if a person called Paula exits, and if so, the incorrect speech recognition of Paul could be overruled.

Other scenarios I have worked on include cinema ticket reservation, in-car presentation system, in-car safety system, theatre seat reservation, and turn-taking.

All the best,

Glenn.