Audio-visual support system for Mobile E-meeting Participant

Summary

Web-based collaboration using the wireless devices that have multimedia playback capabilities is a viable alternative to traditional face-to-face meetings. E-meetings are popular in businesses because of their cost savings. To provide quick and effective engagement to the meeting activity, the remote user should be able to perceive whole events in the meeting room and have the same possibilities like participants inside. The technological framework of the developing intelligent meeting room implements multichannel audio-visual system for participant activity detection and automatically composes actual multimedia content for remote mobile user. The developed web-based application for remote user interaction with equipment of the intelligent meeting room and organization of E-meetings will tested with Nokia mobile phones.

Project Description

The novelty of the proposed web-based application for hybrid meeting consists in automatic selection of web-camera of active speaker by multichannel speech activity algorithm and designing a user interface adaptive to mobile device features used by remote participant. A technological framework of intelligent meeting room and a model of media stream fusion used for designing actual content of the web-based application are considered.

The developed intelligent meeting room is equipped by projector, touchscreen TV for smart desk application, several cameras for video monitoring of audience and presentation area, personal web-cameras for analysis of behavior of participants sitting at the conference table, as well as three microphone array for sound processing. A combination of the desktop web-cameras and microphone arrays provides both spatial localization of sound sources and record of participants’ speech with high quality. Multimedia content compiled from audio and video signals captured by the referred devices are used in the web-based application for organization of hybrid E-meetings.

Graphical interface of the web-page, on which remote participants could observe a meeting organized inside the intelligent meeting room, contains several basic forms connected with corresponding source, which transmits graphical data (the projector – a presentation slide; the smart desk – window with handwriting sketches; the video and web-cameras – frames with an image; the software services – time indicators, logo and other data about meeting).

Parameters of objects (equipment, software, participants), which are located inside the intelligent meeting room are analyzed to change the graphical content of the web-page. Values of parameters of hardware and software are determined by a query of its states via TCP/IP protocol or by means of OLE automation. Behavior of participants sitting at the conference table, as well as the main speaker located in the presentation area is analyzed by developed technologies of sound source localization, video tracking of moving objects, seeking and tracking of human face. The logical-temporal model of compilation of web-page graphical interface will be tested on a personal computer and several models of Nokia smartphones.

Timing and Deliverables

June 2010 – survey of modern systems of E-meeting support.

October 2010 –web-platform for audio-visual stream processing.

November 2010 – paper for the 8th FRUCT workshop.

June 2011 – prototype of audio-visual support system for mobile E-meeting participant.

Project Team

Contact person: Victor Budkov, programmer of SPIIRAS, M.Sc. Student, project leader

Developers:

Sergey Glazkov, PhD student of SPIIRAS

Maria Prischepa, programmer of SPIIRAS, M.Sc. Student

Alexander Ronzhin, programmer of SPIIRAS, M.Sc. Student