Table of Contents
Table of Figures
The following report is an assessment of the usability of the video conferencing and tele-education tools developed under the MECCANO project. While the deliverable is concerned with the overall usability of the MECCANO tools it is particularly interested in addressing two of issues defined in the original project outline. In particular the report has focused on:
The report is divided into two parts;
Part one is an overview of the usability issues encountered during usage of the MECCANO tools in real world situations. These situations involved the use of the tools in three different scenarios;
These scenarios represent the range of use, which has been carried out and supported by the various MECCANO partners. The section briefly describes the various sessions in which the tools have been used, and identifies the primary usability issues, which were identified during these sessions. The information used in this assessment has been gathered both within the MECCANO project and through other projects, which have used the tools. In particular it has used information gathered from the various support and validation projects, which were supported in workpackage 9 of the MECCANO project. In addition it has used information gathered directly from the PIPVIC projects, which assessed the use of IP videoconferencing for tele-education in the UK using the core MECCANO tools [pipvic].
Based on the conclusions derived from part one of the report part two provides an overview of the various tools developed and used within the project, and gives a general assessment of their usability. The usability analysis in this section focuses on general functionality specific to each tool, such as ease of use, quality of sound and video channels, interaction with other tools, suitability for task etc. In particular this section looks at the different tools developed and refined within the MECCANO project which address the usability issues raised in part one.
The range of applications and use, which the MECCANO partners supported, has involved real world usage of the tools for conferencing. This has provided valuable information for the usability assessment of the tools. The drawback of this approach is that real world usage patterns tend to be more difficult to obtain accurate user feedback from. This is largely due to the requirements of obtaining user feedback in a structured manner which often tends to take second place to the requirements of supporting and establishing the conference in the first place. It is also difficult to gather data in a formal manner, since asking participants to fill out qualitative appraisals during a videoconferencing session is time-consuming and can lead to interference with the real purpose of the videoconference.
Within the PIPVIC projects for instance participants were encouraged to add comments below their quality ratings. However few of them did this more than once a lesson, and as the course progressed, the comments became fewer and further between [sasse 25]. As well the nature of the comments made regarding the tools and sessions can often not be meaningful enough to be helpful to assessors and designers: comments such as 'not very clear today' are really not sufficient to identify the problem [sasse 20].
However it is felt that gathering information from non experienced users in "realworld" usage situations is able to provide a more accurate and realistic assessment of the usability of the tools then that gathered from controlled, laboratory assessment procedures. In short it is felt that sufficient information has been drawn from users responses within the various validation and support projects to enable a general assessment of the tools and to identify key issues, which need to be addressed in future developments of the tools and multicasting technologies. A positive development in the difficulty of gathering user opinions in multicast sessions has been addressed relatively successfully through tools developed by the CRC. These tools are discussed in Section 3
The following section provides an overview of the partners and projects which have provided user feedback on tool usability issues for this report through the various validation projects they have supported. While there has been a range of input to this report from the various partners in the MECCANO project the report has particularly relied on information gathered from support and validation work carried out by these partners. For a more detailed description of the various projects and activities supported by the MECCANO partners, see the MECCANO Deliverables available from the project web site [mecdel]:
R8.1 and R8.2 “Description of and Conclusions from the Seminars”
R9.1 and R9.2, “The support provided to validation projects”
UCL implemented and supported the tools in a number of projects which it has supported. This report focuses in particular on their support of the tools in the UKERNA [ukerna] projects PIPVIC1 and PIPVIC2 [pipvic] which involved an assessment of the issues involved in implementing IP multicast videoconferencing service for educational purposes [bennett].
UIO has been actively monitoring and assessing the use of the MECCANO tools for the delivery of online Seminars and has provided feedback on the project seminars with assessments of the effectiveness of the tools in the deliverables R8.1 and R8.2 [mecdel].
ACC has provided support for the usage of the tools for video conferencing application to Warsaw University, John Paul II hospital as well as scientists from the Faculty of Physics and Nuclear Techniques, at the University of Mining and Metallurgy , Krakow. Additional support for MECCANO tools users from Poland has been provided at the polish MECCANO tools web site [mecpol].
TELES has been active in providing support to the Contraband project. The contraband project, carried out in Germany by TELES, dealt with the handling of routine work and emergency situations in the harbour environment [bennett].
CRC has provided support to the Mbone Knowledge Networking Project which studied Mbone tools in order to evaluate the degree to which they could enhance the process of coordinating networked Knowledge productions. In particular they were interested in this within the in the context of professional research and graduate curricula [bennett].
UM and UF have provided support for the virtual University Project VIROR [viror] which is described in the deliverables D9.1 and D9.2 [mecdel]. The project broadcast regular lectures and seminars as tele-teaching events. These broadcasts were carried out in support of real users through four weekly tele-teaching events. These events were broadcast over the Mbone, However a dedicated ATM backbone is used between the project partners to guarantee high quality and to avoid any network related unreliability. The activities undertaken within VIROR were also observed and evaluated by psychologists in order to provide feedback to tool developers and lecturers.
At Hewlett Packard the tools were tested and evaluated for their general usability in multicast seminars and for collaborative conferencing to assess the requirements of establishing a networked collaborative system within a corporate setting. The feedback of users who did not have an IT background was used as a base for the evaluations and conclusions in this section on the usage of the tools.
This following section describes the seminar activities held within the MECCANO project and the key usability issues which arose from these seminars. It does not undertake an in depth description of the seminars themselves as this is covered in detail in deliverables R8.1 and R8.2 [mecdel].
The seminars carried out at Hewlett Packard were primarily concerned with investigating the use of the Mbone for the transmission of a series of seminars and workshops, which it sponsors through Brims (Basic Research Institute in Mathematical Sciences) and the IRI (the Internet Research Institute). Hewlett Packard Laboratories relies on seminars, workshops, and presentations for a large part of its communication of research programs and issues being undertaken within their research laboratory. Such seminars, and presentations of project work and research findings are a regular occurrence and make up a large part of HP research archives. Recently HP has been investigating methods of making these seminars available for wider dissemination through both the public Internet and its internal Intranet, both as recorded material and for live delivery. In this context the Mbone tools have been evaluated to determine the suitability for delivering seminars and for project meetings.
The seminars were multicast using RAT, and VIC. The machine was an HP Vectra running windows NT4.0. The Video Capture card was a Hauppauge Win TV card. And the audio card was a standard 16 bit Creative labs card. In setting up the multicast system at Hewlett Packard, it was decided to physically locate the machine in a different room from the actual theatre in which the seminars took place. The delivery of multicast seminars is carried out through a transmission connection between the central seminar room and the multicast enabled machine connected to the Mbone. A video and audio signal is sent from the seminar room to a Desktop PC with a class C connection to the Internet. This machine is required for broadcasting to the Mbone as it maintains a direct connection to the external Internet, bypassing the need to filter traffic through the corporate firewall.
In addition to the seminars carried out at Hewlett Packard, Wolfgang Hurst gave a seminar which was organized by Jochen Lienhard from the University of Freiburg. The seminar was designed to introduce and test the Authoring on the Fly toolkit. Only five partners were able to participate in the seminar due to connectivity problems experienced by many partners. Sound quality was good but the video was not received by all of the participating sites. AOFrec itself worked correctly. This seminar is described in the deliverable R8.1 [mecdel].
The Institute of Social Studies, Warsaw University, transmitted a series of ten lectures under the title “Social Change: Adaptation and Resistance”. The main audience for the seminars was at the School of Slavonic and East European Studies (SSEES) – part of UCL in London and a MECCANO partner, but the lectures were also freely available to other participants. ACC helped Warsaw University (UW) in organising transmission at their site, and provided them with technical support throughout the project and particularly during the lecture transmissions. As a validation site, UW was responsible for providing seminar content and operating the MECCANO tools during the lectures.
Figure 1: UW lectures – Prof. Jan Kieniewicz lecturing
Figure 2: UW lectures - Prof. Lukasz Turski lecturing
Lectures took place every Wednesday from 12.30 to 14.00 MET. We used both multicast connectivity and a unicast reflector to provide the best quality for the seminar, not dependent from the current Mbone network status. We used both RAT and VIC tools for transmitting audio and video between the sites. Additionally NTE was used for behind the scenes technicians discussions, which did not disturb the lecture, but where also very important.
All the lectures were announced in the SDR tool under the common title ‘UW Lectures’. The detailed schedule for the seminars is shown in the table below.
12 January 2000
Country on the Eastern Borderland of Europe: introduction to the history of Poland
19 January 2000
The contemporary political map of Poland: Historical continuities
26 January 2000
From revolution to stagnation: Polish science missed chance (1989-1999)
2 February 2000
Personality and adaptation to social change: the case of unemployment
9 February 2000
Bogdan Cichomski & Pawel Morawski
Monitoring social change with survey data: Polish General Social Survey 1992-1999.
16 February 2000
The population dimension of the transition from state to market economy.
23 February 2000
Polish agriculture and countryside: Problems in adapting to the market economy and EU.
1 March 2000
The changes of female identities in Poland.
8 March 2000
Women in politics - attitudes and behaviour
15 March 2000
Group-soul stereotypes and social prejudices in a changing political context
More information about the seminar series can be found at ACC MECCANO site [mecpol] under ‘UW Lectures’.
Issues of usability arising from these activities are considered below.
One of the primary concerns in assessing the usability of the tools in Hewlett Packard was the ease of installation and configuration. While Hewlett Packard has the IT resources to support IP multicasting it is not a core function of its computer support strategy. For this reason it was deemed important to establish a system capable of installing and transmitting Seminars in as non technically demanding a manner as possible. In this respect the MECCANO tools proved to be well suited to the task.
An individual with no previous experience with multicast conferencing undertook the installation of the tools. The installation of the tools was undertaken in conjunction with the Audio Video technician for Hewlett Packard who also had no knowledge of video conferencing issues or multicast technologies. The system was designed to be integrated as much as possible with the existing procedure and system used for recording HP seminars and presentations to video tape. The initial installation of the tools proceeded with no initial difficulty, and the tools operated effectively from the initial installation. The installation procedure demonstrated that, providing a network connection existed, the tools could be installed and operable by individuals with little or no experience of networked conferencing.
Within Hewlett Packard, the initial issue encountered was the limitations and requirements of setting up a Multicast session, which would meet the requirements of the security procedures of Hewlett Packard. In order to satisfy these requirements the multicasting machine used for seminar broadcast needed to be disconnected from the rest of the HP network in order to maintain security. While setting up the machine in this manner made it suitable for multi-casting seminars and for partaking in multicast activity outside the HP network, it subsequently made it unsuitable for HP internal collaborative work, educational applications, or any scenario in which a user may need to access resources on the internal network. While this was not an overall hindrance to the quality of the seminar transmissions, it does limit the situations in which such seminars can be transmitted and received within HP.
This difficulty with Firewall access highlighted one of the primary issues affecting the deployment of Multicast technologies within corporations such as Hewlett Packard and increasingly within University institutes. This issue was addressed within the MECCANO project and the results made available in the MECCANO report “Approaches to Multicast over Firewalls: an Analysis” [oria] which examined and addressed the issue of firewall security and multicast technologies.
An additional security issue is the ability to deliver secure multicast seminars. Within Hewlett Packard a number of the seminars and presentations which occur are of a sensitive nature and it is essential to ensure that only authorized personal, either within or outside the HP network, are allowed to access the material. The most common concern expressed when setting up the multicast seminars was on the security features of the seminars. Therefore, for seminars which are multicast over the Mbone the facility within the MECCANO tools for establishing encrypted sessions is an important function of the tools. In particular the development of the Secure Conferencing Store has proven to be useful in its ability to establish and control conferences and seminars in a secure and straightforward manner. However, although secure sessions are achievable, lack of awareness and distrust of the multicasting seminars over the Mbone has resulted in a reluctance to fully trust Multicasting of seminars. This may be largely a matter of education and awareness that the growth of, and familiarity with, Multicast technologies will likely alleviate.
Typically seminars involved more than just the transmission of audio and video, often including the use of slides, or, occasionally, the demonstration of applications. In this situation the use of a video channel to capture and transmit such material is typically inadequate. This is due to the low resolution of video which is usually unable to capture small textual details on the slides with sufficient detail. A solution to this is to broadcast the slides and the material on a different channel from the video. The use of a whiteboard for this purpose seems obvious; however, placing slides from a presentation into a whiteboard application is not easily achievable in a typical live seminar broadcast.
Most presenters typically use either Powerpoint presentations or overhead transparencies. Currently methods for incorporating PowerPoint slides or overhead transparencies into seminar presentations are generally unsatisfactory. A method is required to enable the integration of these materials into the seminar broadcast. Currently the use of slides in WBD uses postscript format. While this is common in a Unix environment, it is increasingly being superseded on the windows platforms by the use of PowerPoint slides and PDF as a document format.
A few lecturers in the ACC seminar multicasts who had some, or even appreciable, experience in using videoconferencing for teaching, particularly adapted their lesson to the rather modest technical capabilities of the tools and just spoke. The majority of them, however, desired to present their lecture in a typical manner – with transparencies, maps and figures. Unfortunately, these aids were not adequately visible at the remote site . On the other hand, these aids made perception easier for local students. The organisers of the course chose a friendly solution – graphics were sent to London via e-mail before the lecture.
The quality of the seminars was variable. Indications are that this was due to the poor connectivity within the European Mbone Network, which was often sporadic. This resulted in variable quality of the reception by viewing members. This was a common problem throughout the MECCANO project and has been documented in other deliverables as well. In conditions in which there was little packet loss, the quality of the sound and audio was of an acceptable standard allowing for easy understanding of the speaker. We have addressed the consistent difficulty with network performance by such methods as the redundancy features within RAT and the new layered codec in Vic. However conditions at times were so poor, that even the redundancy coding in RAT was unable to cope [bringsrud].
In addition to transmitting live seminars, there is a need for methods of recording and later multicasting seminars. Brims and IRI, like many research and academic institutions have a large archive of seminars and presentations that could be made available to the general public and the research community. Suitable methods are needed for recording and, particularly, archiving and controlling the play back of these seminars. To this end, ACC has developed the Scalable Video Distribution Architecture (SVDA) within the project. In addition, the AOF toolkit developed by the University of Freiburg and MMCR offer solutions to this issue. These solutions are discussed in more detail in part two.
Despite the difficulties with network reliability and the limitations raised by firewall issues, the use of the current MECCANO toolset for transmitting live seminars has been fairly successful. The tools are easy to install and configure for transmission, requiring no specialized equipment or knowledge of the Internet and the Multicast backbone. The tools work with the most basic of video capture cards. The primary limitations to the establishment of the seminar multicast facility within HP arose from the difficulties of negotiating the corporate firewall.
The necessity for providing networked desktop video conferencing facilities is well understood; however, there is little current usage of desktop conferencing tools within corporations like HP. The use of the MECCANO tools for desktop conferencing by the partners has enabled us to identify the current limitations and requirements for implementing desktop conferencing. Videoconferencing of this nature has been the most common use of the tools.
Technical and management meetings, held regularly by MECCANO partners, played an important role in checking the international Mbone connectivity status, verifying network parameters required for video conferencing purposes and, most importantly, for checking whether the tools are working properly and are easy to use and efficient. This provided the MECCANO project with the opinions of people directly involved in the development of those tools. Usability information in this section has been gathered through feedback from the wide range of activities supported within the MECCANO project. These activities have been described in greater detail in deliverables R8.1 and R8.2 [mecdel].
One of the primary difficulties with desktop video conferencing is lack of proper eye contact mechanisms and localization. This in turn makes it difficult to recognize cues about who is talking or who is likely to talk. The bandwidth restrictions currently impose a frame rate of 6.5 frames a second on conferencing, which is not sufficient to allow for natural interactions between participants in the session. One of the difficulties with group conferencing arises from this limitation; it can produce overlapping attempts at discourse or, conversely, long silences in which the participants are waiting for someone to take the floor.
Having a highly formal conference, in which one participant acts as chair designating who will speak at each turn, can alleviate this. However. this is less than ideal and hinders the spontaneous exchange of information, which is often crucial to successful meetings.
Improvements in compression schemes and network bandwidth will help to address this matter. Already, within the short period of this report, there has been noticeable improvements in the quality of the videoconferencing interactions within the MECCANO meetings, despite the highly variable performance of the Mbone network.
Indications are that video conferencing is unlikely to be able to replace the flexibility of physical co-presence. Collaborative systems therefore need to provide mechanisms to compensate for the lack of multimodal communications. There are a number of criteria for insuring adequate control over user interaction within a multiuser environment:
To limit inappropriate usage of resources that may be constrained by bandwidth. This said there would still be some difficulties arising from the simple logistics of coordinating multiple users and multiple tasks within a single interface and computer screen. The implementation of voice switched windows and synchronization of sound with RAT attempts to alleviate this problem and adds to the usability of both Vic and RAT. In particular voice switched windows promise the ability to improve participant interaction and floor control. However, these features are not widely supported and still require reliable network performance in order to be effective. The planned implementation of sound localization within RAT will also be of potential benefit to conference management.
Information gathered from the Contraband project highlighted the need for application sharing systems for Microsoft windows platforms [bennett 4]. The feedback gathered on tool usability and additional user requirements have indicated that users felt it essential to have support for windows based platforms that would enable them to use application and presentation tools with which they are familiar.
A recurring limitation of multicast networks is interaction with users on non-multicast enabled networks, such as standard ISP provided internet access. This is one of the primary hindrances to the larger deployment of multicast technologies within corporate environments. While it is feasible to set up a Multicast conferencing system within a corporate Intranet, the usefulness of this is limited to participation by members of the conference all being able to access the same network. Individuals without access to the multicast network are unable to participate in the conference.
While there are a number of solutions to this problem, such as the use of RTP packet reflectors, these solutions tend to require experienced networking personal to set up. This places them beyond the grasp of casual or inexperienced users to implement. Gatewaying tools and reflectors, such as those developed within the MECCANO project, are designed to address this situation.
As in the case of the seminars, the issue of firewall security has proved to be a hindrance to multicast conferencing. While this problem is only an issue to members who are part of institutions with firewalls, such as Hewlett Packard, it is a sufficiently significant problem to warrant investigation.
User response within activates supported by the MECCANO project generally revealed that the current release of the MECCANO tools provide sufficient functionality and stability for the purpose of enabling basic collaborative video conferencing. The results from the support ACC has provided to Warsaw University, John Paul II hospital as well as scientists from the Faculty of Physics and Nuclear Techniques, at the University of Mining and Metallurgy, Krakow and the support provided to the CONTRABAND project [bennett] by TELES have indicated a general overall satisfaction with the videoconferencing tools and procedure. There was little indication of any difficulties with user interface design or the ability to engage with the tools. This is particularly applicable to later releases of the tools which have incorporated improvements to interfaces and functionality identified by earlier uses of the tools. In particular users commented on the increased stability and reliability of the later versions of the tools released in the spring of 1999. The validation projects have revealed, however, that there is a need to implement more sophisticated features within the tools such as application sharing and gateway access
However, there is also a certain inconsistency amongst users in their responses to, and satisfaction with, Video conferencing technologies. Results from the CONTRABAND project have indicated, for instance, that the users had to be urged to try and use video conferencing technologies, even though equipped with simple and straight forward GUIs and a number of simplifying control mechanisms developed within the project. Users indicated that the use of multicast conferencing seemed initially too complex. In addition, the potential gains in productivity were not immediately apparent so there seemed little incentive to use the tools.
This situation is reflected within Hewlett Packard where the potential for establishing multicast tools for video conferencing exist, yet have not been widely taken up. In general the usage of desktop IP multicast conferencing, while increasing, is still faced with limitations. Within Hewlett Packard for instance the primary limitation on its use is the lack of a coherent multicast-aware infrastructure. However, it is also likely that a large part of the hesitancy is due to the lack of a communication culture which employs this form of interaction. While initial responses to the concept of desktop conferencing and trials with the tools were positive, the actual willingness to install and implement the tools has met with a lack of enthusiasm. While users who were approached within Hewlett Packard indicated that they would be interested in desktop conferencing solutions, the current approaches appeared too difficult and unpredictable to set up within the current environment. While they worked well enough, the added functionality they provide still does not appear to offer enough of a benefit to be deemed to be worth the trouble. These results would indicate that a large hindrance to the use of multicasting is based on education and that, as the dissemination of multicast technologies tends to grow and mature, the use of desktop multimedia conferencing will also grow and become a more integrated aspect of everyday office working.
A major area of interest for the use of multicast video conferencing tools has been in the area of education, in particular for language learning. However for this type of course, the ability to integrate effective multi-modal communications is particularly important. The use of networked communication programs like the Mbone tools can enhance a learning program by providing immediate or delayed support from teachers to students in an interactive and meaningful way. The facilities of being able to engage with a teacher both visually and audibly for memorizing and retaining material are a desirable aspect of tele-educational tools.
The conclusions from this report are largely based on information derived from the PIPVIC projects and from research undertaken at Hewlett Packard into the role of multimodal communication in distance education. In addition information gathered from support provided by ACC to the University of Warsaw and that provided to the virtual University Project VIROR by UM and UF have given valuable information on the usability of the tools and the quality of the interactions they provide in a tele-education situation.
The activities of the PIPVIC2 project involved an investigation of the suitability of the video conferencing tools for delivering tele-education courses. The project undertook a number of language training trials with a number of HEIs across Britain [sasse 5]. The trials implemented a wide range of approaches to setting up and delivering the courses. For a more detailed description of the PIPVIC2 project see the project’s website [pipvic]. The studies undertaken by the PIPIVIC2 project focussed on participants’ subjective assessment of audio and video quality experienced during the activities. A number of collaborative teaching activates were conducted as part of the small scale pilot most of them foreign language teaching. The project’s focus on quality assessment of audio for foreign language teaching offered a stringent test. [sasse 5]
In addition to the evaluation within the PIPVIC2 project, the tools were evaluated within HP as part of support provided to a small business, which delivers language training for corporate clients. The business “Fluency Training” was interested in evaluating the feasibility of delivering language courses over the Internet. The support extended over a four-month period and involved the use of a number of approaches to delivering online training in this manner. The MECCANO tools were evaluated as part of this process.
The tutorial sessions were structured largely to assess the effectiveness for delivering language training in small group, and one to one scenarios. The MECCANO tools used were RAT, VIC, and NTE. The text of the day’s lesson was loaded into NTE. This proved useful, as students were more able to test pronunciation during the tests etc. The primary evaluation criteria for these sessions were ease of implementation, and the quality of the audio and video.
The installation of the tools in the PIPVIC2 project occurred on a number of different platforms. The installation of the tools on PC platforms consists of simply running a self-extracting binary installation file. On workstation platforms such as Sun the software was provided as a binary file.
Difficulties encountered in the installation process tended to arise from variations in the specification of the hardware components in PC’s. Typical of these problems were incompatibility with audio cards and or driver in the early stages. Unsuitable analogue peripherals such as headsets for desktop conferencing activities and echo cancellation equipment for room based activates were also a problem [sasse 5]. The speakers involved in the VIROR project were generally unfamiliar with the tools and needed both an introduction to the tools and instruction on how to do a tele-presentation. This is a typical problem with newly developed tools, especially within the PC market where the range of peripheral components is large and can often involve the use of non-standard drivers, etc.
While replicating multimodal communication channels in order to help manage sessions is a general problem within video conferencing situations it is particularly problematic for educational situations. Mechanisms such as raising hands, and other physical actions are difficult to reproduce in a remote situation, and are often central to an educational interaction. Response from students in the PIPVIC2 and HP trials have indicated that initially there was a sense of awkwardness amongst users about the interactions between people in the sessions.
In the PIPVIC2 trials a rating system was used to assess the quality of the sessions delivered. The rating system was based on a scale between 0 and 100 and was gathered on a session-by-session basis. Users were asked to place a check on the scale at the level for which they would rate the quality of a particular session. The students rated the adequacy of the audio in general to be between 73 and 90. The quality of the video was typically rated around 84-93 [sasse 25].
For video sessions in both PIPVIC2 and HP the most commonly mentioned limitation was lip synchronization. As the video images in the sessions were generally set with relatively low frame rates there was little possibility to synchronize the audio and video. In general it was found that the video images did not play a large role in aiding language understanding [sasse]. However it did serve to confirm the presence and reactions of the participants, and while students indicated that the presence of the video did not make a large difference to the quality of the teaching experience, they still felt that the presence of a video channel was desirable.
While the quality of the audio and video is important in any conferencing scenario users in the educational situations indicated the greatest dissatisfaction and intolerance for poor quality audio and video. Tutors commented that the inconsistency of the audio made teaching the course difficult, as they were uncertain of how well the students were hearing them. [sasse 28]. In the HP trials the comments from the tutor was much the same who commented on the variability of the audio quality, which arose from the unreliability of the network conditions.
The ability to record sessions is of particular advantage to tele-educational applications. Revision is an important aspect of any educational experience and the capacity for students to be able to record sections of a teaching session would have a high value. Currently the MECCANO toolset has a number of different tools for recording sessions. One set of these, AOFwb and AOFrec was used within the VIROR project and proved quite successful. The recordings from a lecture held in year one of the project by Prof. Dr. Effelsberg is currently sold to students on CD-Rom and is now used by students for examination preparations.
As with the video conferencing the primary limitation, which arose in the initial HP trials, was the limited multicast network access for the participants. In order to use the MECCANO tools all participants were required to operate on a multicast enabled network. In practice this meant providing multicast machines that were set up and connected to the multicast network to enable them to join the session. In a real world usage this is unlikely to be the case and prevented the tutor from delivering the session from his home or a remote location. This highlights the need for resolving access for participants across multiple networks, allowing users who are logging in over a non multicast IP enabled network to have full access to multicast conferences. The tutor who was delivering the course at HP felt that although Multicast technologies offered distinct advantages not only as an educational tool but as a business model as well (i.e. he could deliver language course to multiple participants rather then just the one to which current point to point solutions, such as Netmeeting, restrict him), the lack of proper multicast support in the internet made using multicast conferencing tools unrealistic at this point.
As with the conferencing scenarios, the ability to integrate support material into the teaching session is important. While NTE and WBD allow this feature through the importation of postscript slides, the tools need to be expanded to include a greater range of options. In particular they should be able to import PowerPoint slides, or jpg, and gif images. Lectures in the VIROR project frequently indicated that the process of getting slides in Wb/Wbd, DLB and AOFwb is still complicated. This was especially a problem when using Powerpoint presentations. This is also a problem with the Digital Lecture board in that it uses Postscript for slide importation, a format with which it is difficult to work.
This section has attempted to provide an overview of the current state and usability of desktop conferencing systems for particular session types. It has outlined the basic issues and requirement for these systems and has evaluated different approaches to delivering the three primary session types. The trials would indicate that there are four broad areas, which need to be addressed to insure general usability of the tools and broaden their usage.
Video conferencing research, and results form user response within MECCANO has shown that collaborative work requires the ability to easily integrate separate components into a integrated conferencing application. The tools however also have to retain enough modularity to be able to be used separately or to be adapted and used for differing or specific applications settings.
The ability to record store and deliver multicast sessions in a controlled manner is an essential component of multimedia conferencing sessions. This is particularly important for educational applications and seminars sessions. Managing sessions for later evaluation will be a particularly useful feature of any conferencing scenario.
Typically there are a number of tools, which are used to assess Mbone traffic and performance, such as Mtrace and MRTG. These programs are useful for monitoring Mbone traffic statistics. However the difficulty in developing a clear understanding of the usability of the MECCANO tools has been in understanding user responses in a dynamic manner, to a session in terms of quality.
In addition online collaborations particularly of an educational or seminar style session can gain added value from the ability to monitor and gather feedback from participants within the session. There is a need for some form of multicast tools, which can gather user assessment of a session in a dynamic manner, and compare them to traffic statistics, etc.
It is apparent that the development of future communications networks will extend beyond any single network infrastructure. The facilitation therefore of support to bridge differing network models is necessary. Gateway tools which allow ISDN, Satellite, cellular, Plain old telephone networks, Multicast and Unicast networks as well as others to participate in collaborative conferencing will be increase the usability of the core conferencing tools. These tools function to enhance and improve the accessibility to and effectiveness of the conference.
Strictly speaking issues of network reliability were out side the scope of the MECCANO project. In practice however network conditions were so inextricably linked to the ability to implement and assess the tools that it is an issue, which needs to be discussed. In general the quality reported by users across the MECCANO projects can be characterized as highly variable. In some sessions users were quite satisfied with the tools and the interactions, in others they found the quality of the session to be extremely poor. Typically for example users may comment about the poor quality of the session based on poor network performance, or on difficulties with their hardware such as non-echo canceling microphones, etc. These sessions however do not necessarily reflect on the actual usability of the tools.
This points out one of the biggest hindrances to assessing the usability of the tools, which is the unpredictability of network performance. It was quite common for persons in the same session but at different sites to report widely differing levels of quality in the audio and video transmissions. Often for example CRC was unable to join in Partner meetings due to the poor connections between the Canadian Mbone and the European Mbone. While these sorts of real world conditions make it difficult to gather assessments of the tools, it does aid in pointing out the interdependent nature of multicast conferencing. In order for multicast conferencing to work not only are sophisticated tools required, but network quality needs to be able to be guarantied to a certain level.
This section provides an overview of the current versions of the tools used within the MECCANO project evaluates their usability. This section is divided into four sections, which reflect the various usability issues encountered in the usage and support provided within the MECCANO project. This section of the report is not an exhaustive list of the individual’s tools features, nor is this section designed to provide an introduction to the tools. The intention of this section is to instead give an overview of the tools and to outline their usability advantages and disadvantages
During the period of this report the tools have undergone constant refinement. Some of these releases are experimental releases, which are designed to test new features. The basis for the usability report has therefore used the most commonly used and most stable version of the tools.
The over view of the tools provides a general description of the tools purpose and functionality. It then discusses any features of the tool which are designed to address any issues encountered in the assessments described in section 2. An overview of the usability of the tool is then provided based on their suitability for their specific task, and target user base.
While the core conferencing tools here are listed individually, in actuality they closely function together in their ability to deliver a successful collaboration session. Ideally the separate functionality embodied by each tools should be able to be integrated with the other tools in order to provide a fully functional session. On the other hand it is desirable to be able to use the tools separately in order to accommodate members of a session who may only wish, or have to join using only an audio connection or the whiteboard. One of the advantages of the current approach in the MECCANO project is that this is often the case. Users can join sessions using a variety of different tools.
SDR, the Session Directory developed at UCL, is a directory service interface to the Mbone. SDR’s primary purpose is to allow participants to access sessions without the need to explicitly know the address of the initiating member. It is used for browsing and the announcement of public multicast sessions as well as the establishment of private sessions. SDR employs a simple interface, which allows users to browse advertised sessions, join sessions, and create their own sessions.
Figure 3: SDR
SDR incorporates a user-friendly interface which aids in creating and announcing sessions. It has an extensive online help facility, and a clear step by step procedure for initiating sessions. SDR also incorporates the ability to create secure conferences in a relatively intuitive manner.
Figure 4: SDR - UW lectures details
SunOS, Solaris, SGI, DEC OSF, HPUX, FreeBSD, NetBSD, Linux, NeXT, RS6000/AIX, Windows 95/NT.
To date SDR implements a sufficiently full set of features for organizing and handling session announcements. The interface for browsing sessions is reasonably clean, and employs a number of features, which enable the categorization of the session displayed. For example users can choose to show only public sessions, or seminar broadcasts or only the current days sessions. SDR also allows for intuitive step-by-step procedure for initiating a session. The procedure is explained in a manner, which would make it possible for a non-experienced user to announce and start a session. This is a key usability feature of SDR as it allows sessions to be announced and created with a minimal amount of difficulty. Users are not required to understand Multicast addressing systems, or media delivery transport protocols and codecs. Version 2.7 specifically contains a number of improvements to its user interface. The creation of sessions is a particularly well designed improvement to earlier versions and provides an easily understandable guide through the procedure of setting up and announcing a session.
The additional feature of integration with web sites, such as the ability to provide links to a session owners web site is a particularly useful feature, as it allows for the provision of additional information about session originators to be delivered to users. As will be discussed later the ability to integrate conferencing tools within a web environment has advantages for the delivery of conferencing solutions.
The only current limitation in SDR is in the handling of large numbers of sessions. Currently browsing sessions on the Mbone through SDR is trivial in terms of cognitive load. However as the amount of multicast sessions grows revisions will need to be made to the design of SDR in order to more efficiently categorize, filter and display very large number of sessions. While SDR therefore is quite usable under current Mbone conditions it is likely that further growth and development in the sophistication of Multicast conferencing will require either serious redevelopment of SDR or development of alternative methods of announcing and controlling multicast conferences.
The secure conference store allows for the creation and management of secure conference announcements.
Figure 5: Secure Conference Store
The secure Conference Store allows for secure local control of session announcements. Users can log on to the conference store and using either secure certificates or a password to access information on and join or create sessions.
The primary advantage of the tool is that it provides greater control of managing conferences than does SDR. In addition it allows local network administrators to set up and control secure conferences. It is straightforward to access and use for users, and allows sessions to be joined with little effort or need for configuration of multicast addresses on the users part. In particular the ease with which secure conferences and the handling of certificates is managed makes it useful for non technical users for whom the use of secure certificates may not be clear.
RAT is an audio tool which allows users to participate in audio conferences over the Internet. RAT can be used for either point to point conferences between two participants or for Multicast conferencing between multiple participants. RAT was developed at UCL and was originally designed to provide increased resistance to network loss in order to insure more reliable transmission of audio in conferencing scenarios.
Figure 6: RAT
RAT enables Encryption for enabling secure conferencing, and advanced redundancy coding for improved audio quality. RAT is also able to handle and deliver high quality encoding, and stereo transmission. Currently work is now proceeding on providing sound localization. This feature will be of useful for creating effective conferencing scenarios, (especially in virtual environments) as it will facilitate the recognition of speaker location, which is a key factor in aiding group communication. RAT also provides Full Duplex Audio Transmission, which allows users to listen to the audio channel and talk at the same time.
RAT is currently stable and provides a sufficient set of functions for delivering realtime audio communications over the internet. Using RAT is uncomplicated and appears to present no initial difficulties for the non-experienced user. The Interface makes it obvious as to the how to control the conferencing options. Minor difficulties with the interface are largely confined to the size of the application. With screen space at a premium the RAT interface could be made smaller by reducing the size of the scroll bars and, and overall window size. A particularly useful feature in RAT is the identification in the participant list of which member of a group is currently talking. This is accomplished by highlighting their name in the participant list. This is especially useful when the participating member is not using a video channel as it aids in identifying which members of a conference are talking. RAT delivers audio quality at a sufficient quality level to allow for near natural audio communication between participants. It also allows for more in depth features to be accessed by individuals with more experience in multicast conferences such as control over specific compression codecs.
VIC is the video capturing application originally developed at Network Research Group at the Lawrence Berkeley National Laboratory in collaboration with the University of California, Berkeley. The UCL version discussed here has made a number of changes to the initial design VIC also has the ability to adjust the bandwidth rate at which the user is transmitting, which will allow them to configure VIC for low bandwidth or high bandwidth scenarios.
Figure 7: VIC
Like RAT, VIC has a simple interface which is easily grasped by novice users with little or no training. VIC supports a number of encoding formats, including H.261 and Mjpeg and allows users to make changes to the various features of Vic in order to cope with problems, which may occur as a result of network congestion. One of the key usability features of the current Mbone tools is lip synchronization between Vic and RAT. While the performance of this feature is limited by network performance, it is a useful addition for improving interactions in a conferencing scenario. Other useful features are the ability to generate notes that users can associate with their video image and the support for secure conferencing.
Vic operates on the following platforms, Linux, Windows, Sun (Solaris and SunOS), HPUX, Dec (Alpha and Ultrix), RS/6000 AIX and SGI IRIX.
In general VIC is well developed and provides the basic functionality required of a video tool. The interface is straightforward and provides easy access to the primary functions, which users may need to set up or change during a conference. The essential features for transmitting and receiving video options are easy to access and understand. It also allows users to modify more complex features like transmission rate control, encoding formats, and display options. These features however often require a more experienced understanding of networking and video conferencing. While there are minor changes which may be required at the interface level VIC along with RAT and NTE (Network Text Editor) are the most stable and mature set of conferencing tools examined. It is quite feasible to accomplish most basic conferencing tasks through the use of these tools. However as is discussed later there are current limitations to VIC, which arise as a result of limitations in the current bandwidth capacity of the underlying network layer for delivering video rather then in any inherent limitations in VIC basic functionality.
NTE (Network Text Editor), was developed at UCL and allows users to simultaneously access and edit text documents. NTE allows users to open standard text files, which the participating members of the conference can subsequently edit and comment upon.
Figure 8: NTE
NTE displays individual users’ contributions in different colours, which makes it easy for users to rapidly identify the source of comments and changes. It employs similar menu structure as standard text editing programs and allows loading and saving of plain text files, as well as importing of text as editable blocks
Allows for explicit assignment of colours to specific text blocks. This allows the colours to be used to distinguish between different people editing the same document. The font styles for the current editable block can also be set. Users can identify parts of the application interface to be hidden to save screen space. NTE provides a locking feature that enables users to control whether or not other members may update or delete other specific text blocks. This feature aids floor control. NTE also supports secure conferencing.
SunOS, Solaris, SGI, HPUX, Linux, Windows 95/NT.
NTE has proven quite usable for networked conferences even by users with little experience of networked applications. A particularly useful feature is the color and text coding facility as this greatly facilitates interactions between group members. It is particularly useful for collaborative work where multiple authors are workings on a single document. This feature has also proved useful in creating and maintaining meeting agendas as it allows members of the meeting to make amendments to the agenda as required and for those changes to be easily identified. It was also useful in the delivery of language lesson over the web as it allowed for the identification of text, which the pupil was meant to learn and speak.
The only significant hindrances to NTE are document control and management, and file importation. Perhaps the most immediately useful feature for improving NTE would be a greater range of document conversion filters. This would allow for the importation of a wider range of documents. However it is not clear whether this is in practice desirable or advantageous. The number of differing file formats which currently exist are large. While NTE is able to import standard text files it is hindered by the inability to import more structured text formats like rtf or pdf. Although NTE is currently sufficient for small documents and basic network interactions it lacks the facility for sophisticated large document control and navigation. In the light of the growing interest in the use of video conferencing for collaborative work the implementation of more sophisticated document management features may be desirable within NTE.
WB is a shared whiteboard application for Unix based systems.
Figure 9: WB
WB consists of two parts, the Main WB Window which is used for sharing whiteboard style graphics, and a control panel which lists and manages members of the conference.
Figure 10: WB – control panel
The Control Panel allows users to mute other members in a session, and to get information about the members of a session, as well as information on the activity which has occurred in a session. WB automatically switches to pages based on where a user is drawing. It can import postscript files and text files and pages in WB can be saved or printed.
As a whiteboard WB functions well and is very stable. It provides a sufficient range of features to aid in network collaboration. While it lacks the features of tools like DLB it is suits its purpose well.
WBD is shared whiteboard, which is based on WB. It runs on a number of platforms but is mainly aimed at providing Windows95/NT4.0 users with a whiteboard which interoperates with WB on the UNIX platforms.
Figure 11: WBD
WBD allows participants to share a graphical space, which they can draw upon, or enter text, which can be view and edited by other participants. WBD however is not a text editor like NTE and once text is entered it cannot be reedited. Its primary usefulness is for sharing graphics, or slide type material, which members of a group can comment upon, or as a Whiteboard for discussing ideas, etc. It has the typical range of features of a shared whiteboard, such as participant’s lists.
SunOS, Solaris, SGI, DEC (Ultrix/OSF), HPUX, FreeBSD, NetBSD, Linux.
As a shared whiteboard WBD is reasonably useful, and provides the basic functionality, which would be required of this type of tool. Like NTE however its primary limitation is the limited range of options for importing existing slide material into the program, and more sophisticated document management features. While it can import Postscript slides and text files it is unable to import PowerPoint slides or PDF formats. Equally it is not as sophisticated in its control of conferencing sessions as WB or as tools such as DLB
The ReLaTe interface provides an integrated interface for the separate MECCANO tools discussed in the previous sections. It is especially effective for applications needing limited numbers of video windows. Using this interfaces provides the user with a single window in which are embedded video (VIC), audio (RAT) and shared workspace tools (NTE and WBD) between which the user may switch easily. The ReLaTe interface was developed for the project of that name [relate] by UCL and Exeter University; it was packaged by the UKERNA project SHRIMP [shrimp] as the SHRIMP Interface.
Figure 12: The integrated interface (SHRIMP)
The primary feature of ReLaTe (SHRIMP) interface is that it provides an integrated interface for the tools and enables switching between the whiteboard and NTE.
Sun solaris, IRIX, Windows
While an integrated interface does not quantifiably improve the actual usability of the tools, it is a desirable addition to the tools in that it allows individual users to choose between interface styles. Some individuals prefer a decoupled set of windows for each tool in the conference. Other users, especially novice users may prefer to use a single integrated interface, which allows them to access and configure the various tools from a single set of menus. This works particularly well with NTE and WBD in SHRIMP and ReLaTe as users can easily switch between the two applications. By incorporating a single user interface into the tools the MECCANO tools add an additional layer of functionality to the tools which improves the overall usability across a broad range of users. In conferencing scenarios this would be a distinct advantage as it is likely that a disparate range of users will wish to use the tools. This feature highlights one of the advantages of a replicated architecture in that it allows individual users to tailor the applications interface to best suit their methods of working, without affecting the other members of a conference.
Figure 13: FreePhone
Like RAT it has provisions for redundant encoding techniques and implements a variety of codecs for use in differing network conditions [bennett 5]. Provides support for concurrent unicast and multicast conversations. FreePhone allows the user to establish direct calls to remote participants using a simple call button.
SunOS, Solaris, Linux, FreeBSD, SGI (IRIX 5.3), and PC’s
FreePhone is highly usable in its basic implementation. The interface is simply to understand and use. The mechanisms for placing and receiving calls are straightforward and present little difficulty. The primary limitation of the tool is only in the lack of direct support for Windows NT. While this is not a major limitation currently the wider distribution of FreePhone will require support for NT as well
Also developed by MECCANO partner INRIA, Rendez-Vous is a Audio and video tool for participating in multicast or unicast conferences.
Figure 14: Rendez-Vous
Rendez-Vous’ functionality is similar to VIC’S and RAT’S, but has the advantage of integrating the audio (FreePhone) and the video into a unified interface [bennett 4]. It also implements new approaches to video encoding and process management to optimize the coding and decoding of video. Allows for easy configuration of conferencing options and functions.
Sun Sparc Solaris, Linux , FreeBSD machines, Windows 95
Like FreePhone, Rendezvous is very user friendly and allows novice
users to easily join and participate in multicast or unicast conferences. The
interface design is especially useful for novice users in that configuring
options are reasonably easy, and accessible. Although more advanced options are
likely to be confusing to novice users, Rendez-Vous represents a good
refinement in general usability from a non expert point.
The tools in the following section reflect the development of more sophisticated session management systems for the delivery of conferencing and educational interactions. DLB and AOFwb/AOFrec attempt to provide a teaching and learning environment consisting of full-featured document sharing and management along with provisions for recording and replaying sessions. MMCR and SVDA address session recording and playback requirements in a conferencing session.
DLB was developed at the University of Mannheim and is intended to address some of the current limitations with shared whiteboards. DLB is largely designed to address the shortcomings in RAT, VIC and WB which hinder their functionality for collaborative working. In particular DLB is intended to facilitate tele-teaching, and collaboration through a more sophisticated whiteboard application and through support for session recording and management.
Figure 15: DLB
DLB is intended to be able to be used in an integrated teaching environment, which will support most of the synchronous tele-teaching requirements (construction, transmission, recording, retrieval, playback, and preparation of lectures and teaching materials). It also supports a variety of media formats: ps, gif, ppm, pcx, bmp, eps, ascii and is network compatible with the AOF whiteboard [geyer].
DLB implements enhanced floor control and session control mechanisms and policies. Floor control provides concurrency control for cooperation between people in a session. by using the metaphor of a floor. Session control increases social awareness in distributed work groups because members gain knowledge on each other and their status in the session. DLB supports synchronous features such as, reference pointing, the ability to form sub-groups and control over the course of instruction. It also supports on-the-fly development and importation of materials.
AOFwb [lienhard] is a shared whiteboard tool developed by The University of Freiburg. This toolkit includes a complete system for creating, transmitting, storing and reproducing multimedia content to be used in distributed presentations.
Figure 16: AOFwb
Like DLB, AOFwb and AOFrec have a number of additional features in comparison to the standard Mbone whiteboard. AOFwb's main features are: save and load options, the incorporation of thumbnails, integrated audio and whiteboard recording, drawing tools, grouping and an external application controller. In addition to plain presentation of the prepared material, AOFwb allows the lecturer to interactively edit texts, point to or outline parts of the text, show high-resolution graphics etc. AOFwb and AOFrec add to the functionality of shared whiteboards by also allowing users to record audio and whiteboard streams. A multicast presentation, for example, can be recorded with the synchronization information and can be reproduced later. The AOFsync tool is used to do this synchronization.
Figure 17: AOFrec
An additional component to the toolkit is wbcontrol which allows users to view thumbnail previews of the slides and jump interactively jump through the various slides of a presentation. wbcontrol can actually be considered a separate tool as it can be used with the standard Mbone tool WB.
Linux, Sun Solaris, and IRIX systems.
Currently AOFwb and AOFrec are only available for Linux, Sun Solaris, and IRIX systems. This places a limitation on the environments they can be operated within. Within some corporate environments for instance the range of platforms may be limited to one or two supported platforms, as is the case at Hewlett Packard. However as a basic tool the functionality of AOFwb and AOFrec, seems reasonably promising. In particular the conceptual approach to allowing the recording of presentations by the receiver is a useful feature for seminars, presentations and educational situations as it gives greater control to individual members of a session. Usability constraints on AOF tools are related to the importing of Postscript documents which may not be straightforward for non experienced users, and the lack of alternative document formats such as pdf .
AOFwb, AOFrec and AOFsync all have simple and effective graphical interfaces. AOFsync in particular makes it relatively straightforward for the user to reproduce recorded sessions. AOFsync adds to the usability by allowing users to recreate the sessions sequentially or by using a thumbnail control feature they can reproduce a specific section of the session.
wbcontrol improves session management by giving users greater control over the presentation. By using a thumbnail control feature wbcontrol also aids in the preparation of a session. This is of particular advantage for educational and seminar style sessions. Like the other AOF tools wbcontrol has a very easy to use interface and is clear and straightforward in its usage.
MMCR [lambrinos] from UCL is designed to enable the recording of multicast conferences. MMCR allows for full browsing, storage and retrieval of multimedia sessions.
Figure 18: MMCR
MMCR consists of a client-server architecture. The Server runs on a remote machine from the client and upon request stores the media streams received from the multicast network. The stored streams can then be edited and reviewed by individual members ore or replayed into other conferences. MMCR also allows the storage and retrieval of encrypted sessions.
MMCR has obvious benefits for seminar type sessions and for educational applications, by allowing a facility for recording sessions, institutes can potentially create archives of educational material that it can provide to clients, and can record educational sessions, for later review or for the purpose of assessment. In terms of usability for basic conferencing scenarios the use of MMCR is not essential. However it is potentially one of the most useful of the Mbone tools for seminar sessions, and educational scenarios. MMCR would be particularly useful in an educational setting as it would enable the storage and re transmission of teaching sessions, evaluations exercises etc.
The client part of MMCR employs a graphical front end, which allows users to easily set up the recording of a multicast session. The client end is easy to install, and logical in its layout and access to features. This provides local control to users for browsing recorded sessions and for recording current sessions. In addition the client does not have to be an active participant in the session, in order to record it. MMCR also allows users to be able to browse and playback specific streams in a session.
Figure 19: MMCR – recording window
The server part needs to be installed on a suitable Unix machine. While this is reasonable in the current stage of videoconferencing development, easier to install and configure implementations would be desirable. In particular support for a wider range of platforms should be considered. This would broaden the potential user base for MMCR as it would make it accessible to small groups, and institutions with limited IT resources.
The Scalable Video Distributed Architecture (SVDA) is an environment that introduces enormously large storage capacity devices into multimedia environments. In modern multimedia environment there is a tendency of moving the responsibility of storing and delivering video data onto video servers. Such a solution has a lot of well-known advantages: clients don't need to be equipped with large discs, a video stored on a video server can be delivered at the same moment to many clients and thus the need of keeping the same data at many places is refused, etc. The solution presented in SVDA follows that idea. Furthermore, it enhances the functionality of video servers by adding third-party devices that can save video files on its low access time, but large storage capacity devices.
The SVDA system has been built upon the following assumptions:
Figure 20: SVDA client application with Java GUI
Figure 21: SVDA - a user session
The first SVDA release usage gave developers a lot of feedback about system inconveniences and unstable running. Since then lots of work has been put to correct bugs found in the previous system release. Current, second release of SVDA system is much more stable and reliable. To enable better monitoring of the system state and behaviour additional modules has been incorporated into the servers code – on both VFS and AS Proxy servers like also on Main SVDA server. They are responsible for collecting information about the system component current state and sending it to the main monitoring application which visualise them
This step enabled users to perform some test about system configurations used and their impact on the efficiency and reliability of movie delivery and the whole system performance. Monitoring application visualisation components were written in Java language with Swing GUI library. They provide a comfortable way of continuously observing the system behaviour.
For example it is possible to observe how the movie priority, stored on AS Proxy server in the system is changing, when the system client request it at the given moment and what is the system behaviour according to those changes – see Figure 22 for priority value changes over the time when the request is made
Figure 22: SVDA - Movie priority monitoring window
Together with server side improvements serious corrections and improvement were done to the client side of the system. The whole client application has been developed once again, this time based on Java and Swing GUI. Current version of the SVDA client may be run as a standalone Java application as well as a Java applet – from a Java capable web browser such as Netscape for example. The only limitation to the applet version is that the player application cannot be remotely started – due to security issues in Java applets. That’s why we decided to divide the JMFPlayer from SVDA system and put it as additional standalone piece of software. The user can now download [mecpol] a JMFPlayer separately and run it on his workstation with a given parameters to receive media stream delivered by the SVDA system.
Java MPEG-1 Player application developed by ACC firstly as a part of SVDA system now is a standalone component, which has a capability of playing MPEG-1 streams received from the Internet or a given path to file . This is a Java language application based on a Java Media Framework API (JMF) – a part of Sun’s Java platform extension for media-based application development. It is capable of receiving unicast and multicast transmission of video streams.
JMFPlayer development was initially based on JMF1.0 implementation, which was rather rough in usage and had very small capabilities in comparison with current JMF2.0. That version was finally used in JMFPlayer release. Sun’s implementation of JMF API covers only Solaris and Windows operating systems. This gave us a possibility to compare Java media-based application usage on those two platforms. Due to those operating system limitations in JMF, one of the Java language advantage system independence could not be used as it should be. Some works on preparing JMF release on Linux are carried on, but they didn’t achieve any serious success yet.
The main directions of our software component usage are:
§ AVSC-based multimedia system
§ playback of MPEG-1 real-time streams or files
§ as a receiver endpoint of SVDA system
The first system is a distributed environment for control and management of multimedia data which is currently still under development as a part of work undertaken under EU Esprit Project CRIT-2 – Co-operative Research in Information Technology. The JMFPlayer was used as a core part while implementing MMDevice – user specific multimedia device – a part of the whole environment focused on a user desktop station and interacting with other multimedia devices such as video servers (like Sun MediaCenter or Oracle Video Server) or MPEG-1 real-time streaming devices (like NPoint 6000).
The second direction is tool usage for playing MPEG-1 files stored on local disk as files or those accessible via URL mechanism. The most important part, however, which is not present in many MPEG-1 enabled tool implementations, is the capability of playing real-time streams transmitted over the network. JMFPlayer was developed having that potential in mind and this was assumed as its main feature. That scenario also implies JMFPlayer usage together with SDR tool. SDR plugin specification has been created and the tools can now be used just like other MECCANO tools – RAT, VIC or NTE when announced session is using MPEG-1 coding standard.
SVDA system also makes use of JMFPlayer as the component responsible for receiving transmitted stream at the user end point. That means that JMFPlayer is the second part of the SVDA client application. It is a separate application used merely as an MPEG-1 video player. It is executed by the SVDA module in response for the "Play" operation request.
JMFPlayer further development is dependent on Java JMF API possibilities. If that library will be further developed by Sun and will provide additional MPEG features we will be able to improve possibilities of our software. Currently some works are undertaken to add capture capabilities and enable the component become bi-directional video device. Also some additional works with other codecs implementation would be welcome.
The tools in this section have been designed to provide improved monitoring and assessment of multicast sessions, and network performance. This addresses the issue raised about gathering user feedback.
MPoll [patrick]was developed by the Communications Research Centre in Canada. It is a specialized multicasting tool designed to collect quality ratings and opinions from session participants. MPoll was developed with the following uses in mind:
Figure 23: MPoll
MPoll is easy to install and implement. It requires no special knowledge of multicast issues. Users a can quickly install and set up session questionnaires for a particular session and gather results from users throughout the session. While MPoll does not have a direct impact on the usability of the conferencing tools themselves, it is a useful tool for collecting and organizing session ratings. This can enable creators of session to assess the success of a session. This is a particularly useful tool for educational based sessions as it can allow session organizers and network administrators, to retrieve and analysis ratings on sessions. Difficulties can therefore be pin pointed and addressed for future sessions. This provides a greater responsiveness to user requirements. While MPoll is ostensibly aimed at gathering quality opinions it would be possible to incorporate its features for other aspects of an educational session. Further development on MPoll should therefore consider more specialized adaptation for this usage. In particular a closer integration between the shared whiteboards, such as the AOF tools or DLB would prove useful for educational situations.
MultiMON is a multicast Mbone monitoring tool developed by MECCANO partner CRC, the Canadian Communications Research Centre.
Figure 24: MultiMON
Figure 25: MultiMON - time-based graphs
This tool monitors a network segment, reports on multicast traffic seen and optionally allows one to view and record RTCP statistics. It is built on a client/server basis, which allows the data collectors (Servers) to be distant from the GUI front end displays (Clients). MultiMON is capable of providing both real-time display of traffic patterns as well as the ability to log the traffic patterns. It supports the display of the traffic breakdown by application type, and a time by bitrate bar chart. Sub-windows for each identified session can be spawned from the Client. These sub-windows show the session name, type, current and max bitrate, and allow one to join the session (locally), or to run an rtcp monitor on it. It also allows the collection of long term information on traffic patterns.
MultiMON, is a particularly useful tool for session organizers and network administrators. It allows the users to monitor the amount of network capacity, which is currently being used by multicast traffic, and to measure and monitor the quality of service across the Mbone network. Network administrators on a LAN can therefore pin point network problems and diagnose the performance of the network. MultiMON is of particular advantage to corporate LANs as it can help network administrators increase the overall efficiency of the network.
MultiMON provides a sophisticated range of methods for extracting information on network conditions for multicast applications. Although it requires an understanding of network principles to interpret the data meaningfully it is easy to use and to generate reports on network traffic. It is therefore appropriate in its functionality for the user base it is aimed at.
RQM [perkins] is designed to provide users in a Multicast session dynamic information on the reception quality of that session.
Figure 26: RQM
RQM displays a matrix, which displays the participants in a session and their details. To the right of each participant is a row of cells which shows the packet loss rate observed for data sent from the participant indicated at the left of that row. If a cell is rolled over with the mouse a pop up will appear giving the names of the source and destination of the traffic represented by that cell.
The cells are color coded according to packet loss, and show the percentage of the loss if any. Clicking on a cell will initiate an mtrace between the indicated participants.
RQM is extremely easy to set up and implement. The information it provides allows for easy identification of the quality of reception at each point in the multicast session. Particularly useful is the ability to initiate mtraces between specific participants from the RQM interface. There is no real limitation to the usability of RQM as it accomplishes the specific task it is designed for in a straightforward and easily accessible manner.
The following section discusses tools that are being developed to transmit Multicast traffic to selected destinations using Unicast transport. Results from a number of the support projects have demonstrated that not all IP hosts are connected to the Mbone. In order to broaden the adaptation of the Mbone tools and increase their usability the following tools have been developed within the MECCANO project.
The UCL Transcoding Gateway (UTG) is a relay and transcoding gateway, which is incorporated into Internet multicast systems to permit access by workstations with only unicast connectivity. It consists of a Server Application running on a Solaris workstation on the Mbone and a Java Client Application running on a PC or other workstation having only a limited-bandwidth unicast connection to the server. The client interface controls the gateway links to the underlying Mbone conferencing tools which may be running on the same unicast host as the client or on some other unicast host .
Figure 27: UTG
UTG consists of a Server running on a Sun workstation connected to the Mbone and a client running on a Windows 95/NT machine at the end of a Unicast link [kirstein 33]. UTG allows users to access multicast sessions over N-isdn or other low bandwidth link lines. The client can connect to the server and receive Session information via SDR; when a session is selected, the server will send the relevant traffic to the client at the rates defined by the client. In this way a user is able to allocate the bandwidth between the available media for the session. In order to implement UTG a user at the client end must have access to a provider, or the institute in which the client is operating must have a machine, which is running the Server part of UTG. To date this is not a widespread situation, although there appears to be little hindrance to its adaptation.
Figure 28: SDR plugin window for use with UTG
UTG promises to greatly increase the usability of the Mbone tools in that it will open access to Mbone sessions to a much broader base of users. This will be especially useful for private users, as a large obstacle to multicast conferencing is the limited scope of the current multicast network.
The UTG client is trivial to install and use. It is integrated into SDR presenting the option of connecting to a regular session or a UTG managed session. From a user stand point joining a UTG managed session is easily managed and requires little extra effort than that to join a regular multicast session.
The UTG server is somewhat more difficult to set up, but is not beyond what would be expected of a network administrator’s knowledge.
What is unclear is the whether there are any quality related issues to implementing UTG, i.e. does it have any sort of adverse impact on the quality of the audio and video segments of a session. Initially this does not seem to be a problem, but further development and testing under real world conditions is desirable.
AudioGate (Audio Mbone-Telephony Gateway) is designed to provide participants on telephone networks access to the audio channel of an Mbone Conference. AudioGate would allow users to call a phone number, which would transfer the participant to a pre-selected Mbone session.
AudioGate offers one of the more interesting tool developments within the project, as it opens access to multicast sessions to users of mobile phones. Initial demonstrations of AudioGate showed it to be easy to use allowing for easy connection to a multicast session. AudioGate has the capacity to make access to multicast conferences more transparent and accessible. AudioGate allow users who wish to participate in a conference, yet are unable to access a desktop computer to dial into the conference from a phone line. This feature would greatly increase the usability of the overall system, particularly as it usable in conjunction with mobile phones. The primary limitation of AudioGate is a lack of broad Platform support. Currently it requires Linux and a special Audio card using isdn to implement.
The Multicast-Unicast Reflector is designed to allow users to participate in Mbone sessions without direct access to the Mbone, The Multicast-Unicast reflector, consists of a multicast-to-unicast packet reflector which joins a chosen multicast group and replicates the data traffic to a given set of unicast addresses. A modified RTSP server controls this unit which, makes integration with the WWW simple. A user reads the information about the current Multicast sessions using a web browser. Clicking a link for a Multicast session returns an RTSP-file containing the URL for the session description such as rtsp://rtsp.ifi.uio.no/seminar.sdp.
The reflector is controlled using Real-Time Streaming Protocol [rtsp]. RTSP simplifies Web-integration and includes the necessary streaming control functionality for both Unicast and Multicast transport.Design criteria for usability were:
1) using the reflector to stream media must be as simple as browsing the Web
2) the only assumptions about users’ capabilities is that they be able to use a Web-browser and run an installer application.
The Multicast Unicast Reflector has two parts a client and a server. The Client is available for Windows platforms and for Unix based platforms. The Client element is easy to install and demands little on the side of the user to set up. Its primary advantage is that it is incorporated into the web browser so can be integrated into a users normal Internet browsing activity.
Implementing the server module of multicast Unicast reflector engine is more difficult although not beyond what would be expected of a network administrator.
The MECCANO call signaling and media transcoding gateway StarGate is supposed to provide connectivity between different kinds of endpoints interconnected through different types of networks (hence the name “*Gate”). This is expected to include in particular:
The architecture of StarGate also allows us to easily extend the number of supported call signaling protocols. In addition, if feasible from the standardization point of view (i.e. the necessary specification are complete and stable), security aspects will be incorporated into the StarGate implementation.
In general the MECCANO project has adequately identified the main issues, which affect multicast sessions, and are developing solutions to address these issues. The wide range of tools being developed within the MECCANO project address the primary requirements for delivering real time multimedia conferencing. The primary usability limitations encountered in early trials and usage, such as conference recording, session management and session access are currently being addressed within the MECCANO project by tools such as MMCR, AOFrec/AOFwb, and SVDA, as well as gateway such as UTG which would increase the accessibility of the tools. In addition the core tools have shown a demonstratable growth in their sophistication and usability. Both the features of the tools and the support for more sophisticated collaborative aspects such as session control, session recording and management, and the ease of use of the tools has significantly progressed since the beginning of the MECCANO project.
The current development of the core conferencing tools has indicated an acceptable level of stability and performance. In particular the variety of tools and diversity of approaches to expanding the functionality of the core tools is useful. The variety of tools within the project allow for a number of different types of implementation to be adapted to suit a specific users unique requirements.
While there are a number of usability issues, which hinder current multicast conferencing sessions, these issues have been identified and either have been addressed or are in the process of being addressed within the context of the MECCANO project.
In terms of basic video conferencing features the core tools such as Vic, RAT, Rendezvous, NTE, DLB, and AOFwb/AOFrec are well suited to the task of providing basic video conferencing facilities. In terms of improving general usability the primary issues remaining is to broaden the platform distribution of these tools, and refine the document management features. Some of the most promising developments within the project are the tools, DLB, and AOFwb./AOFrec which are currently available on a limited range of platforms. In particular the development of these programs for a wider range of platforms will allow for further and more detailed testing of their session management and recording features. In addition a wider range of document formats needs to be incorporated into NTE, DLB and AOFwb. In particular support for RTF and pdf format would be desirable from a user convenience perspective. The development of Wbcontrol, is typical of the type of document management features which need to be expanded in order to enhance the usability of the tools.
The primary limitation still remaining in the tools set is largely Gateway related issues, and Session management and recording. In particular further development of Gateway tools, and resolution of network access and performance would seem to be the most urgent task for improving the general usability of realtime multimedia conferencing. In particular the tools UTG, Audiogate, and Unicast Multicast Reflector should be further tested and refined in real world situations. In addition the issue of accessing multicast session through firewalls needs to be addressed in conjunction with the development of other gateway tools.
The current state of development of the Gateway tools indicates an initially useful solution to the issue of accessibility for multicast conferencing. Further refinement in the set up and control of the tools and in session management issues such as session performance, and end user monitoring need to be examined in order to insure the usability of the tools. The further development and testing of the gateway tools will help to broaden the distribution of the tools beyond the scope of the Mbone and the current research community.
Future improvements in the usability of the tools also require further testing and refinement of the session recording and management tools such as SVDA and MMCR, and AOfwb/AOfrec. These tool in particular offer distinct advantages for educational situation, which would be better exploitable by a broader distribution on other platforms. Further development of these tools should ideally be undertaken in order to improve the recording set up and retrieval features in an educational context.
In general it would be desirable to undertake the further development of the toolkit within the scope of a series of specific dedicated sessions for testing and developing the session management and gateway access tools developed within the MECCANO project. Particularly, continued development of the tools would be best suited if this were undertaken in a systematic method to implement recording of sessions, and the delivery of sessions over both unicast and multicast networks. While this generally should be undertaken across a range of sessions types results from the validation projects undertaken within MECCANO and the work undertaken in the PIPVIC projects have indicated that particularly useful information can be gathered, and development enhanced within the context of educational based session.
While Real time multimedia conferencing solutions such as the MECCANO tools cover a range of applications, educational scenarios in particular are capable of providing the widest range of users in terms of experience. In addition the typically diverse range of platforms available in educational institutions provide a broad test bed for testing and development of the tools. Additionally educational scenarios have proven to be the most easily monitored and the most consistent for obtaining user feedback. Continued development of the tools in this area will allow for a broader distribution of the tools and most importantly provide continued input and incentive in the development of distance education and collaboration systems.
ACC: Radoslaw Ruchala
UCL: Roy Bennett, Angel Sasse
University of Freiburg: Jochen Lienhard
[bennett] Ed. Bennett, R, “The support provided to validation projects during the year 1 (June 1998 – May 1999)”, MECCANO deliverable R9.1. Available from:
[bringsrud] Bringsrud, K Å, “Description of , and conclusions from, the seminars carried out in the first year of MECCANO”, MECCANO deliverable R8.1. Available from:
[geyer] Geyer, W & Effelsberg, W: "The Digital Lecture Board - A Teaching and Learning Tool for Remote Instructions in Higher Education", In: Proc. of EDMEDIA'98, Freiburg, Germany, June 1998.
[kirstein] Kirstein, P T, Bormann, C, Ott, J, Perkins, C, Turletti, T, Cicic, T: “The MECCANO Internet Multimedia Conferencing Architecture”, MECCANO deliverable R3.1. Available from:
[lambrinos] Lambrinos, L et al., “The Multicast Multimedia Conference Recorder”, in Proceedings of the Seventh International Conference on Computer Communications and Networks, 12-15 October, Lafayette, Louisiana, USA.
[lienhard] Lienhard, J & Maass, G, "AOFwb - a new Alternative for the Mbone Whiteboard wb", Proceedings of ED-Media '98, Freiburg, June 1998
[oria] Loic Oria, “Approaches to Multicast over Firewalls: an Analysis”, Hewlett-Packard Laboratories, Bristol, England, August 1999. Available from http://www-mice.cs.ucl.ac.uk/multimedia/projects/meccano/
[mecdel] MECCANO deliverables web site:
[mecpol] ACC MECCANO web site: http://www.cs.agh.edu.pl/Meccano/
[patrick] Patrick, A, "User-Centred an Mbone Videoconference Polling Tool", March 11, 1998. http://debra.dgbt.doc.ca/mbone/mpoll/development/
MPoll application at http://www.merci.crc.doc.ca/mbone/mpoll/
[perkins] RTP Quality Matrix (RQM)
[pipvic] Piloting IP Video Conferencing, a UKERNA funded project in the UK.
[relate] Remote Language Teaching (ReLaTe) a BT-funded project.
[rtsp] H. Schulzrinne, H et al., "Real Time Streaming Protocol (RTSP)", IETF RFC 2326, April 1998.
[sasse] Sasse, A, Chesterfield, J, Watson, A “Final Report on the PIPVIC-2 Project”, see: http;//www-mice.cs.ucl.ac.uk/multimedia/projects/pipvic2/
[shrimp] SHRink-Wrapping Internet Multicast Packages (SHRIMP)
[ukerna] United Kingdom Educational and Research Networking Association http://www.ukerna.ac.uk/ukerna.html
[viror] The virtual university project (VIROR) http://www.viror.de/
Barua, Anita, Chellapa, Ramnath and Whinston, Andrew B., Social Computing, Computer Supported Co-operative work and Groupware. In Gavriel Salvendy, Ed., Handbook of Human Factors and Ergonomics. New York, John Wiley and Sons.
Blatner, Meera. Interactive Multimedia User Interfaces. In Borko Furht, Ed. Handbook of Internet and Multimedia Systems and Applications. Boca Raton, CRC Press.
Brock, John T., Computer Based Instruction. In Gavriel Salvendy, Ed., Handbook of Human Factors and Ergonomics. New York, John Wiley and Sons. 578-593
Bullinger, Hans Jorg, Bauer, Wilhelm and Braun, Martin, Virtual Environments. In Gavriel Salvendy, Ed., Handbook of Human Factors and Ergonomics. New York, John Wiley and Sons 1725-1759
Chignell, Mark H. And Waterworth, John A., Multimedia. In Gavriel Salvendy, Ed., Handbook of Human Factors and Ergonomics. New York, John Wiley and Sons
Hall, Wendy, white, Su, and Park Woolf, Bevelry, Interactive Systems for Learning and Teaching. In Borko Furht, Ed. Handbook of Internet and Multimedia Systems and Applications. Boca Raton, CRC Press.
Koubek, Richard J, Benysh, Susan A. H and Tang, Eric. Learning. In Gavriel Salvendy, Ed., Handbook of Human Factors and Ergonomics. New York, John Wiley and Sons. 130-149
Nielsen, Jakob, Usabiltiy testing, In Gavriel Salvendy, Ed., Handbook of Human Factors and Ergonomics. New York, John Wiley and Sons 1154-1568
Patrick, Andrew, S. The Human Factors of Mbone Videoconferences: Recommendations for Improving Sessions and Software. Ottawa, Communications Research Centre.
Preece, Jenny., Rogers, Yvonne., Sharp, Helen., Benyon, David., Holland, Simon., and Carey, Tom. Human Computer Interaction, Essex, Addison Wesley.
Proctor, Robert and Janet D. Sensations and Perception. In Gavriel Salvendy, Ed., Handbook of Human Factors and Ergonomics. New York, John Wiley and Sons. 43-88
Rada, R., Multimedia Education. In Borko Furht, Ed. Handbook of Internet and Multimedia Systems and Applications. Boca Raton, CRC Press.
Rada, R, and Michailidis Antonio Multimedia and virtual Organizations. In Borko Furht, Ed. Handbook of Internet and Multimedia Systems and Applications. Boca Raton, CRC Press.
Reaux, R and C, John M. Human Factors in Information Access of Distributed Systems. In Gavriel Salvendy, Ed., Handbook of Human Factors and Ergonomics. New York, John Wiley and Sons
Schooler, E M, Conferencing and Collaborative Computing. In Borko Furht, Ed. Handbook of Internet and Multimedia Systems and Applications. Boca Raton, CRC Press.
Shneiderman, Ben Designing the User Interface, Strategies for Effective Human Computer Interactions, (3rd ed.) Reading Massachusetts, Addison Wesley, 1998
UCL CS Multimedia group, Introduction to Multimedia Conferencing, Sept. 1998
Whelan, E, Secure Conferencing User Guide, UCL CS, Sept. 1998
Wickers, C D. and Carswell, M C: Information Processing. In Gavriel Salvendy, Ed., Handbook of Human Factors and Ergonomics. New York, John Wiley and Sons 89 –129
Software download: http://www-mice.cs.ucl.ac.uk/multimedia/software/
Supplementary information on the HP seminar, Oct.7th 1998: