Sociology course evaluation - first half of course.

The sociology course was transmitted from Essex to SSEES over a unicast link using RAT and VIC. At SSEES the audio was played out through speakers, and the VIC image (of the lecturer) was projected. The video frame rate was about 5 fps. A teaching assistant at SSEES displayed copies of the slides that the lecturer was showing, so the students had two visual sources of information - the lecturer and the slides. Initially these two sources were not displayed on the same wall, but this situation was soon rectified so at least the students didnít need to divide their visual attention between two directions.

The sociology course is running until the end of this term, but the results here were gathered from a questionnaire administered at the end of the 1998 winter term, after 6 lectures.

There were 20 respondents, between the ages of 18 and 26, 10 female and 10 male. All had attended at least 4 of the lectures.

The questionnaire addressed the following areas:

Listening effort

The students were asked how easy it was to hear and understand the remote lecturer on a scale of 1 (Very Easy) to 5 (Very Difficult). The mean result was 3.08, with a standard deviation of 0.69.

They were asked, if they experienced any difficulty, to describe the problems. 13 students made comments, most of which complained that the voice signal was occasionally 'crackly', and sometimes cut out. There were also a few comments that the volume was too quiet or 'muffled'.

All 20 respondents agreed that yes, it required more effort on their behalf to understand what the lecturer was saying than it might do in a normal lecture setting. Asked to explain why, 14 of the 17 respondents blamed the break-up of the audio signal, 2 commented that it is harder to concentrate when the lecturer is not in the same room (participation is not enabled), and 1 remarked that the local teaching assistant did not keep up with the slides at the same time as the remote lecturer, meaning that she had to talk over the lecturer to explain a point.

Audio quality

Asked whether they thought the quality of the audio varied a great deal throughout a lecture, 13 students said yes, 7 said no.

Asked to rate the quality of the audio in the lectures, where 1 represented Very Good and 5 represented Very Poor, the mean response was 3.22, with a standard deviation of 0.5

They were asked, if they found the quality less than acceptable, to explain why. 10 students made comments, all of which cite the 'crackly', unnatural nature of the sound and the way words can be missed through the signal cutting out.

The final audio question addressed the overall adequacy of the audio in addressing the needs of the lectures. The students were asked to place a mark on a vertical 1-100 scale where the top of the scale was labelled 'Audio met all the needs of the lecture', and the bottom was labelled 'Audio met none of the needs of the lecture'. The mean position was 59.79, the standard deviation 17.7.

Video use

Where 1 represents Never, and 5 represents All of the time, the students were asked to indicate how often they looked at the remote lecturer. The mean response was 3.68, the standard deviation 0.94.

They were asked whether they felt that they were missing certain non-verbal communication signals made by the remote lecturer e.g. gestures or facial expressions which would have been available face-to-face. 1 represented Never, and 5 represented All of the time. The mean response was 3.1 and the standard deviation was 1.07.

If they thought that they were missing these types of signals, they were asked to comment on its importance. 11 students responded, and opinion was varied: 4 felt it was very important, commenting that the lecturer/student relationship was impaired; the others indicated that it was kind of important, but not critical.

Asked whether they felt that the availability of a video image of the lecturer helped in following/understanding the explanations of the lecturer, 12 students said yes, 8 said no. The students that said no were asked to tick as many of a number of possible reasons for this as applied, and to add additional reasons. The choice 'I was too busy looking at the slides' was selected 7 times; 'I was too busy taking notes' 6 times; 'The image of the lecturer was too small' 2 times; 'The image of the lecturer was distracting' 2 times; and 'There was no additional information to be gained from looking at the lecturer' 2 times. Additional reasons provided were indifference, that the picture was not very clear and that the slides could not be read easily, so at least there was something to look at.


Video quality

13 of the students felt that the quality of the video did not vary greatly throughout a lecture, 7 thought that it did.

In rating the quality of the video in the lectures, where Very Good is indicated by 1 and Very Poor is indicated by 5, the mean was 3.08, and the standard deviation was 0.83.

The overall adequacy of the video for the purpose of the lectures was measured on a vertical line as in the audio section. The mean response was 55.9, the standard deviation 20.54.


Further issues

The students were asked to indicate how satisfied they were with this method of delivering lectures, where 1 represented Very Satisfied, and 5 represented Very Dissatisfied. The mean response was 3.13, the standard deviation 0.89.

17 students would have preferred face-to-face lectures, 3 would not.

Asked to explain why they would have preferred FTF lectures, 16 subjects cited a variety of reasons, including that the sound quality would be better, it would be easier to follow, it would be more personal, it would be better lit, and it would be easier to ask the lecturer if something is missed.

Asked for any additional comments about any technical aspects of the lectures, requests were made for the sound quality to be improved, and one for the video quality to be improved (and to be projected onto a screen rather than a wall). Many of the comments made were actually about the 'unfairness' of the set-up, in that Essex got the lecture FTF, and SSEES didn't. SSEES would have appreciated 'a shot' at the lecturer, FTF.