Diversity in Video Lectures: Aid or Hindrance?

International Review of Research in Open and Distributed Learning

Volume 20, Number 2

April – 2019

Mik Fanguy

, Jamie Costley

2,3*

, Matthew Baldwin

, Christopher Lange

, and Holly Wang

Korea Advanced Institute of Science and Technology (KAIST), Daejeon, South Korea,

Department for Management of Science

and Technology Development, Ton Duc Thang University, Ho Chi Minh City, Vietnam,

Faculty of Applied Sciences, Ton Duc

Thang University, Ho Chi Minh City, Vietnam,

Dankook University, Suji, South Korea

*corresponding author [email protected]

Abstract

Media diversity within video lectures has been shown to have an effect on students who participate in both

flipped classes as well as online courses. While some research claims that content delivered through

multiple sources leads to more learning, contrasting research makes the claim that too much media hinders

cognitive processing. The present study investigated the effects of varying levels of instructional media

delivered to students (n=110) within a flipped scientific writing course to investigate the relationship

between higher levels of media diversity and student performance. Results showed that more diversity led

to lower levels of performance. It was also found that higher levels of media diversity correlated with higher

levels of students’ scanning between different forms of media, possibly contributing to the lower levels of

performance. The implications of these results provide insight into the optimal level of media diversity, and

on student behavior that can affect learning.

Keywords: flipped learning, Korea, multimedia, scanning, summaries, video lectures

Diversity in Video Lectures: Aid or Hindrance?

Fanguy, Costley

Baldwin, Lange, and Wang

Introduction

Video lectures are a key component to most e-learning environments and the relationship between the

effective design of video lectures and other aspects of online environments needs to be examined in order

to optimize the effectiveness of online learning. One method instructors have used to improve e-learning

experiences is to create more diverse lecture videos that incorporate various types of media (Kim, Kwon, &

Cho, 2011; Zhang, Zhao, Zhou, & Nunamaker, 2004). The effects of such diversity on student engagement

and learning is a topic worth exploring. This can be done by examining how diversity in the presentation of

lecture videos affects germane load, which is widely accepted to contribute to learning through increased

comprehension of the course content that the videos present (Cierniak, Scheiter, & Gerjets, 2009; De Jong,

2010; Sweller, 2005; Sweller, van Merriënboer, & Paas, 1998).

Media diversity, which refers to the various audio and visual means of presenting information in online

video lectures, influences student perception of video lectures and affects their cognitive processing

(Kalyuga et al., 1998; Lowe, 1999; Mayer, 2014; Mayer & Moreno, 2003; Rasch & Schnotz, 2009; Schnotz

& Rasch, 2005; Sims & Hegarty, 1997; Sweller, 1999; Sweller et al., 1998; van Merriënboer, 1997). However,

this is still a contentious area, as some research suggests that diversity in media presentation increases

germane load, while other studies have shown that such diversity can hinder the development of germane

load. For instance, some studies claim that presenting the same information several times through diverse

forms of media enables students to improve their comprehension of the material (Paivio, 1991; Schmidt-

Weigand & Scheiter, 2011), while others claim that doing so causes a redundancy effect, leading to

unnecessary cognitive processing (Kalyuga, Chandler, & Sweller, 1999; Mayer & Moreno, 2003) and

decreased germane load (Sweller et al., 1998). Further to this, lecture design has been shown to influence

whether or not students finish watching a video in an online class

(Costley, Hughes, & Lange, 2017; Costley

& Lange, 2017a).

The present study quantitatively measures student recall of the content in video lectures that contain

varying degrees of media diversity. Additionally, the effect of media diversity on scanning between different

media sources is investigated, which may provide more insight on the results of this study. This research

differs from extant studies in the literature in that the videos examined herein contain the same lecture by

the same lecturer, but with differing levels of media diversity. Tailoring the media diversity in each

experimental group while maintaining the content being presented offers a clearer insight into the effects

of media diversity in online lecture videos.

Literature Review

The Effects of Visual Diversity on Retention

Studies indicate that the use of visual media is beneficial for student understanding. Images and animations

have been shown to facilitate the learning process so students can better comprehend the content (Salomon,

1994; Sweller & Chandler, 1994; van Gog, Ericsson, Rikers, & Paas, 2005). This improved understanding

leads to better retention of information and quiz scores. The use of animated text was also shown to benefit

the learning process; Luzón and Letón (2015) found that the addition of handwritten animated text to

Diversity in Video Lectures: Aid or Hindrance?

Fanguy, Costley

Baldwin, Lange, and Wang

lecture videos was more helpful to the students’ learning compared to cases when such text was not

included. Chen and Wu (2015) point to Mayer’s (2001) cognitive theory of multimedia learning and its claim

that visual modalities or animations with verbal explanations have the edge over either text or narration in

performance on retention tests.

Videos that show the instructor have also been shown to be more effective. For example, Day, Foley, and

Catrambone (2006) found that videos where the instructor was shown led to higher retention of

information and greater understanding and ability to apply the principles featured in the lecture compared

to the same content presented using either audio and a slide deck created using Microsoft PowerPoint (PPT)

or a PPT slide deck with transcription text. The amount of such recall improves depending on the form in

which the instructor is presented (Li, Kizilcec, Bailenson, & Ju, 2016). For example, Pi, Hong, and Yang

(2017) state that more knowledge is achieved when the image of the instructor is small – in their study,

defined as “8.4% of the space of the video lecture” (p. 347).

Some research suggests that instructors should use caution, particularly under certain conditions, when

adding visual media such as animation or simulated pictures. While visuals are widely acknowledged to

have a facilitating effect (Lowe, 1999; Rasch & Schnotz, 2009; Salomon, 1994; Schnotz & Rasch, 2005; Sims

& Hegarty, 1997; Sweller & Chandler, 1994; van Gog et al., 2005), some studies have shown that the

inclusion of visuals may not always be beneficial to learning (Rasch & Schnotz, 2009; Schnotz & Rasch,

2005). For instance, it has been stated that the use of images and animations often leads to superfluous

cognitive processing for students who are able to comprehend the content without the use of such visuals

(Rasch & Schnotz, 2009; Schnotz & Rasch, 2005). Schnotz and Rasch (2005) point out that the use of such

visuals can reduce germane load for learners who do not require them to comprehend the information

because of unnecessary mental processing.

Abrupt changes within the lecture video can also be problematic, as some have suggested that sudden

transitions or scene changes be avoided when creating online lectures (Fanguy, Costley, & Baldwin, 2017;

Kim et al., 2014). Other studies have cautioned against the improper use of videos that show the instructor.

While Kizilcec, Bailenson, and Gomez (2015) maintain that there are advantages to showing the professor

speaking in a lecture video and that students prefer this presence (Kizilcec, Papadopoulos, &

Sritanyaratana, 2014), the former state that continuous use of this type of video may actually lead to

cognitive overload, as student are forced to over-rely on their working memory while focusing on the

instructor, particularly when the instructor directs their attention to particular points in the lecture.

Visual Diversity and Germane Load

Cognitive load theory can provide a framework with which to understand multimedia instruction and its

effects on students. According to cognitive load theory, when instruction is unnecessarily complicated or

confusing, students may experience extraneous cognitive load, which can impede learning (Leppink, Paas,

van der Vleuten, van Gog, & van Merriënboer, 2013; Sweller et al., 1998). Extraneous load can be defined

as the amount of cognitive effort required by ineffective instruction that does not help to achieve the

learning objective (De Jong, 2010). A key concern when designing multimedia instruction is cognitive

overload, which occurs when a learner engages in cognitive processing that exceeds their useable cognitive

capacity (Mayer & Moreno, 2003). Ideally, instruction should be designed to increase levels of germane

Diversity in Video Lectures: Aid or Hindrance?

Fanguy, Costley

Baldwin, Lange, and Wang

cognitive load in students (Kolfschoten, Lukosch, Verbraeck, Valentin, & Vreede, 2010; Sweller et al., 1998).

Germane load contributes to learning directly and reflects the learner’s attempt to construct schema to

improve comprehension of relevant information. Furthermore, germane load has been shown to strongly

influence a student’s likelihood to maintain focus within a learning environment (Sweller et al., 1998). To

increase the level of germane load, extraneous load must be reduced so that a greater portion of the learner’s

available cognitive capacity can be devoted to mental processes relevant to the learning task (Cierniak et

al., 2009; Leppink et al., 2013; Schmeck, Opfermann, van Gog, Paas, & Leutner, 2015). To do so, instruction

should be presented in a format that can be easily understood by learners (Bruner, 2009).

Diversity in the presentation of media influences student perception of video lectures and affects their

cognitive processing (Kalyuga et al., 1998; Mayer & Moreno, 2003; Lowe, 1999; Mayer, 2014; Sims &

Hegarty, 1997; Rasch & Schnotz, 2009; Schnotz & Rasch, 2005; Sweller, 1999; Sweller et al., 1998; van

Merriënboer, 1997). A number of empirical studies add support to the theoretical claims that diverse

presentation of visual media increases germane load by enabling greater comprehension of the information

being presented. Day et al. (2006) found that more audio and visual diversity in lessons led to increased

levels of understanding and recall, as shown in post-test retention scores and indicated levels of

comprehension. A study by Kim et al. (2011) indicated that perceived learning increased when students

were exposed various integrated media such as images, graphics, audio, and video clips. Other studies claim

that presenting the same information several times through diverse forms of media enables students to

improve their comprehension of the material (Paivio, 1991; Schmidt-Weigand & Scheiter, 2011). Zhang,

Zhou, Briggs, and Nunamaker (2006) provide empirical evidence that germane load increased with the

total diversity of media. The results of the study showed that learners who experienced both auditory and

visual delivery (PPT slides and video with audio) achieved improved learning outcomes compared to those

who received only visual delivery (PPT slides and lecture notes). Cheon and Grant (2012) found that a

metaphorical interface containing pictorial form as well as text can enhance germane load and positively

affect learning, while Costley and Lange (2017b) found that overall, there was a positive relationship

between diversity of media used in lectures and germane load. These results support the idea that total

media diversity helps to increase student levels of germane load with regard to information that is presented

in e-learning lectures.

Other research suggests that instructors should use caution, particularly under certain conditions, when

adding visual media such as animation or simulated pictures. While visuals are widely acknowledged to

have a facilitating effect (Lowe, 1999; Sims & Hegarty, 1997; Schnotz & Rasch, 2005; Rasch & Schnotz,

2009; Salomon, 1994; Sweller & Chandler, 1994; van Gog et al., 2005), some studies have shown that the

facilitating effect may not always increase levels of germane load (Schnotz & Rasch, 2005; Rasch & Schnotz,

2009). For instance, it has been stated that the use of images and animations often leads to superfluous

cognitive processing for students who are able to comprehend the content without the use of such visuals

(Rasch & Schnotz, 2009; Schnotz & Rasch, 2005). Other studies claim that repeating the same information

through several forms of media causes a redundancy effect, leading to unnecessary cognitive processing

(Kalyuga et al., 1999; Mayer & Moreno, 2003) and decreased germane load (Sweller et al., 1998).

Visual Diversity and the Split-Attention Effect

When viewing lecture videos, learners may engage in a number of lecture behaviors such as pausing the

Diversity in Video Lectures: Aid or Hindrance?

Fanguy, Costley

Baldwin, Lange, and Wang

video, rewinding and rewatching, skipping ahead, increasing video playing speed, averting one’s eyes from

screen for more careful listening, scanning one’s eyes between text and images, and temporarily turning off

the sound in order to focus on a visual or text. While these lecture behaviors may seem advantageous, as

they enable learners to control the pace and flow of information (Schwan & Riempp, 2004), such forms of

control may also impede comprehension. For example, Caspi, Gorsky, and Privman (2005) show that even

brief pauses during the viewing of an instructional video disrupted the context of the lecture. In addition,

splitting attention between media sources in an instructional video may increase extraneous cognitive

processing (Kizilcec, Bailenson, & Gomez, 2015; Mayer & Moreno, 2003). A physical manifestation of

attention splitting is scanning one’s eyes back and forth between two types of media (e.g., visuals and text).

Research suggests that multimedia instruction is more effective when it contains change-of-pace elements

and a variety of visual media (Barker & Benest, 1996; Brecht, 2012). However, instructors must be careful

in how they present visual media to avoid overloading the visual channel, which can invoke the split-

attention effect. The split-attention effect occurs when learners are required to divide their focus among

several sources of media in order to understand the learning material (Ayres & Sweller, 2005; Mayer &

Moreno, 1998; Sorden, 2005). This splitting of learner attention represents an increase in extraneous load,

which impedes learning. For example, Chen and Wu (2015) found that instructional videos that showed

lecture slides and the instructor’s face in separate windows on the screen caused the split-attention effect,

as students mentioned in follow-up interviews that they felt burdened by the need to scan their eyes between

the two windows. However, Chen and Wu (2015) did not report a split-attention effect for participants who

were shown videos that pictured the lecturer and slides in the same video window. These findings suggest

that physical separation of visual content may lead to the split-attention effect, as viewers need to scan their

eyes back and forth between media content, which in turn may increase extraneous load.

The Present Study

Previous research has already examined the relationship between media diversity and its effectiveness in

learning (Mayer & Moreno, 2003; Mayer 2014; Rasch & Schnotz, 2009; Schnotz & Rasch, 2005; Sweller et

al., 1998; van Merriënboer, 1997). However, past studies are inconsistent in findings regarding the diversity

of media in lecture videos. Some claim that the diversity actually has a negative effect on germane load at

particular parts of the lecture (Chandler & Sweller, 1991; Kizilcec et al., 2015; Mayer & Moreno, 2003; Rasch

& Schnotz, 2009; Schnotz & Rasch, 2005; Sweller et al., 1998). However, Costley and Lange (2017b) found

that overall, there was a positive relationship between diversity of media used in lectures and germane load.

Therefore, it is worthwhile to investigate whether the increase in specific types of media diversity leads to

better performance in students, as represented by germane load and quiz scores. Additionally, it would be

useful to investigate whether varying levels of media diversity affects levels of students’ scanning between

different media sources, which may provide more insight into the effects on performance. In particular, this

paper examines the effects of four types of media use included with talking-head presentations: on-screen

text, visuals (i.e., photographs, figures, and tables), the instructor’s handwriting on the screen (also known

as “Khan-style”), and summaries given by a guest lecturer. The findings of this study will be useful to

instructors in e-learning who want to understand whether specific types of media aid students in learning.

Research Hypotheses

The present study will test the following research hypotheses:

Diversity in Video Lectures: Aid or Hindrance?

Fanguy, Costley

Baldwin, Lange, and Wang

H1: Students in experimental conditions with higher levels of diversity will have higher quiz scores.

H2: Students in experimental conditions with higher levels of diversity will have higher levels of

germane load.

H3: Students in experimental conditions with higher levels of diversity will exhibit more scanning

behavior.

Methods

Experimental Procedures

In the present study, our goal was to assess how diversity in media presentation in online lecture videos

would affect student perceptions and recall in the graduate-level course Scientific Writing (CC500) at the

Korea Advanced Institute of Science and Technology (KAIST) in Daejeon, South Korea. The course was

taught in an “inverted” or “flipped” format; students were required to watch lecture videos and take online

quizzes for homework while also meeting with professors and fellow classmates once per week, face-to-face,

in a brick-and-mortar classroom. In the present study, we examine five of the 56 total videos that constitute

the lecture content of the course. The five videos included in this study were reproduced in four different

styles with regard to media diversity, with each style corresponding to one of four treatment groups. In

Treatment Group 1, students were provided a video series that was prepared with an instructor delivering

a lecture in front of PPT slides featuring a simple black background with white text on the screen; Treatment

Group 2 received the same type of lectures but also included the addition of visuals, such as photographs,

figures, and tables; Treatment Group 3 also received the same type of lecture as in Treatment Group 2, but

this time with the addition of instances of Khan-style writing with a pen on a glass panel in front of the

instructor; and finally, for Treatment Group 4, the same type of lecture was given as in Group 3, but each

video contained mid and final summaries that were delivered by a “guest” instructor who was also seated

in a “coffee shop” environment projected in the background rather than standing in front of PowerPoint

slide contents. Screenshots of the aforementioned Treatment Groups can be seen in Figures 1-4 below.

Figure 1. A screenshot of a Treatment Group 1 video featuring a lecturer presenting in front of a slide

background showing only text and no other visuals (low diversity).

Diversity in Video Lectures: Aid or Hindrance?

Fanguy, Costley

Baldwin, Lange, and Wang

Figure 2. (Left) A still from a Treatment Group 2 video lecture (medium low diversity). (Right) A second

still from the same Treatment 2 video featuring an addition visual not shown in Treatment 1.

Figure 3. (Left) A still from the Treatment Group 3 video lecture (medium high diversity) that features the

same text on the slide background as in the Treatment 1 and 2 versions along with Khan-style writing by

the instructor. (Right) The Treatment Group 3 version of the video lecture includes the same visual shown

in the Treatment 2 version.

Diversity in Video Lectures: Aid or Hindrance?

Fanguy, Costley

Baldwin, Lange, and Wang

Figure 4. (Upper left) Treatment Group 4 (high diversity) features the same background, Khan-style

writing, and (Upper right) visual as in Treatment Group 3. (Lower left) An additional still of a guest lecturer

summary included in the Treatment Group 4 version but not in other treatment versions of the video.

A total of 110 students were divided up into four treatment groups, with between 20 and 31 students in each

group. The respective videos were posted on the school’s learning management system in Week 4 and were

available to watch at the students’ leisure. Once the videos were viewed, students took a multiple-choice

quiz online, which they could access at any time during the seven-day video viewing period. Students were

required to complete this test before the next face-to-face, brick-and-mortar classroom meeting day. The

quiz was used to measure the students’ comprehension and recall of the contents from the videos. During

the respective brick-and-mortar classroom meeting, students were asked to fill out a survey that involved a

10-point Likert-type scale to assess the videos. Survey forms were assessed, and quiz data was taken from

the online learning management system for the course.

Participants and Context

The present research was conducted at KAIST, a large university located in Daejeon, South Korea. The

majority of students at KAIST specialize in STEM fields. As of 2013, the student population of KAIST was

11,175, with 60% of students exclusively at the graduate level (Korea Advanced Institute of Science and

Technology [KAIST], 2014b). Most of the students are male (80%), with international students comprising

5% the total student population (KAIST, 2014b). Nearly all courses at KAIST are conducted in English,

although the vast majority of KAIST students are non-native English speakers. KAIST provides a variety of

online and blended courses, including Massive Open Online Courses (MOOCs) provided through Coursera,

institution-level online courses through the CyberKAIST program, as well as the Bridge-Program for

prospective freshmen, and global- and institutional-level flipped courses through iPodia and Education 3.0,

respectively (Korea Advanced Institute of Science and Technology [KAIST], 2014a).

Diversity in Video Lectures: Aid or Hindrance?

Fanguy, Costley

Baldwin, Lange, and Wang

Participants in the present study were enrolled in classes that were offered as part of KAIST’s Education 3.0

program. The Education 3.0 initiative was started in 2012 with the aims of reducing the amount of

traditional lecturing in KAIST courses and enabling students to participate in more communicative and

interactive learning activities through a flipped classroom environment (Horn, 2014). In 2014, a total of 5%

of all classes were given in Education 3.0 format at KAIST, with an objective of increasing the percentage

to 30% by 2018 (Horn, 2014).

As a requirement for graduation, graduate students at KAIST must be able to compose articles that are

publishable in scientific journals. To assist them in this goal, KAIST offers Scientific Writing (CC500), a

course that teaches students how to communicate their research in English through writing. The course is

given in English, and enrollment is generally capped at 20 student per class. In the present study, eight

sections of Scientific Writing were included in the experiment. Of the potential 135 total participants, 25

were removed from the study due to not completing all the requirements for inclusion in the study. This left

a total of 110 valid responses, of which 29 were female and 81 were male. The oldest participant was 45, and

the youngest was 22, with a mean age of 27. The eight selected sections of Scientific Writing were taught as

part of KAIST’s Education 3.0 program, and as such, were delivered in a flipped format.

Measures

Student took a quiz that consisted of 20 questions and covered topics given in four video lectures. The quiz

was available online, and students could take it at any time during the one-week video viewing period (week

5 of the course). The quiz consisted of multiple-choice questions, with some of the items permitting only

one answer choice and others allowing one or more possible answer choices. For the latter, partial credit

was given when a correct option was chosen, but no credit was awarded when an incorrect answer option

was chosen. All quiz items were written by the course instructors and were aimed at making the students

demonstrate that they were able to apply the concepts covered in the lecture videos. For example, if a

student viewed a video about how to integrate numbers into their writing, they would see a multiple-choice

question asking them to evaluate whether particular instances of writing with numbers were correct. The

quiz was worth 5% of the total course grade.

To take a look at the germane load of these students, four items were selected from Leppink et al.’s (2013)

research on measuring cognitive load. Specifically, the four items chosen in this study were:

1. The lecture really enhanced my understanding of the topic.

2. The lecture really enhanced my knowledge and understanding of the of the class subject.

3. The lecture really enhanced my understanding of the concepts associated with the class subject.

4. The lecture really enhanced my understanding of concepts and definitions.

However, this research uses slightly different wording than Leppink et al.’s (2013) –this study uses the word

“lecture” instead of the original “activity”– in order to focus the items more specifically on the video lectures

as they were used. The Cronbach’s alpha for the germane load construct was .926 and the Cronbach’s alpha

for the extraneous load construct was .919, which is acceptable for this type of research. To further

Diversity in Video Lectures: Aid or Hindrance?

Fanguy, Costley

Baldwin, Lange, and Wang

understand student behavior while they were watching the videos, the survey described above included one

Likert-type item to respond to: “I had to scan my eyes back and forth between the text and the

graphs/images in the videos I watched.” This item as with the germane load construct was scored between

1 and 10.

Results

To get an overall picture of the main variables used in this study and the relationships between them,

descriptive statistics were calculated and the variables were correlated with each other (Table 1). There were

four different categories for the experimental conditions, while scanning and germane load ranged

between 1 and 10. Scanning had a mean of 4.62, and germane load had a mean of 7.52. The quiz score

variable had a range of 1.76 to 10, with a mean score of 7.43. The experiment condition variable was

positively correlated with scanning (.250), but negatively correlated with quiz score (-.295) and germane

load (-.231). All of the relationships between the experimental condition and the other variables were

statistically significant. Quiz score also had a negative relationship with scanning (-.257), which was

statistically significant, though quiz score did not have a statistically significant relationship with germane

load (-.018).

Table 1

Descriptive Statistics and Correlations Between the Main Variables

Min

Max

Mean

Experiment

condition

Scan

Quiz

score

Germane

load

Experiment condition

110

Scan

110

4.62

2.5

.250

Quiz score

110

1.76

7.43

.90

-.295

-.257

Germane load

110

7.52

1.9

-.231

-.149

-.018

Note. * p = < .05; ** p = <.01

The correlations between the experimental condition and dependent variables give some insight into the

effect of the levels of diversity on scanning, quiz score, and germane load. As can be seen in Table 2, for

scanning, the low diversity group had the lowest mean (4.06), followed by medium low diversity (4.13),

medium high diversity (4.38), with high diversity having the highest mean scanning score (5.90). However,

in respect of quiz score and germane load, high diversity had the lowest mean (6.79), followed by medium

high diversity (7.23), then medium low diversity (7.54), with low diversity having the highest levels of quiz

score (8.40). Germane load followed a similar pattern to quiz score though low diversity and medium low

diversity were similar (7.95, 7.96), followed by medium high diversity (7.39), with high diversity having the

lowest levels of germane load.

Diversity in Video Lectures: Aid or Hindrance?

Fanguy, Costley

Baldwin, Lange, and Wang

Table 2

Main Variable Means by Experiment Condition

Scanning

Quiz

score

Germane

load

Low diversity

4.06

8.40

7.95

Medium low diversity

4.13

7.54

7.96

Medium high diversity

4.38

7.23

7.39

High diversity

5.90

6.79

The data from Table 2, is visually represented below in Figure 5. This shows the trends discussed in the

preceding paragraph, and shown in Table 2. As the levels of diversity increase from low, to medium low, to

medium high, and finally to high, both germane load and quiz score decrease, and scanning increases.

Figure 5. Main variable means by experiment condition.

To establish if the differences in relationships seen in Table 2 are statistically significant, one-way ANOVA

was used. As can be seen in Table 3, the between-group difference for both scanning (p = .02) and quiz

score (p = .02) were statistically significant. However, the between-group differences for germane load were

not statistically significant (p = .09).

Diversity in Video Lectures: Aid or Hindrance?

Fanguy, Costley

Baldwin, Lange, and Wang

Table 3

One-Way ANOVA Results for the Main Variables Used in this Study

Sum of

squares

Mean square

Sig.

Scanning

Between groups

60.616

20.205

3.279

.024

Within groups

653.166

6.162

Total

713.782

Quiz score

Between groups

8.087

2.696

3.593

.016

Within groups

79.528

.750

Total

87.615

Germane load

Between groups

24.880

8.293

2.269

.085

Within groups

387.501

3.656

Total

412.382

Discussion

The results of the present study show that as diversity increased from the lowest to the highest level, the

rate of students scanning their eyes between contents on the screen increased, while germane load and

recall both decreased. Contrary to Hypothesis 1, students’ quiz scores showed an inverse relationship with

the amount of diversity of media within the lecture videos. These findings are unlike those of Costley and

Lange (2017b), who found that lecture videos containing a wider variety of media resulted in students

recalling more of the instructional content. However, their study relied upon self-reported levels of recall

rather than qualitative analysis, so it is uncertain whether these perceived levels of retention would have

been demonstrated in actual test scores. At a glance, the findings of the present study seem to contradict

those of Day et al. (2006), who found that increased diversity in lessons resulted in increased

comprehension and recall in post-test retention scores and self-reported levels of understanding. However,

in Day et al.’s study, there was only one experimental condition (out of a total of four) in which the video

showed a lecturer speaking on screen with PPT slides, while all four experimental conditions in the present

study contained these features. As Day et al. do not mention Khan-style on-screen handwriting or

summaries, it is likely that the most diverse “Video + Audio + PPT” experimental condition in Day et al.’s

study was most similar to the low and mid-low diversity conditions in our own study, negating the apparent

contradiction.

Likewise, germane load was relatively higher in the low and mid-low diversity groups, at 7.95 and 7.96,

respectively, than in the mid-high and high diversity groups, at 7.39 and 6.79, respectively, contrary to our

expectation indicated in Hypothesis 2. The present results contradict those of a number of studies that have

shown that diversity in lecture videos is beneficial to student learning. For example, the results of Zhang et

al., (2006) showed that the learning process is enhanced for learners exposed to both auditory and visual

delivery (PPT slides and video with audio) compared to those who viewed only visual delivery (PPT slides

and lecture notes). However, this apparent contradiction with the present results can be explained when we

consider that the low diversity condition of the present study was already more diverse than the “PPT slide

+ video and narration condition” since the former showed the instructor speaking while the latter did not.

Featuring the instructor in the low diversity condition of the present study is likely to have benefitted

student learning, as Kizilcec, Papadopoulos, and Sritanyaratana (2014) indicated the importance of seeing

the instructor in such videos, since the instructor can provide social cues and emphasis through gestures

Diversity in Video Lectures: Aid or Hindrance?

Fanguy, Costley

Baldwin, Lange, and Wang

and body language. Taking this point into account along with the results of the present study, it may be that

online lessons that contain both audio and video represent the lower bound of media diversity for effective

online lessons that lead to increased germane load and recall. Other studies have suggested that lecture

videos were more effective when containing “strong presentation of relief and change-of-pace elements,”

including changes in the video or audio stimulus, modifications to the background or setting, or dramatic

transitions between various parts of the video (Brecht, 2012). Barker and Benest (1996) suggested that the

inclusion of multimedia contents is beneficial since doing so may prevent students from losing focus.

Although such studies indicate that it is beneficial to increase media diversity, they do not identify an upper

bound on the level at which diversity is helpful in achieving desirable learning outcomes and improved

recall. The results of the present study offer insight into where that upper bound might lie: the most effective

experimental condition involved an instructor speaking in front of PPT slides containing text only. In the

present study, the addition of images, figures, tables, Khan-style writing, and summaries given by a guest

lecturer yielded diminishing returns in terms of germane load and quiz scores. Students in the high diversity

experimental condition exhibited the lowest germane load and quiz scores and the highest rate of eye

scanning behavior by a significant margin. In addition to the high level of media diversity in this group of

videos, the mid- and end-point summaries may have induced the redundancy effect (Sweller et al., 1998),

as students were exposed to the same information more than once in a different manner. This redundancy

may have led to additional cognitive processing and reduced germane load.

In the most diverse conditions in the present study, students increasingly scanned their eyes back and forth

between contents, which was in agreement with our hypothesis. As noted by Sweller et al. (1998), perhaps

the split-attention effect that causes this scanning behavior is a sign of cognitive overload, as students were

unable to fix their gaze and attention on just one part of the screen while viewing the lecture. As scanning

increased with the amount of diversity in the videos, it is plausible that students found the diversity in the

videos to overwhelming and unhelpful. This result may help to explain why germane load and quiz scores

decreased as media diversity was increased.

Conclusion

As online lecture videos are a major component of flipped and online courses, it is important to consider

the most effective ways to deliver content through this medium. Lecture videos should deliver the course

contents in a meaningful, memorable, and engaging way that adds to the learning experience of the course.

The present results are important in that they contradict the findings of a number of studies that suggest

that diversity of media is beneficial in online lecture videos. In the present study, increased diversity led to

decreased germane load and recall, and increased instances of students scanning their eyes back and forth

between contents on the screen. Based on these results, it seems reasonable to conclude that there is an

upper bound to how much diversity is useful and productive in online lecture videos and that above this

threshold, diversity may become a detriment to student attention and learning. The present results lead one

to consider whether there is a “Goldilocks zone” in the diversity of media in online lecture videos, where

content is presented in an engaging way that is beneficial to learning without creating extraneous load or

the redundancy effect. Considering that Day et al. (2016) found decreased student performance on recall

tests with videos containing relatively less media diversity than our low diversity condition, and that our

Diversity in Video Lectures: Aid or Hindrance?

Fanguy, Costley

Baldwin, Lange, and Wang

low diversity condition led to the highest quiz scores and reported levels of germane load, a possible

conclusion is that the most effective videos will contain a level of media diversity similar to the low or low-

mid diversity conditions, where an instructor speaks in front of slides containing text and possibly a few

visuals such as images, figures, and tables. Such findings may be useful to instructors who are designing

lecture videos for MOOCs, flipped, or any other type of e-learning environment.

The present study provides a starting point in answering the question, “How much diversity of media is too

much in online lecture videos?” The fact that the highest level of diversity examined in this study caused a

significant decrease in student germane load and recall, and a significant increase in scanning behavior,

should give course designers pause when considering how much media diversity to include in a lecture

video. The present work provides a valuable counterexample to numerous studies in the literature that

suggest that increased media diversity is beneficial to student learning. In this regard, moderation in media

diversity may be a useful guiding principle.

Diversity in Video Lectures: Aid or Hindrance?

Fanguy, Costley

Baldwin, Lange, and Wang

References

Ayres, P., & Sweller, J. (2014). The split-attention principle in multimedia learning. In Mayer, R.E.(Ed.),

The Cambridge handbook of multimedia learning (pp. 206-226). Cambridge University Press.

https://doi.org/10.1017/CBO9781139547369.011

Barker, P. G., & Benest, I. D. (1996). The on-line lecture concept-a comparison of two approaches. IEE

Colloquium on Learning at a Distance: Developments in Media Technologies (pp.9/1-9/7).

London, UK: IET. https://doi.org/10.1049/ic:19960882

Brecht, H. D. (2012). Learning from online video lectures. Journal of Information Technology Education:

Innovations in Practice, 11, 227-250. https://doi.org/10.28945/1712

Bruner, J. S. (2009). The process of education. Cambridge: Harvard University Press.

Caspi, A., Gorsky, P., & Privman, M. (2005). Viewing comprehension: Students’ learning preferences and

strategies when studying from video. Instructional Science, 33(1), 31-47.

https://doi.org/10.1007/s11251-004-2576-x

Chandler, P., & Sweller J. (1991). Cognitive load theory and the format of instruction. Cognition and

Instruction, 8(4), 293-332. https://doi.org/10.1207/s1532690xci0804_2

Chen, C. M., & Wu, C. H. (2015). Effects of different video lecture types on sustained attention, emotion,

cognitive load, and learning performance. Computers & Education, 80, 108-121.

https://doi.org/10.1016/j.compedu.2014.08.015

Cheon, J., & Grant, M.G. (2012). The effects of metaphorical interface on germane cognitive load in web-

based instruction. Educational Technology Research and Development, 60(3), 399-420.

https://doi.org/10.1007/s11423-012-9236-7

Cierniak, G., Scheiter, K., & Gerjets, P. (2009). Explaining the split-attention effect: Is the reduction of

extraneous cognitive load accompanied by an increase in germane cognitive load? Computers in

Human Behavior, 25(2), 315–324. https://doi.org/10.1016/j.chb.2008.12.020

Costley, J., Hughes, C., & Lange, C. (2017). The effects of instructional design student engagement with

video lecture at cyber universities. Journal of Information Technology Education: Research 16,

189 - 207. https://doi.org/10.28945/3728

Costley, J., & Lange, C. (2017a). Video lectures in e-learning: effects of viewership and media diversity on

learning, satisfaction, engagement, interest, and future behavioral intention. Interactive

Technology and Smart Education 14(1), 14-30. https://doi.org/10.1108/ITSE-08-2016-0025

Costley, J., & Lange, C. (2017b). The effects of lecture diversity on germane load. International Review of

Research in Open and Distributed Learning 18(2). https://doi.org/10.19173/irrodl.v18i2.2860

Day, J. A., Foley, J. D., & Catrambone, R. (2006). Investigating multimedia learning with web lectures

Diversity in Video Lectures: Aid or Hindrance?

Fanguy, Costley

Baldwin, Lange, and Wang

(GVU Technical Report; GIT-GVU-06-25). doi: http://hdl.handle.net/1853/13141

De Jong, T. (2010). Cognitive load theory, educational research, and instructional design: some food for

thought. Instructional Science, 38(2), 105-134. https://doi.org/10.1007/s11251-009-9110-0

Fanguy, M., Costley, J., & Baldwin, M. (2017). Pinch hitter: The effectiveness of content summaries

delivered by a guest lecturer in online course videos. International Review of Research in Open

and Distributed Learning, 18(7). https://doi.org/10.19173/irrodl.v18i7.3208

Horn, M. (2014, March 17). KAIST doesn't wait for change in Korea, Pioneers 'Education 3.0' (Blog post).

Forbes. Retrieved from

http://www.forbes.com/sites/michaelhorn/2014/03/17/kaist-doesnt-

wait-for-change-in-korea-pioneers-education-3-0/#5ae890b01a06

Kalyuga, S., Chandler, P., & Sweller, J. (1998). Levels of expertise and instructional design. Human

Factors: The Journal of Human Factors and Ergonomics Society, 40(1), 1-17.

https://doi.org/10.1518/001872098779480587

Kalyuga, S., Chandler, P., & Sweller, J. (1999). Managing split-attention and redundancy in multimedia

instruction. Applied Cognitive Psychology, 13(4), 351-371. https://doi.org/10.1002/(sici)1099-

0720(199908)13:4%3C351::aid-acp589%3E3.0.co;2-6

Kim, J., Guo, P. J., Seaton, D. T., Mitros, P., Gajos, K. Z., & Miller, R. C. (2014). Understanding in-video

dropouts and interaction peaks in online lecture videos. In Proceedings of the First ACM

Conference on Learning@ Scale Conference. (pp. 31-40). New York, NY: ACM.

https://doi.org/10.1145/2556325.2566237

Kim, J., Kwon, Y., & Cho, D. (2011). Investigating factors that influence social presence and learning

outcomes in distance higher education. Computers & Education, 57(2), 1512–1520.

https://doi.org/10.1016/j.compedu.2011.02.005

Kizilcec, R. F., Bailenson, J. N., & Gomez, C. J. (2015). The instructor’s face in video instruction: Evidence

from two large-scale field studies. Journal of Educational Psychology, 107(3), 724-739.

https://doi.org/10.1037/edu0000013

Kizilcec, R. F., Papadopoulos, K., & Sritanyaratana, L. (2014). Showing face in video instruction: Effects

on information retention, visual attention, and affect. In Proceedings of the SIGCHI Conference

on Human Factors in Computing Systems (pp. 2095-2102). ACM.

https://doi.org/10.1145/2556288.2557207

Kolfschoten, G., Lukosch, S., Verbraeck, A., Valentin, E., & de Vreede, G. J. (2010). Cognitive learning

efficiency through the use of design patterns in teaching. Computers & Education, 54(3), 652-

660. https://doi.org/10.1016/j.compedu.2009.09.028

Korea Advanced Institute of Science and Technology. (2014a). Center for excellence in teaching and

learning. Retrieved from http://www.kaist.edu/html/en/edu/edu_030405.html

Diversity in Video Lectures: Aid or Hindrance?

Fanguy, Costley

Baldwin, Lange, and Wang

Korea Advanced Institute of Science and Technology. (2014b). Student handbook. Retrieved from

http://www.kaist.edu/html/en/campus/campus_0508.html

Leppink, J., Paas, F., Van der Vleuten, C. P., Van Gog, T., & Van Merriënboer, J. J. (2013). Development

of an instrument for measuring different types of cognitive load. Behavior Research Methods,

45(4), 1058-1072. https://doi.org/10.3758/s13428-013-0334-1

Li, J., Kizilcec, R., Bailenson, J., & Ju, W. (2016). Social robots and virtual agents as lecturers for video

instruction. Computers in Human Behavior, 55, 1222-1230.

https://doi.org/10.1016/j.chb.2015.04.005

Lowe, R. K. (1999). Extracting information from an animation during complex visual learning. European

Journal of Psychology of Education, 14(2), 225– 244. https://doi.org/10.1007/bf03172967

Luzón, J. M., & Letón, E. (2015). Use of animated text to improve the learning of basic mathematics.

Computers & Education, 88, 119-128. https://doi.org/10.1016/j.compedu.2015.04.016

Mayer, R. E. (2001). Multimedia learning. New York: Cambridge University Press. Retrieved from

http://steinhardtapps.es.its.nyu.edu/create/courses/2015/reading/Betrancourt.pdf

Mayer, R. E. (2014). Cognitive theory of multimedia learning. In R.E. Mayer (Ed.), The Cambridge

handbook of multimedia learning (2

ed., pp. 43-71). Cambridge: Cambridge University Press.

https://doi.org/10.1017/cbo9781139547369.005

Mayer, R. E., & Moreno, R. (1998). A split-attention effect in multimedia learning: Evidence for dual

processing systems in working memory. Journal of Educational Psychology, 90(2), 312-320.

https://doi.org/10.1037//0022-0663.90.2.312

Mayer, R. E., & Moreno, R. (2003). Nine ways to reduce cognitive load in multimedia learning.

Educational Psychologist, 38(1), 43-52. https://doi.org/10.1207/s15326985ep3801_6

Paivio, A. (1991). Dual coding theory: Retrospect and current status. Canadian Journal of Psychology,

45(3), 255-287. https://doi.org/10.1037/h0084295

Pi, Z., Hong, J., & Yang, J. (2017). Does instructor's image size in video lectures affect learning outcomes?

Journal of Computer Assisted Learning, 33(4), 347-354. https://doi.org/10.1111/jcal.12183

Rasch, T., & Schnotz, W. (2009). Interactive and non-interactive pictures in multimedia learning

environments: Effects on learning outcomes and learning efficiency. Learning and Instruction,

19(5), 411-422. https://doi.org/10.1016/j.learninstruc.2009.02.008

Salomon, G. (1994). Interaction of media, cognition, and learning: An exploration of how symbolic

forms cultivate mental skills and affect knowledge acquisition. Mahwah, NJ: Lawrence Erlbaum

Associates. https://doi.org/10.4324/9780203052945

Diversity in Video Lectures: Aid or Hindrance?

Fanguy, Costley

Baldwin, Lange, and Wang

Schmeck, A., Opfermann, M., van Gog, T., Paas, F., & Leutner, D. (2015). Measuring cognitive load with

subjective rating scales during problem solving: Differences between immediate and delayed

ratings. Instructional Science, 43(1), 93-114. https://doi.org/10.1007/s11251-014-9328-3

Schmidt-Weigand, F., & Scheiter, K. (2011). The role of spatial descriptions in learning from multimedia.

Computers in Human Behavior, 27(1), 22-28. https://doi.org/10.1016/j.chb.2010.05.007

Schnotz, W., & Rasch, T. (2005). Enabling, facilitating, and inhibiting effects of animations in multimedia

learning: Why reduction of cognitive load can have negative results on learning. Educational

Technology Research and Development, 53(3), 47-58. https://doi.org/10.1007/bf02504797

Schwan, S., & Riempp, R. (2004). The cognitive benefits of interactive videos: learning to tie nautical

knots. Learning and Instruction, 14(3), 293-305.

https://doi.org/10.1016/j.learninstruc.2004.06.005

Sims, V. K., & Hegarty, M. (1997). Mental animation in the visuospatial sketchpad: Evidence from dual-

task studies. Memory & Cognition, 25(3), 321– 332. https://doi.org/10.3758/bf03211288

Sorden, S. D. (2005). A cognitive approach to instructional design for multimedia learning. Informing

Science: The International Journal of an Emerging Transdiscipline, 8, 263-280.

https://doi.org/10.28945/498

Sweller, J. (1999). Instructional design in technical areas. Camberwell, Australia: ACER Press.

Sweller, J. (2005). Implications of cognitive load theory for multimedia learning. In R. E. Mayer (Ed.),

The Cambridge handbook of multimedia learning (pp. 19–30). Cambridge: Cambridge

University Press. https://doi.org/10.1017/cbo9780511816819.003

Sweller, J., & Chandler, P. (1994). Why some material is difficult to learn. Cognition and Instruction,

12(3), 185–233. https://doi.org/10.1207/s1532690xci1203_1

Sweller, J., Van Merriënboer, J. J., & Paas, F. (1998). Cognitive architecture and instructional design.

Educational Psychology Review, 10(3), 251–296. https://doi.org/10.1023/A:1022193728205

van Gog, T., Ericsson, K. A., Rikers, R. M., & Paas, F. (2005). Instructional design for advanced learners:

Establishing connections between the theoretical frameworks of cognitive load and deliberate

practice. Educational Technology Research and Development, 53(3), 73–81.

http://doi.org/10.1007/BF02504799

van Merriënboer, J. J. (1997). Training complex cognitive skills. Englewood Cliffs, NJ: Educational

Technology Publications.

Zhang, D., Zhao, J. L., Zhou, L., & Nunamaker, J. F., Jr. (2004). Can e-learning replace classroom

learning? Communications of the ACM, 47(5), 75–79. https://doi.org/10.1145/986213.986216

Diversity in Video Lectures: Aid or Hindrance?

Fanguy, Costley

Baldwin, Lange, and Wang

Zhang, D., Zhou, L., Briggs, R. O., & Nunamaker, J. F., Jr. (2006). Instructional video in e-learning:

Assessing the impact of interactive video on learning effectiveness. Information & Management,

43(1), 15-27. https://doi.org/10.1016/j.im.2005.01.004