July 2004 Journal of Engineering Education 1
MICHAEL P
RINCE
Department of Chemical Engineering
Bucknell University
ABSTRACT
This study examines the evidence for the effectiveness of active
learning. It defines the common forms of active learning most
relevant for engineering faculty and critically examines the core
element of each method. It is found that there is broad but
uneven support for the core elements of active, collaborative,
cooperative and problem-based learning.
I. INTRODUCTION
Active learning has received considerable attention over the
past several years. Often presented or perceived as a radical change
from traditional instruction, the topic frequently polarizes faculty.
Active learning has attracted strong advocates among faculty look-
ing for alternatives to traditional teaching methods, while skeptical
faculty regard active learning as another in a long line of educa-
tional fads.
For many faculty there remain questions about what active
learning is and how it differs from traditional engineering educa-
tion, since this is already “active” through homework assignments
and laboratories. Adding to the confusion, engineering faculty do
not always understand how the common forms of active learning
differ from each other and most engineering faculty are not inclined
to comb the educational literature for answers.
This study addresses each of these issues. First, it defines active
learning and distinguishes the different types of active learning
most frequently discussed in the engineering literature. A core ele-
ment is identified for each of these separate methods in order to dif-
ferentiate between them, as well as to aid in the subsequent analysis
of their effectiveness. Second, the study provides an overview of rel-
evant cautions for the reader trying to draw quick conclusions on
the effectiveness of active learning from the educational literature.
Finally, it assists engineering faculty by summarizing some of the
most relevant literature in the field of active learning.
II. DEFINITIONS
It is not possible to provide universally accepted definitions for
all of the vocabulary of active learning since different authors in the
field have interpreted some terms differently. However, it is possi-
ble to provide some generally accepted definitions and to highlight
distinctions in how common terms are used.
Active learning is generally defined as any instructional method
that engages students in the learning process. In short, active learn-
ing requires students to do meaningful learning activities and think
about what they are doing [1]. While this definition could include
traditional activities such as homework, in practice active learning
refers to activities that are introduced into the classroom. The core
elements of active learning are student activity and engagement in
the learning process. Active learning is often contrasted to the tra-
ditional lecture where students passively receive information from
the instructor.
Collaborative learning can refer to any instructional method in
which students work together in small groups toward a common goal
[2]. As such, collaborative learning can be viewed as encompassing all
group-based instructional methods, including cooperative learning
[3–7]. In contrast, some authors distinguish between collaborative
and cooperative learning as having distinct historical developments
and different philosophical roots [8–10]. In either interpretation, the
core element of collaborative learning is the emphasis on student in-
teractions rather than on learning as a solitary activity.
Cooperative learning can be defined as a structured form of group
work where students pursue common goals while being assessed in-
dividually [3, 11]. The most common model of cooperative learn-
ing found in the engineering literature is that of Johnson, Johnson
and Smith [12, 13]. This model incorporates five specific tenets,
which are individual accountability, mutual interdependence, face-
to-face promotive interaction, appropriate practice of interpersonal
skills, and regular self-assessment of team functioning. While dif-
ferent cooperative learning models exist [14, 15], the core element
held in common is a focus on cooperative incentives rather than
competition to promote learning.
Problem-based learning (PBL) is an instructional method where
relevant problems are introduced at the beginning of the instruction
cycle and used to provide the context and motivation for the learn-
ing that follows. It is always active and usually (but not necessarily)
collaborative or cooperative using the above definitions. PBL typi-
cally involves significant amounts of self-directed learning on the
part of the students.
III. COMMON PROBLEMS INTERPRETING THE
LITERATURE ON ACTIVE LEARNING
Before examining the literature to analyze the effectiveness of
each approach, it is worth highlighting common problems that en-
gineering faculty should appreciate before attempting to draw con-
clusions from the literature.
A. Problems Defining What Is Being Studied
Confusion can result from reading the literature on the effec-
tiveness of any instructional method unless the reader and author
Does Active Learning Work? A Review
of the Research
[QA1]
J. Engr. Education, 93(3), 223-231 (2004).
take care to specify precisely what is being examined. For example,
there are many different approaches that go under the name of
problem-based learning [16]. These distinct approaches to PBL
can have as many differences as they have elements in common,
making interpretation of the literature difficult. In PBL, for exam-
ple, students typically work in small teams to solve problems in a
self-directed fashion. Looking at a number of meta-analyses [17],
Norman and Schmidt [18] point out that having students work in
small teams has a positive effect on academic achievement while
self-directed learning has a slight negative effect on academic
achievement. If PBL includes both of these elements and one asks if
PBL works for promoting academic achievement, the answer seems
to be that parts of it do and parts of it do not. Since different appli-
cations of PBL will emphasize different components, the literature
results on the overall effectiveness of PBL are bound to be confus-
ing unless one takes care to specify what is being examined. This is
even truer of the more broadly defined approaches of active or col-
laborative learning, which encompass very distinct practices.
Note that this point sheds a different light on some of the avail-
able meta-analyses that are naturally attractive to a reader hoping
for a quick overview of the field. In looking for a general sense of
whether an approach like problem-based learning works, nothing
seems as attractive as a meta-analysis that brings together the results
of several studies and quantitatively examines the impact of the ap-
proach. While this has value, there are pitfalls. Aggregating the re-
sults of several studies on the effectiveness of PBL can be mislead-
ing if the forms of PBL vary significantly in each of the individual
studies included in the meta-analysis.
To minimize this problem, the analysis presented in Section IV
of this paper focuses on the specific core elements of a given instruc-
tional method. For example, as discussed in Section II, the core ele-
ment of collaborative learning is working in groups rather than
working individually. Similarly, the core element of cooperative
learning is cooperation rather than competition. These distinctions
can be examined without ambiguity. Furthermore, focusing on the
core element of active learning methods allows a broad field to be
treated concisely.
B. Problems Measuring “What Works”
Just as every instructional method consists of more than one ele-
ment, it also affects more than one learning outcome [18]. When
asking whether active learning “works,” the broad range of out-
comes should be considered such as measures of factual knowledge,
relevant skills and student attitudes, and pragmatic items as student
retention in academic programs. However, solid data on how an in-
structional method impacts all of these learning outcomes is often
not available, making comprehensive assessment difficult. In addi-
tion, where data on multiple learning outcomes exists it can include
mixed results. For example, some studies on problem-based learn-
ing with medical students [19, 20] suggest that clinical performance
is slightly enhanced while performance on standardized exams de-
clines slightly. In cases like this, whether an approach works is a
matter of interpretation and both proponents and detractors can
comfortably hold different views.
Another significant problem with assessment is that many rele-
vant learning outcomes are simply difficult to measure. This is par-
ticularly true for some of the higher level learning outcomes that are
targeted by active learning methods. For example, PBL might nat-
urally attract instructors interested in developing their students’
ability to solve open-ended problems or engage in life-long learn-
ing, since PBL typically provides practice in both skills. However,
problem solving and life-long learning are difficult to measure. As a
result, data are less frequently available for these outcomes than for
standard measures of academic achievement such as test scores.
This makes it difficult to know whether the potential of PBL to
promote these outcomes is achieved in practice.
Even when data on higher-level outcomes are available, it is easy
to misinterpret reported results. Consider a study by Qin et al. [21]
that reports that cooperation promotes higher quality individual
problem solving than does competition. The result stems from the
finding that individuals in cooperative groups produced better solu-
tions to problems than individuals working in competitive environ-
ments. While the finding might provide strong support for cooper-
ative learning, it is important to understand what the study does not
specifically demonstrate. It does not necessarily follow from these
results that students in cooperative environments developed
stronger, more permanent and more transferable problem solving
skills. Faculty citing the reference to prove that cooperative learning
results in individuals becoming generically better problem solvers
would be over-interpreting the results.
A separate problem determining what works is deciding when
an improvement is significant. Proponents of active learning some-
times cite improvements without mentioning that the magnitude of
the improvement is small [22]. This is particularly misleading when
extra effort or resources are required to produce an improvement.
Quantifying the impact of an intervention is often done using effect
sizes, which are defined to be the difference in the means of a sub-
ject and control population divided by the pooled standard devia-
tion of the populations. An improvement with an effect size of 1.0
would mean that the test population outperformed the control
group by one standard deviation. Albanese [23] cites the benefits of
using effect sizes and points out that Cohen [24] arbitrarily labeled
effect sizes of 0.2, 0.5 and 0.8 as small, medium and large, respec-
tively. Colliver [22] used this fact and other arguments to suggest
that effect sizes should be at least 0.8 before they be considered sig-
nificant. However, this suggestion would discount almost every
available finding since effect sizes of 0.8 are rare for any intervention
and require truly impressive gains [23]. The effect sizes of 0.5 or
higher reported in Section IV of this paper are higher than those
found for most instructional interventions. Indeed, several decades
of research indicated that standard measures of academic achieve-
ment were not particularly sensitive to any change in instructional
approach [25]. Therefore, reported improvements in academic
achievement should not be dismissed lightly.
Note that while effect sizes are a common measure of the mag-
nitude of an improvement, absolute rather than relative values are
sometimes more telling. There can be an important difference be-
tween results that are statistically significant and those that are sig-
nificant in absolute terms. For this reason, it is often best to find
both statistical and absolute measures of the magnitude of a report-
ed improvement before deciding whether it is significant.
As a final cautionary note for interpreting reported results, some
readers dismiss reported improvements from nontraditional in-
structional methods because they attribute them to the Hawthorne
effect whereby the subjects knowingly react positively to any novel
intervention regardless of its merit. The Hawthorne effect is gener-
ally discredited, although it retains a strong hold on the popular
imagination [26].
2 Journal of Engineering Education July 2004
C. Summary
There are pitfalls for engineering faculty hoping to pick up an ar-
ticle or two to see if active learning works. In particular, readers
must clarify what is being studied and how the authors measure and
interpret what “works.” The former is complicated by the wide
range of methods that fall under the name of active learning, but
can be simplified by focusing on core elements of common active
learning methods. Assessing “what works” requires looking at a
broad range of learning outcomes, interpreting data carefully, quan-
tifying the magnitude of any reported improvement and having
some idea of what constitutes a “significant” improvement. This last
will always be a matter of interpretation, although it is helpful to
look at both statistical measures such as effect sizes and absolute val-
ues for reported learning gains.
No matter how data is presented, faculty adopting instructional
practices with the expectation of seeing results similar to those re-
ported in the literature should be aware of the practical limitations
of educational studies. Educational studies tell us what worked, on
average, for the populations examined and learning theories suggest
why this might be so. However, claiming that faculty who adopt a
specific method will see similar results in their own classrooms is
simply not possible. Even if faculty master the new instructional
method, they can not control all other variables that affect learning.
The value of the results presented in Section IV of the paper is that
they provide information to help teachers “go with the odds.” The
more extensive the data supporting an intervention, the more a
teacher’s students resemble the test population and the bigger the
reported gains, the better the odds are that the method will work for
a given instructor.
Notwithstanding all of these problems, engineering faculty
should be strongly encouraged to look at the literature on active
learning. Some of the evidence for active learning is compelling and
should stimulate faculty to think about teaching and learning in
nontraditional ways.
IV. T
HE E
VIDENCE FOR ACTIVE LEARNING
Bonwell and Eison [1] summarize the literature on active learn-
ing and conclude that it leads to better student attitudes and im-
provements in students’ thinking and writing. They also cite evi-
dence from McKeachie that discussion, one form of active learning,
surpasses traditional lectures for retention of material, motivating
students for further study and developing thinking skills. Felder
et al. [27] include active learning on their recommendations for
teaching methods that work, noting among other things that active
learning is one of Chickering and Gamson’s “Seven Principles for
Good Practice” [28].
However, not all of this support for active learning is compelling.
McKeachie himself admits that the measured improvements of dis-
cussion over lecture are small [29]. In addition, Chickering and
Gamson do not provide hard evidence to support active learning as
one of their principles. Even studies addressing the research base for
Chickering and Gamson’s principles come across as thin with re-
spect to empirical support for active learning. For example, Scorcelli
[30], in a study aimed at presenting the research base for Chicker-
ing and Gamson’s seven principles, states that, “We simply do not
have much data confirming beneficial effects of other (not coopera-
tive or social) kinds of active learning.”
Despite this, the empirical support for active learning is exten-
sive. However, the variety of instructional methods labeled as active
learning muddles the issue. Given differences in the approaches la-
beled as active learning, it is not always clear what is being promoted
by broad claims supporting the adoption of active learning. Perhaps
it is best, as some proponents claim, to think of active learning as an
approach rather than a method [31] and to recognize that different
methods are best assessed separately.
This assessment is done in the following sections, which look at
the empirical support for active, collaborative, cooperative and prob-
lem-based learning. As previously discussed, the critical elements of
each approach are singled out rather than examining the effective-
ness of every possible implementation scheme for each of these dis-
tinct methods. The benefits of this general approach are twofold.
First, it allows the reader to examine questions that are both funda-
mental and pragmatic, such as whether introducing activity into the
lecture or putting students into groups, is effective. Second, focusing
on the core element eliminates the need to examine the effectiveness
of every instructional technique that falls under a given broad catego-
ry, which would be impractical within the scope of a single paper.
Readers looking for literature on a number of specific active learning
methods are referred to additional references [1, 6, 32].
A. Active Learning
We have defined the core elements of active learning to be intro-
ducing activities into the traditional lecture and promoting student
engagement. Both elements are examined below, with an emphasis
on empirical support for their effectiveness.
1) Introducing student activity into the traditional lecture: On
the simplest level, active learning is introducing student activity into
the traditional lecture. One example of this is for the lecturer to
pause periodically and have students clarify their notes with a part-
ner. This can be done two or three times during an hour-long class.
Because this pause procedure is so simple, it provides a baseline to
study whether short, informal student activities can improve the ef-
fectiveness of lectures.
Ruhl et al. [33] show some significant results of adopting this
pause procedure. In a study involving 72 students over two courses
in each of two semesters, the researchers examined the effect of in-
terrupting a 45-minute lecture three times with two-minute breaks
during which students worked in pairs to clarify their notes. In par-
allel with this approach, they taught a separate group using a
straight lecture and then tested short and long-term retention of
lecture material. Short-term retention was assessed by a free-recall
exercise where students wrote down everything they could remem-
ber in three minutes after each lecture and results were scored by the
number of correct facts recorded. Short-term recall with the pause
procedure averaged 108 correct facts compared to 80 correct facts
recalled in classes with straight lecture. Long-term retention was as-
sessed with a 65 question multiple-choice exam given one and a half
weeks after the last of five lectures used in the study. Test scores
were 89.4 with the pause procedure compared to 80.9 without
pause for one class, and 80.4 with the pause procedure compared to
72.6 with no pause in the other class. Further support for the effec-
tiveness of pauses during the lecture is provided by Di Vesta [34].
Many proponents of active learning suggest that the effectiveness
of this approach has to do with student attention span during lecture.
Wankat [35] cites numerous studies that suggest that student
July 2004 Journal of Engineering Education 3
attention span during lecture is roughly fifteen minutes. After that,
Hartley and Davies [36] found that the number of students paying
attention begins to drop dramatically with a resulting loss in reten-
tion of lecture material. The same authors found that immediately
after the lecture students remembered 70 percent of information
presented in first ten minutes of the lecture and 20 percent of infor-
mation presented in last ten minutes. Breaking up the lecture might
work because students’ minds start to wander and activities provide
the opportunity to start fresh again, keeping students engaged.
2) Promoting Student Engagement: Simply introducing activ-
ity into the classroom fails to capture an important component of
active learning. The type of activity, for example, influences how
much classroom material is retained [34]. In “Understanding by
Design” [37], the authors emphasize that good activities develop
deep understanding of the important ideas to be learned. To do
this, the activities must be designed around important learning out-
comes and promote thoughtful engagement on the part of the stu-
dent. The activity used by Ruhl, for example, encourages students
to think about what they are learning. Adopting instructional prac-
tices that engage students in the learning process is the defining fea-
ture of active learning.
The importance of student engagement is widely accepted and
there is considerable evidence to support the effectiveness of student
engagement on a broad range of learning outcomes. Astin [38]
reports that student involvement is one of the most important pre-
dictors of success in college. Hake [39] examined pre- and post-test
data for over 6,000 students in introductory physics courses and
found significantly improved performance for students in classes
with substantial use of interactive-engagement methods. Test
scores measuring conceptual understanding were roughly twice as
high in classes promoting engagement than in traditional courses.
Statistically, this was an improvement of two standard deviations
above that of traditional courses. Other results supporting the effec-
tiveness of active-engagement methods are reported by Redish et al.
[40] and Laws et al. [41]. Redish et al. show that the improved
learning gains are due to the nature of active engagement and not to
extra time spent on a given topic. Figure 1, taken from Laws et al.,
shows that active engagement methods surpass traditional instruc-
tion for improving conceptual understanding of basic physics con-
cepts. The differences are quite significant. Taken together, the
studies of Hake et al., Redish et al. and Laws et al. provide consider-
able support for active engagement methods, particularly for ad-
dressing students’ fundamental misconceptions. The importance of
addressing student misconceptions has recently been recognized as
an essential element of effective teaching [42].
In summary, considerable support exists for the core elements of
active learning. Introducing activity into lectures can significantly
improve recall of information while extensive evidence supports the
benefits of student engagement.
B. Collaborative Learning
The central element of collaborative learning is collaborative vs.
individual work and the analysis therefore focuses on how collabora-
tion influences learning outcomes. The results of existing meta-stud-
ies on this question are consistent. In a review of 90 years of research,
Johnson, Johnson and Smith found that cooperation improved learn-
ing outcomes relative to individual work across the board [12]. Simi-
lar results were found in an updated study by the same authors [13]
that looked at 168 studies between 1924 and 1997. Springer et al.
[43] found similar results looking at 37 studies of students in science,
mathematics, engineering and technology. Reported results for each
of these studies are shown in Table 1, using effect sizes to show the
impact of collaboration on a range of learning outcomes.
What do these results mean in real terms instead of effect sizes,
which are sometimes difficult to interpret? With respect to academic
achievement, the lowest of the three studies cited would move a
4 Journal of Engineering Education July 2004
Figure 1. Active-engagement vs. traditional instruction for im-
proving students’ conceptual understanding of basic physics concepts
(taken from Laws et al., 1999)
Table 1. Collaborative vs. individualistic learning: Reported effect size of the improvement in different learning outcomes.
student from the 50
th
to the 70
th
percentile on an exam. In absolute
terms, this change is consistent with raising a student’s grade from
75 to 81, given classical assumptions about grade distributions.*
With respect to retention, the results suggest that collaboration re-
duces attrition in technical programs by 22 percent, a significant
finding when technical programs are struggling to attract and retain
students. Furthermore, some evidence suggests that collaboration is
particularly effective for improving retention of traditionally under-
represented groups [44, 45].
A related question of practical interest is whether the benefits of
group work improve with frequency. Springer et al. looked specifical-
ly at the effect of incorporating small, medium and large amounts of
group work on achievement and found the positive effect sizes associ-
ated with low, medium and high amount of time in groups to be 0.52,
0.73 and 0.53, respectively. That is, the highest benefit was found for
medium time in groups. In contrast, more time spent in groups did
produce the highest effect on promoting positive student attitudes,
with low, medium and high amount of time in groups having effect
sizes of 0.37, 0.26, and 0.77, respectively. Springer et al. note that the
attitudinal results were based on a relatively small number of studies.
In summary, a number of meta-analyses support the premise
that collaboration “works” for promoting a broad range of student
learning outcomes. In particular, collaboration enhances academic
achievement, student attitudes, and student retention. The magni-
tude, consistency and relevance of these results strongly suggest that
engineering faculty promote student collaboration in their courses.
C. Cooperative Learning
At its core, cooperative learning is based on the premise that co-
operation is more effective than competition among students for
producing positive learning outcomes. This is examined in Table 2.
The reported results are consistently positive. Indeed, looking at
high quality studies with good internal validity, the already large ef-
fect size of 0.67 shown in Table 2 for academic achievement in-
creases to 0.88. In real terms, this would increase a student’s exam
score from 75 to 85 in the “classic” example cited previously, though
of course this specific result is dependent on the assumed grade dis-
tribution. As seen in Table 2, cooperation also promotes interper-
sonal relationships, improves social support and fosters self-esteem.
Another issue of interest to engineering faculty is that coopera-
tive learning provides a natural environment in which to promote
effective teamwork and interpersonal skills. For engineering faculty,
the need to develop these skills in their students is reflected by the
ABET engineering criteria. Employers frequently identify team
skills as a critical gap in the preparation of engineering students.
Since practice is a precondition of learning any skill, it is difficult to
argue that individual work in traditional classes does anything to
develop team skills.
Whether cooperative learning effectively develops interpersonal
skills is another question. Part of the difficulty in answering that
question stems from how one defines and measures team skills.
Still, there is reason to think that cooperative learning is effective in
this area. Johnson et al. [12, 13] recommend explicitly training stu-
dents in the skills needed to be effective team members when using
cooperative learning groups. It is reasonable to assume that the op-
portunity to practice interpersonal skills coupled with explicit in-
structions in these skills is more effective than traditional instruction
that emphasizes individual learning and generally has no explicit in-
struction in teamwork. There is also empirical evidence to support
this conclusion. Johnson and Johnson report that social skills tend
to increase more within cooperative rather than competitive or indi-
vidual situations [46]. Terenzini et al. [47] show that students re-
port increased team skills as a result of cooperative learning. In addi-
tion, Panitz [48] cites a number of benefits of cooperative learning
for developing the interpersonal skills required for effective team-
work.
In summary, there is broad empirical support for the central
premise of cooperative learning, that cooperation is more effective
than competition for promoting a range of positive learning out-
comes. These results include enhanced academic achievement and a
number of attitudinal outcomes. In addition, cooperative learning
provides a natural environment in which to enhance interpersonal
skills and there are rational arguments and evidence to show the ef-
fectiveness of cooperation in this regard.
D. Problem-Based Learning
As mentioned in Section II of this paper, the first step of deter-
mining whether an educational approach works is clarifying exactly
what the approach is. Unfortunately, while there is agreement on
the general definition of PBL, implementation varies widely.
Woods et al. [16], for example, discuss several variations of PBL.
“Once a problem has been posed, different instructional methods may be
used to facilitate the subsequent learning process: lecturing, instructor-
facilitated discussion, guided decision making, or cooperative learning. As
part of the problem-solving process, student groups can be assigned to
July 2004 Journal of Engineering Education 5
*Calculated using an effect size of 0.5, a mean of 75 and a normalized grade dis-
tribution where the top 10 percent of students receive a 90 or higher (an A) and the
bottom 10 percent receive a 60 or lower (an F).
Table 2. Collaborative vs. competitive learning: Reported effect size of the improvement in different learning outcomes.
complete any of the learning tasks listed above, either in or out of class. In the
latter case, three approaches may be adopted to help the groups stay on track
and to monitor their progress: (1) give the groups written feedback after each
task; (2) assign a tutor or teaching assistant to each group, or (3) create fully
autonomous, self-assessed “tutorless” groups.”
The large variation in PBL practices makes the analysis of its ef-
fectiveness more complex. Many studies comparing PBL to tradi-
tional programs are simply not talking about the same thing. For
meta-studies of PBL to show any significant effect compared to tra-
ditional programs, the signal from the common elements of PBL
would have to be greater than the noise produced by differences in
the implementation of both PBL and the traditional curricula.
Given the huge variation in PBL practices, not to mention differ-
ences in traditional programs, readers should not be surprised if no
consistent results emerge from meta-studies that group together
different PBL methods.
Despite this, there is at least one generally accepted finding that
emerges from the literature, which is that PBL produces positive
student attitudes. Vernon and Blake [19] looking at 35 studies from
1970 to 1992 for medical programs found that PBL produced a sig-
nificant effective size (0.55) for improved student attitudes and
opinions about their programs. Albanese and Mitchell [20] similar-
ly found that students and faculty generally prefer the PBL ap-
proach. Norman and Schmidt [18] argue “PBL does provide a
more challenging, motivating and enjoyable approach to education.
That may be a sufficient raison d’etre, providing the cost of the im-
plementation is not too great.” Note that these and most of the re-
sults reported in this section come from studies of medical students,
for whom PBL has been widely used. While PBL has been used in
undergraduate engineering programs [49, 50] there is very little
data available for its effectiveness with this population of students.
Beyond producing positive student attitudes, the effects of PBL
are less generally accepted, though other supporting data do exist.
Vernon and Blake [19], for example, present evidence that there is a
statistically significant improvement of PBL on students’ clinical
performance with an effect size of 0.28. However, Colliver [22]
points out that this is influenced strongly by one outlying study with
a positive effect size of 2.11, which skews the data. There is also evi-
dence that PBL improves the long-term retention of knowledge
compared to traditional instruction [51–53]. Evidence also suggests
that PBL promotes better study habits among students. As one
might expect from an approach that requires more independence
from students, PBL has frequently been shown to increase library
use, textbook reading, class attendance and studying for meaning
rather than simple recall [19, 20, 53, 54].
We have already discussed the problems with meta-studies that
compare non-uniform and inconsistently defined educational inter-
ventions. Such studies are easily prone to factors that obscure re-
sults. The approach for handling this difficulty with active, collabo-
rative and cooperative learning was to identify the central element
of the approach and to focus on this rather than on implementation
methods. That is more difficult to do with PBL since it is not clear
that one or two core elements exist. PBL is active, engages students
and is generally collaborative, all of which are supported by our pre-
vious analysis. It is also inductive, generally self-directed, and often
includes explicit training in necessary skills. Can one or two ele-
ments be identified as common or decisive?
Norman and Schmidt [18] provide one way around the difficul-
ty by identifying several components of PBL in order to show how
they impact learning outcomes. Their results are shown in Table 3,
taken directly from Norman and Schmidt using the summary of
meta-studies provided by Lipsey and Wilson [17]. The measured
learning outcome for all educational studies cited by Lipsey and
Wilson was academic achievement.
Norman and Schmidt present this table to illustrate how differ-
ent elements of PBL have different effects on learning outcomes.
However, the substantive findings of Table 3 are also worth high-
lighting for faculty interested in adopting PBL because there seems
to be considerable agreement on what works and does not work in
PBL.
Looking first at the negative effects, there is a significant nega-
tive effect size using PBL with non-expert tutors. This finding is
consistent with some of the literature on helping students make the
transition from novice to expert problem solvers. Research compar-
ing experts to novices in a given field has demonstrated that becom-
ing an expert is not just a matter of “good thinking” [42]. Instead,
research has demonstrated the necessity for experts to have both a
deep and broad foundation of factual knowledge in their fields. The
same appears to be true for tutors in PBL.
There is also a small negative effect associated with both self-
paced and self-directed learning. This result is consistent with the
findings of Albanese and Mitchell [20] on the effect of PBL on test
results. In seven out of ten cases they found that students in PBL
programs scored lower than students in traditional programs on
tests of basic science. However, in three out of ten cases, PBL
students actually scored higher. Albanese and Mitchell note that
these three PBL programs were more “directive” than others,
6 Journal of Engineering Education July 2004
Table 3. Effect sizes associated with various aspects of problem-based learning.
indicating that this element might be responsible for the superior
exam performance for students in those programs. Therefore, facul-
ty might be advised to be cautious about the amount of self-direc-
tion required by students in PBL, at least with regard to promoting
academic achievement as measured by traditional exams.
Looking at what seems to work, there are significant positive ef-
fect sizes associated with placing students in small groups and using
cooperative learning structures. This is consistent with much of the
literature cited previously in support of cooperative learning. While
PBL and cooperative learning are distinct approaches, there is a
natural synergy that instructors should consider exploiting. That is,
real problems of the sort used in PBL require teams to solve effec-
tively. At the same time, the challenge provided by realistic prob-
lems can provide some of the mutual interdependence that is one of
the five tenets of cooperative learning.
Table 3 also shows that positive results come from instruction in
problem solving. This is consistent with much of the advice given
by proponents of problem-based learning [55]. While practice is
crucial for mastering skills such as problem solving, greater gains are
realized through explicit instruction of problem solving skills. How-
ever, traditional engineering courses do not generally teach problem
solving skills explicitly. Table 3 suggests that faculty using PBL
consider doing just that.
In conclusion, PBL is difficult to analyze because there is not
one or two core elements that can be clearly identified with student
learning outcomes. Perhaps the closest candidates for core elements
would be inductive or discovery learning. These have been shown
by meta-studies to have only weakly positive effects on student aca-
demic achievement [56, 57] as measured by exams. This might ex-
plain why PBL similarly shows no improvement on student test
scores, the most common measure of academic achievement.
However, while no evidence proves that PBL enhances academ-
ic achievement as measured by exams, there is evidence to suggest
that PBL “works” for achieving other important learning outcomes.
Studies suggest that PBL develops more positive student attitudes,
fosters a deeper approach to learning and helps students retain
knowledge longer than traditional instruction. Further, just as co-
operative learning provides a natural environment to promote inter-
personal skills, PBL provides a natural environment for developing
problem-solving and life-long learning skills. Indeed, some evi-
dence shows that PBL develops enhanced problem-solving skills in
medical students and that these skills can be improved further by
coupling PBL with explicit instruction in problem solving. Similar-
ly, supporting arguments can be made about PBL and the impor-
tant ABET engineering outcome of life-long learning. Since self-
directed learning and meta-cognition are common to both PBL
and life-long learning, a logical connection exists between this de-
sired learning outcome and PBL instruction, something often not
true when trying to promote life-long learning through traditional
teaching methods.
IV. CONCLUSIONS
Although the results vary in strength, this study has found sup-
port for all forms of active learning examined. Some of the findings,
such as the benefits of student engagement, are unlikely to be con-
troversial although the magnitude of improvements resulting from
active-engagement methods may come as a surprise. Other findings
challenge traditional assumptions about engineering education and
these are most worth highlighting.
For example, students will remember more content if brief activi-
ties are introduced to the lecture. Contrast this to the prevalent con-
tent tyranny that encourages faculty to push through as much materi-
al as possible in a given session. Similarly, the support for collaborative
and cooperative learning calls into question the traditional assump-
tions that individual work and competition best promote achieve-
ment. The best available evidence suggests that faculty should struc-
ture their courses to promote collaborative and cooperative
environments. The entire course need not be team-based, as seen by
the evidence in Springer et al. [43], nor must individual responsibility
be absent, as seen by the emphasis on individual accountability in co-
operative learning. Nevertheless, extensive and credible evidence sug-
gests that faculty consider a nontraditional model for promoting aca-
demic achievement and positive student attitudes.
Problem-based learning presents the most difficult method to
analyze because it includes a variety of practices and lacks a domi-
nant core element to facilitate analysis. Rather, different implemen-
tations of PBL emphasize different elements, some more effective
for promoting academic achievement than others. Based on the lit-
erature, faculty adopting PBL are unlikely to see improvements in
student test scores, but are likely to positively influence student atti-
tudes and study habits. Studies also suggest that students will retain
information longer and perhaps develop enhanced critical thinking
and problem-solving skills, especially if PBL is coupled with explicit
instruction in these skills.
Teaching cannot be reduced to formulaic methods and active
learning is not the cure for all educational problems. However, there
is broad support for the elements of active learning most commonly
discussed in the educational literature and analyzed here. Some of
the findings are surprising and deserve special attention. Engineer-
ing faculty should be aware of these different instructional methods
and make an effort to have their teaching informed by the literature
on “what works.”
ACKNOWLEDGMENTS
The author would like to thank Richard Felder for his thoughtful
critique of this work and for many similar pieces of advice over the
past several years. The National Science Foundation through Project
Catalyst (NSF9972758) provided financial support for this project.
REFERENCES
[1] Bonwell, C.C., and J. A. Eison, “Active Learning: Creating Ex-
citement in the Classroom,” ASHEERIC Higher Education Report No.
1, George Washington University, Washington, DC , 1991.
[2] Online Collaborative Learning in Higher Education, http://clp.
cqu.edu.au/glossary.htm, accessed 12/3/2003.
[3] Millis, B., and P. Cottell, Jr., “Cooperative Learning for Higher
Education Faculty,” American Council on Education, ORYX Press, 1998.
[4] Smith, B., and J. MacGregor, “What is Collaborative Learning?,”
in Goodsell, A., M. Mahler, V. Tinto, B.L.Smith, and J. MacGreger,
(Eds), Collaborative Learning: A Sourcebook for Higher Education (pp. 9–22).
University Park, PA: National Center on Postsecondary Teaching, Learn-
ing and Assessment, 1992.
July 2004 Journal of Engineering Education 7
[5] Cusea, J., “Collaborative & Cooperative Learning in Higher Edu-
cation: A Proposed Taxonomy,” Cooperative Learning and College Teaching,
Vol. 2, No. 2, 2–4, 1992.
[6] Bean, J., “Engaging Ideas: The Professor’s Guide to Integrating
Writing, Critical Thinking, and Active Learning in the Classroom,” Josey-
Bass Publishers: San Francisco, 1996.
[7] Felder, R., Brent, R., and Stice, J., “National Effective Teaching
Institute: Workshop Materials,” 2002 American Society for Engineering
Education Annual Conference, Montreal, Quebec, Canada, 2002.
[8] Bruffee, K., “Sharing Our Toys: Cooperative Learning Versus
Collaborative Learning,” Change, January/February, 1995, p. 12.
[9] Panitz, T., “Collaborative Versus Cooperative Learning-A Com-
parison of the Two Concepts Which Will Help Us Understand the Un-
derlying Nature of Interactive Learning,” http://capecod.net/~tpanitz/
tedsarticles/coopdefinition.htm, accessed 12/2/2003.
[10] http://www.wcer.wisc.edu/nise/CL1/CL/question/TQ13.
htm, accessed 12/3/2003.
[11] Feden, P., and R. Vogel, Methods of Teaching: Applying Cognitive
Science to Promote Student Learning, McGraw Hill Higher Education, 2003.
[12] Johnson, D., R., Johnson, and K. Smith, Active Learning: Cooper-
ation in the College Classroom, 2nd ed., Interaction Book Co., Edina, MN,
1998.
[13] Johnson, D., R., Johnson, and K. Smith, “Cooperative Learning
Returns to College: What Evidence is There That it Works?,” Change,
Vol. 30, No. 4. July/Aug., 1998, p. 26–35.
[14] Stahl, R., “The Essential Elements of Cooperative Learning in
the Classroom,” ERIC Digest ED370881, 1994, available online at
http://www.ericfacility.net/ericdigests/ed370881.html.
[15] Slavin, R., “Cooperative Learning. Research on Teaching Mono-
graph Series,” ERIC Digest ED242707, 1983.
[16] Woods, D., R Felder, A. Rugarcia, and J. Stice, “The Future of
Engineering Education. III. Developing Critical Skills,” Chemical Engi-
neering Education, Vol. 34, No. 2, 2000, pp. 108–117.
[17] Lipsey, M.W., and Wilson, D.B., “The Efficacy of Psychological,
Educational and Behavioral Treatment: Confirmation from Meta-
Analysis,” American Psychology, Vol. 12, 1993, p. 1181–1209.
[18] Norman, G., and H. Schmidt, “Effectiveness of Problem-Based
Learning Curricula: Theory, Practice and Paper Darts,” Medical Education,
Vol. 34, 2000, pp. 721–728.
[19] Vernon, D., and R. Blake, “Does Problem-Based Learning
Work? A Meta-Analysis of Evaluative Research,” Academic Medicine, Vol.
68, No. 7, July 1993.
[20] Albanese, M. and S. Mitchell, “Problem-Based Learning: A Re-
view of Literature on Its Outcomes and Implementation Issues,” Academic
Medicine, Vol. 68, No. 1, January 1993.
[21] Qin, Z., Johnson, D., and Johnson, R., “Cooperative Versus
Competitive Efforts and Problem Solving,” Review of Educational Research,
Vol. 65, No. 2, Summer 1995, p. 129.
[22] Colliver, J., “Effectiveness of Problem-Based Learning Curricu-
la,” Academic Medicine, Vol. 75, 2000, p. 259.
[23] Albanese, M., “Problem-Based Learning: Why Curricula Are
Likely to Show Little Effect on Knowledge and Clinical Skills,” Medical
Education, Vol. 34, No. 9, 2000, p. 729.
[24] Cohen, J., Statistical Power Analysis for Behavioral Sciences, Revised
Edition, Englewood Cliffs, New Jersey: Erlbaum; 1977.
[25] Dubin R., and T. Taveggia, “The Teaching -Learning Paradox.
A Comparative Analysis of College Teaching Methods,” University of
Oregon, USA: Center for the Advanced Study of Educational Administra-
tion, 1968.
[26] Bracey, G., “Tips for Readers of Research: Beware the ‘Classic
Study’,” Phi Delta Kappan, Vol. 83, No. 8, April 2002, p. 642.
[27] Felder, R., D. Woods, J. Stice, and A. Rugarcia, “The Future of
Engineering Education: II.Teaching Methods that Work,” Chemical Engi-
neering Education, Vol. 34, No. 1, 2000, pp. 26–39.
[28] Chickering, A., and Z. Gamson, “Seven Principles for Good
Practice,” AAHE Bulletin, Vol. 39, ED 282 491, March 1987, pp. 3–7.
[29] McKeachie, W., “Research on College Teaching,” Educational
Perspectives, Vol. 11, No. 2, May 1972, pp. 3–10.
[30] Sorcinelli, M., “Research Findings on the Seven Principles,” in
A.W. Chickering and Z.F. Gamson, eds., Applying the Seven Principles for
Good Practice in Undergraduate Education, New Directions in Teaching and
Learning, #47, San Francisco: Jossey-Bass, 1991.
[31] http://trc.ucdavis.edu/trc/active/defini.html, accessed
12/3/03.
[32] MacGregor, J., Cooper, J., Smith, K., and Robinson, P. (Eds.)
“Strategies for Energizing Large Classes: From Small Groups to Learning
Communities,” New Directions for Teaching and Learning, Vol. 81, Jossey-
Bass, 2000.
[33] Ruhl, K., C. Hughes, and P. Schloss, “Using the Pause Procedure
to Enhance Lecture Recall,” Teacher Education and Special Education, Vol.
10, Winter 1987, pp. 14–18.
[34] Di Vesta, F., and D. Smith, “The Pausing Principle: Increasing
the Efficiency of Memory for Ongoing Events,” Contemporary Educational
Psychology, Vol. 4, 1979.
[35] Wankat, P., The Effective Efficient Professor: Teaching, Scholarship
and Service, Allyn and Bacon: Boston, MA, 2002.
[36] Hartley, J., and Davies, I., “Note Taking: A Critical Review,”
Programmed Learning and Educational Technology, Vol. 15, 1978,
pp. 207–224.
[37] Wiggins, G., and J. McTighe, “Understanding by Design,”
Merrill Education/ASCD College Textbook Series, ASCD, Alexandria,
Virginia, 1998.
[38] Astin, A., What Matters in College?; Four Critical Years Revisited,
Josey-Bass: San Francisco, CA, 1993.
[39] Hake, R., “Interactive-Engagement vs. Traditional Methods: A
Six-Thousand-Student Survey of Mechanics Test Data for Introductory
Physics Courses,” American Journal of Physics, Vol. 66, No. 1, 1998, p. 64.
[40] Redish, E., J. Saul, and R. Steinberg, “On the Effectiveness of
Active-Engagement Microcomputer-Based Laboratories,” American
Journal of Physics, Vol. 65, No. 1, 1997, p. 45.
[41] Laws, P., D. Sokoloff, and R. Thornton, “Promoting Active
Learning Using the Results of Physics Education Research,” UniServe
Science News, Vol. 13, July 1999.
[42] Bransford, J., A. Brown, and R. Cocking, (Commission on Be-
havioral and Social Science and Education, National Research Council),
“How People Learn: Body, Mind, Experience and School,” National
Academy Press, Washington D.C., 2000. Available online at
http://www.nap.edu/html/howpeople1/.
[43] Springer, L., M. Stanne, and S. Donovan, “Effects of Small-
Group Learning on Undergraduates in Science, Mathematics, Engineer-
ing and Technology: A Meta-Analysis,” Review of Educational Research,
Vol. 69, No. 1, 1999, pp. 21–52.
[44] Fredericksen, E., “Minority Students and the Learning Commu-
nity Experience: A Cluster Experiment,” U.S.: Texas 1998-0400,
ED423533, 1998.
[45] Berry, Jr., L., “Collaborative Learning: A Program for Improving
the Retention of Minority Students, U.S.: Virginia, 1991-00-00,
ED384323, 1991.
8 Journal of Engineering Education July 2004
[46] Johnson, D., and R. Johnson, Cooperation and Competition, Theory
and Research, Edina, MN: Interaction Book Company, 1989.
[47] Terenzini, P., Cabrera, A., Colbeck, C., Parente, J., and Bjorklund
S., “Collaborative Learning vs. Lecture/Discussion: Students’ Reported
Learning Gains,” Journal of Engineering Education, Vol. 90, No. 1, 2001.
[48] Panitz, T., “The Case for Student Centered Instruction via
Collaborative Learning Paradigms,” U.S. Massachusetts, 1999-12-00, ED
448444, 1999.
[49] http://www.udel.edu/pbl/others.html#undergrad, accessed
1/16/04.
[50] http://pbl.cqu.edu.au/content/online_resources.htm,
accessed 1/16/04.
[51] Norman, G., and H. Schmidt, “The Psychological Basis of Prob-
lem-Based Learning: A Review of Evidence,” Academic Medicine, Vol. 67,
1993, pp. 557–565.
[52] Martensen, D., H. Eriksson, and M. Ingleman-Sundberg, “Med-
ical Chemistry: Evaluation of Active and Problem-Oriented Teaching
Methods,” Medical Education, Vol. 19, 1985, p. 34.
[53] Gallagher, S., “Problem-Based Learning: Where did it comes
from, what does it do and where is it going?,” Journal for Education of the
Gifted, Vol. 20, No. 4, Summer 1997, pp. 332–362.
[54] Major, C., and B. Palmer, “Assessing the Effectiveness of Prob-
lem-Based Learning in Higher Education: Lessons from the Literature,”
Academic Exchange Quarterly, Vol. 5, No. 1, 2001, p. 4.
[55] Woods, Donald R., http://www.chemeng.mcmaster.ca/pbl/
pbl.htm, accessed 12/3/03
[56] El-Nemr, M., “A Meta-Analysis of the Outcomes of Teaching
Biology as Inquiry,” (Doctoral Dissertation, University of Colorado, 1979),
Dissertation Abstracts International, 40, 5812A. (University Microfilms
International No 80-11274), 1980.
[57] Lott, G., “The Effect of Inquiry Teaching and Advance Organiz-
ers upon Student Outcomes in Science Education,” Journal of Research in
Science Teaching, Vol. 20, 1983, p. 437.
AUTHOR
’S BIOGRAPHY
Dr. Michael Prince is a professor in the Department of Chemi-
cal Engineering at Bucknell University, where he has been since re-
ceiving his Ph.D. from the University of California at Berkeley in
1989. He is the author of several education-related papers for engi-
neering faculty and gives faculty development workshops on active
learning. He is currently participating in Project Catalyst, an NSF-
funded initiative to help faculty re-envision their role in the learning
process.
Address: Department of Chemical Engineering, Bucknell Uni-
versity, Lewisburg, PA 17837; telephone: 570-577-1781; e-mail:
July 2004 Journal of Engineering Education 9
Query to Author
Title: Does Active Learning Work? A Review of the Research
Author: Michael Prince
QA. Author please provide keywords for this article.
