The Google Online Marketing Challenge: An Opportunity to Assess

ABD Journal, Volume 2, 2010

The Google Online Marketing Challenge:

An Opportunity to Assess Experiential Learning

Marilyn Lavin

University of Wisconsin-Whitewater

Whitewater, Wisconsin

[email protected]

Abstract

The Google Online Marketing Challenge offers an important opportunity for student

experiential learning. Google has also developed a 30 variable algorithm that tracks

participants’ online activity, and uses that tool to rate all Challenge teams. This

information, together with written reports students provided related to learning

expectations, team evaluations, and client interactions are used in this study to identify

the advantages and limitations of experiential learning.

Introduction

During the past two academic years, Google has sponsored the Online Marketing

Challenge. This is an opportunity for student teams to use $200 vouchers to run three-

week Adwords (search advertising) campaigns for the SMEs of their choice. Students

develop campaign strategies, select keywords appropriate to their client businesses, write

copy related to the keywords, geo-target their ads, and participate in Google’s auctions to

determine the “cost per click” for each keyword. They must also budget their $200 to

maximize results across the three-week period, and they obtain almost immediate

feedback on the success of their activities by monitoring the analytics related to

impressions, click- through- rates, and average cost per click provided in their accounts.

The “real money, real business, real time” Challenge is an important means for students

to gain first – hand experience with search advertising and to benefit from the almost

immediate feedback available in the online environment. It also requires team effort,

and allows students to work with an actual client. Moreover, at the end of the three-week

campaign, Challenge participants have metrics by which to judge the outcome of their

campaigns; unlike many other types of class assignments, the students have information

about whether their efforts succeeded or failed.

The Google Online Marketing Challenge (GOMC) offers students an excellent

opportunity to move from the classroom theory to “real world” application, and it may

also provide important insight about the effectiveness of experiential learning. As part of

the Post Campaign Strategy, Google requires students in the Challenge to submit a

discussion of their learning expectations and outcomes. In addition, Google analysts

Google’s Online Marketing Challenge Lavin, Marilyn

have developed a 30 variable algorithm to assess the online activity of each team during

its three-week campaign, and using this tool, Google provides an evaluation of each

team’s effort at the end of the Challenge. This paper will use these resources to begin an

assessment of the effectiveness of the Challenge in assisting student learning, and to

examine broader issues related to experiential learning.

Literature Review

The idea of learning by doing is not new. The 4

century BC philosopher, Sophocles

noted, “One must learn by doing the thing. For though you think you know it – you have

no certainty, until you try.” Gentry (1990) attempted to synthesize the insights of earlier

scholars, including Hoover and Whitehead (1974) and Wolfe and Bryne (1975), with the

definition of experiential learning put forth by the AACSB task force in 1986. He

suggested that “experiential learning is participative, interactive, and applied. It allows

contact with the environment, and exposure to processes that are highly variable and

uncertain.”

Academic advocates of experiential learning come from all business disciplines. Stiller

and LeBlanc (2002), for example, argued that only by completing a semester-long

project that permits students to experience the benefits of software engineering

techniques will they become convinced of the value of those techniques. Similarly,

Munoz and Huser (2008) suggested a semester-long project in which students analyze a

product category, competition, and consumer base to support a new product proposal as a

means to put into practice the concepts and principles taught in the introductory

Marketing class. However, despite such arguments and AACSB’s encouragement of

“real world” learning, Navarro’s (2008) study of the MBA core curricula of the top

business schools in the U.S. revealed that these institutions did not provide adequate

experiential opportunities.

Employers also strongly endorse experiential learning. A poll of 301 employers

conducted by Peter Hart Research Associates (2008) indicated that they favor assessment

of real world and applied learning rather than standardized multiple choice testing. Those

results are in keeping with the results of a Wall Street Journal survey of 4,430 recruiters

that showed interpersonal and communication skills, a teamwork orientation, analytical

and problem-solving abilities, and a strong work ethic to be the most important attributes

they looked for in new hires (Alsop 2007).

Rosso et al. (2009) concluded that the GOMC offers a valuable important opportunity for

experiential learning. Students must learn to work with real business people to solve a

real problem. They need to develop a strategy for their advertising campaigns, use the

online tools and reports provided by Google, adjust their plans as they receive feedback

on what works and what does not, budget their $200 over the three-week period, and

write a final business report. The authors also note the GOMC provides important

assistance to faculty. The Challenge offers a project that has reasonable size and scope to

be used as a class assignment, and that is also sufficiently uniform to permit fair and

consistent grading. In addition, a panel of academics developed highly specific rubrics

Google’s Online Marketing Challenge Lavin, Marilyn

for the Pre and Post Campaign reports that must be submitted to Google. These rubrics

should help students set up their campaigns and write their final reports; they can also

assist faculty in grading both of these documents.

Study Questions

The GOMC allows students to do a real world project where they can work for a “real”

client, spend “real money,” and almost immediately see the results of their efforts. Each

competing team provides a discussion of its members’ learning expectations. In addition,

the online environment in which the students operate for three weeks permits tracking of

all of their activity. These circumstances suggest the possibility that the GOMC may

offer an important means of understanding the value of experiential learning. More

specifically, the following questions will be examined.

1. Does the opportunity for experiential learning lead to high student satisfaction?

2. What did students expect to learn by participating in the Google Challenge,

and how well could they articulate an expectation?

3. Is student work effort commensurate with learning expectations?

4. Does an experiential opportunity result in better team participation?

5. Do client businesses actively assist the learning of the student teams?

Method

During the Spring 2009 semester, the author supervised 29 teams that competed in the

GOMC. In all, 14 teams were from the online graduate Internet Marketing class, 9 were

from the face-to-face undergraduate Internet Marketing class, and 5 were from the online

undergraduate Internet Market class. All of these teams completed the “Learning

Component” of the Google Post Campaign Summaries. This rubric required students to

reflect on four aspects of their learning during the Challenge:

1. Learning objectives and outcomes – what did the team hope to learn? How well did the team

meet their learning expectations? What else did they learn? What key outcomes will the team

remember? What were the expected and unexpected outcomes from participating in the

Challenge?

2. Group dynamics – what problems did the team encounter and more importantly, how did they

overcome these problems? What were some of the expected and unexpected outcomes from

working as a group?

3. Client dynamics – what problems did they encounter and as importantly, how did they overcome

these problems? What were some of the expected and unexpected outcomes from working with

the client?

4. Future recommendations – what would they do differently in the future to improve

their campaign strategy, learning experience, group dynamics and client dynamics?

(Google Academic Guide 2009.)

This information will provide insight related to students’ learning expectations, how they

assessed their learning experience, team issues and client interactions. All members also

did evaluations of the performance of their respective team members, and completed an

evaluation of the Internet Marketing course. The first of these documents will provide

Google’s Online Marketing Challenge Lavin, Marilyn

additional information related to team dynamics, while the latter offers the means of

assessing overall student satisfaction with the Internet Marketing class of which the

Challenge was a major component.

To assess the performance of the teams competing in the Challenge, Google analysts

developed a 30 variable algorithm. This algorithm considers five factors: account

structure, optimization techniques, account activity and reporting, budgeting, and

relevance. Google tracks the activity within each team’s account, and uses that

information to determine the effectiveness of each campaign. This method of student

assessment is a major departure from traditional methods of measuring “experiential

learning” which have relied on qualitative reports from client firms or instructor grading

of final reports. Instead, the algorithm directly measures the “work effort” as well as the

resulting “work product” of each team. It considers whether the team set up the ad

campaign set up efficiently, whether Ad Groups had relevant Ad texts related to their

respective keywords, whether the team used the various online tools available to AdWord

customers and followed the “best practices” outlined in the guides, whether the team

regularly monitored the account and adjusted its campaign to maximize performance,

whether the team effectively budgeted its $200 across the three-week period, and whether

team ads were relevant and achieved reasonable click-through-rates. Google reported its

scoring of each team participating in the 2009 Challenge in mid July 2009, when the

company identified Challenge winners, finalists, semi-finalists, and those teams rated

“strong,” “good,” “fair,” and “needs improvement” (Google 2009). The Google results

provide an “objective measure” of student performance.

Findings

Q1. Does the opportunity for experiential learning lead to high student satisfaction?

Student evaluations of the three classes which participated in the GOMC were all

high, and, in fact, fell in the “Outstanding” range of scores. Students in the graduate class

were also asked to indicate the strengths of the class. Of the 50 class members, 38

responded to this question and 15 made specific reference to the Google project. One

student noted that “I learned most from my hands on experience with Google AdWords;”

another commented “By running an actual campaign, I think we were able to learn more

about marketing concepts than just reading about them in a book;” and yet another named

the Google Challenge an “awesome experience” and went on to comment that “it is the

only project I ever participated in, where I got to try out real technology, and apply it to a

business. I would relate the challenge more to an internship than a class assignment.” To

be certain, the students related such factors as enhancement of critical thinking skills and

the timely return of graded assignments to course satisfaction, but the large number that

mentioned the AdWords project suggests the likelihood that the experiential learning

does have a positive relation to student satisfaction.

Q2. What did students expect to learn by participating in the Google Challenge, and

how well could they articulate an expectation? As part of the Post Campaign summary

student teams submitted to Google, the GOMC rubric required students to reflect and

write about their learning objectives related to participation in the Challenge. Many of

Google’s Online Marketing Challenge Lavin, Marilyn

the undergraduate teams mentioned general goals. They stated that they “were hoping to

become more knowledegeable [sic] about how internet marketing worked,” they wanted

to “know how to successfully run a Google ad words campaign,” and they were

interested in understanding how AdWords works. Others focused on highly specific

objectives. They indicated that they “hoped to position our ads and choose keywords

with the highest possible click through rate,” they wanted to “effectively use all the tools

offered by AdWords,” and they wanted to generate more traffic to the client’s website.

The graduate students’ goals were similar to their undergraduate counterparts. They too

wanted to learn how AdWords works, “experience firsthand the nuances of a web-based

advertising campaign”, and apply their new knowledge to a “real life company.” They

also mentioned wanting to be able to interpret the results provided by Google, and to see

the impact of a campaign on a small company. In addition, these students mentioned

some specific objectives. They wanted to be able to select effective keywords related to

“selected landing pages” in the client’s site; they also wanted to learn to “how to optimize

keywords, how geographic locations impact impressions, and how to effectively manage

a set budget over a set time period.”

Being able to use Google’s online tools and benefit from the almost immediate feedback

associated with the tracking of their AdWords campaigns was, for most of the students, a

different kind of learning experience. Given that circumstance, the students may have

had difficulty articulating detailed learning goals for the GOMC project. All the teams,

however, indicated one or more learning goals, and appeared to expect that participation

itself would result in “learning.”

Q3. Is student work effort commensurate with learning expectations? As noted above,

the students had reasonably high expectations with regard to what they expected to learn

from participation in the Google Challenge. However, to benefit from the opportunity to

learn about Pay Per Click advertising and about the various online tools related to such

promotion, students had to be actively involved in the project. Also, because the online

environment constantly changes, the students needed to be vigilant in monitoring their

campaigns over a three-week period; they could neither complete the assignment the

night before its “due date,” nor “set the campaign and then forget it.”

Google’s rating of each team, which was based on the activity in the respective team

account, permits an important insight related to the degree to which students were willing

to put forth the effort needed to assure learning from participation in the Challenge. Of

the 29 teams considered in this study, Google ranked four as semi-finalists; these teams

were among the top 50 teams competing from the Americas region. Of the others,

Google rated one team “strong” or among the top 10 percent of teams not in the semi-

finalist group, nine teams “good” or in the 70

to 89

percentiles, six teams “fair” or in

the 40

to 69

percentiles, and five “needs improvement” or in the bottom 39 percent of

teams. Google also ruled four teams “ineligible,” because their campaigns either ran

longer than 25 days or less than seven days, or because they spent so little of the $200

that there was insufficient data “to allow for competitive algorithm calculation” (Google

2009).

Google’s Online Marketing Challenge Lavin, Marilyn

The broad distribution of ranking outcomes was based only on the effort put forth by the

teams, and suggests the likelihood that motivation varied considerably, even among

students doing experiential rather than traditional classroom exercises. Table 1 shows the

learning expectations of those teams that were ranked “semi-finalist” and those ranked

“needs improvement.”

The teams that Google identified as semifinalists generally did articulate better learning

goals. More specifically, the graduate students identified some highly specific objectives

and demonstrated a rudimentary mastery of the contents of course lectures and the

materials provided by Google when they mentioned learning about ad copy, ad

placement, campaign metrics, and relating ads to relevant site landing pages. By

contrast, all but one of the “needs improvement” teams named very vague learning goals,

including the graduate team that suggested wanting “to learn the general ins and outs of

how to run an online ad campaign.” The difference in presentation of learning goals

appears to be important, and suggests the possibility that those teams with the superior

outcomes had sufficient knowledge of what the “experience” would demand; Google’s

algorithm subsequently identified that they fully participated and were consequently

likely to have “learned by doing.” Those teams that were rated in the bottom segment of

Challenge participants do not seem to adequately thought about what they hoped to learn;

in keeping with this haphazard approach to the project, Google’s analysis of their account

activity suggests they put forth little effort and, as a result, learned little from the

experience.

Q4. Does an experiential opportunity result in better team participation? Recognizing

the importance of joint effort in the work environment, business instructors often build

team assignments into their course offerings. Unfortunately, many students resist such

activities, and they cite such problems as scheduling difficulties, free riding by some

members, and poor communication as factors that compromise both individual learning

and project outcomes. Unlike some class projects, the opportunity to run a three- week

AdWords campaign for an actual business client, budget and spend the allotted $200, and

see actual results from the team effort closely approximates an actual business experience

where co-workers would be expected to cooperate. Moreover, the Post Campaign Rubric

required students to consider group dynamics; most specifically it asked “what problems

did the team encounter and more importantly, how did they overcome these problems?

What were some of the expected and unexpected outcomes from working as a group?”

The students’ responses to these issues, as well as their evaluations of their team

members, offer means of understanding whether experiential opportunities are sufficient

to lead most students to give up the counterproductive activities too often associated with

team projects and provide some insight into the benefits of experiential learning.

The Google Online Marketing Challenge was designed to be group project. Google

recommends that teams be composed of at least three members, but does not suggest a

maximum limit. Likewise, Google does not indicate how teams are to be formed.

The 29 teams considered in this study each began with four members, although during

the course of the semester some teams lost members when illness and other issues caused

Google’s Online Marketing Challenge Lavin, Marilyn

some students to drop the class. Before teams were formed, students were free to identify

classmates with whom they wished to work, and with whom they did not wish to work.

The instructor honored these requests, and also grouped those students, who did not

express a preference, into teams.

Of the total 29 teams, 14 were from an online graduate class, 6 were from an online

undergraduate class, and 9 were from face-to-face undergraduate class. Across the

classes, however, student satisfaction with team members was mixed, and this variation

occurred among graduate and undergraduate students and in both the online and offline

modes of course delivery. Among online graduate students, group members in 3 teams

submitted evaluations indicating that all members contributed equally to the project; the

members of 5 teams noted slight variations in team member effort; and 6 identified

members who were laggards. Among the in-class undergraduate students, team members

of only one group suggested that all members contributed equally; 6 noted marginal

variations in effort; and two reported free-riders. And among the online undergraduate

teams, 2 reported equal involvements of members, 2 indicated slight variation in

participation, and 2 named persons whose participation was unacceptably low.

All of the teams that Google identified as semi-finalists had teams whose members

contributed equal or almost equal effort. Three of the semi-finalist teams were in online

classes. In their Post Campaign Reports, members of those teams noted the importance

of good communication, and mentioned that they utilized email, teleconferencing, and

Google Docs to coordinate their efforts. They also reported keeping logs of changes they

made in their accounts, and assigning specific responsibilities to group members.

Though the members of these teams never met face-to-face, these high performers

worked to assure to they worked cohesively, and one reported that by scheduling e-

meetings that accommodated all members’ needs, they provided “consistency and a

relaxed environment since we all knew we could make the meeting.” The fourth semi-

finalist team was in a traditional classroom setting, but like their online counterparts, this

group reported problems working “around everyone’s school and work schedules.” To

overcome these issues, they did schedule one weekly meeting, and they used Google

Docs to write and edit their reports; even so, they also acknowledged “trouble getting all

members of the group on the same page about AdWords.”

Google gave five of the teams considered in this study its lowest rating of “needs

improvement.” One of these graduate groups acknowledged “communication within our

group was extremely poor,” and attributed this problem to lack of organization. Another

graduate team reported that it “encountered problems with life, work and class conflicts”

and that “communication was difficult,” while the third poor performing graduate team

noted “the hindrance that comes with being in separate physical locations.” The

undergraduate online team that performed poorly similarly complained that

communication was difficult without “an allotted class time for us to meet each week.”

Interestingly, however, the only offline undergraduate team whose work was rated “needs

improvement” also noted that “it was difficult to coordinate schedules and set aside times

each week when everyone was available.”

Google’s Online Marketing Challenge Lavin, Marilyn

These results, which are similar to those often associated with other types of group

projects, suggest the conclusion that experiential learning may not lead to better team

participation. As noted above, the high performing team’s encountered problems and

found means to overcome them; the low performing teams faced the same difficulties, but

used them as excuses for low work effort. The opportunity to work for a real business

client, spend real money, and see the outcomes of their actions in real time does not seem

to have provided equal motivation for all students.

Q5: Do client businesses actively support the learning of student teams? Given the

promise of $200 worth of free online advertising, most SMEs, which were approached by

the students, eagerly agreed to participate in the Challenge. But in establishing the

Challenge, Google hoped that students would enjoy something of a consulting

relationship with their clients. Unfortunately that level of business involvement was not

the norm.

Of the 29 teams, six reported low or no effort on the part of the client. One group noted,

“we expected consistent willingness to work with us, didn’t get it.” The students

complained that their clients were slow to respond to questions or never answered them.

Several of the teams also recognized that the project was not a priority for the business,

or that the client was too busy to be involved. As a consequence, they developed ad

campaigns without adequate information about the goals of the business or what it hoped

to obtain with AdWords. Although these efforts did allow the students to use the Google

tools and analytics, they did not benefit from a true consulting experience in which they

would have used AdWords to improve the client’s performance. In fact, the students on

one team expressed their disappointment that the client never shared the results of the

campaign with them.

Fortunately, most of the teams reported good – and even outstanding – participation on

the part of the client. Most of the teams seem to have been satisfied if they had good

communication with the business and had their questions answered in a timely fashion.

A few, however, noted deeper client interest and involvement in the project. One team

had a client launch a new site to accommodate better its effort, another team expressed its

pleasure in the fact that the business changed website content and added better landing

pages, and a third was impressed that a manager “enjoyed hearing about what ads we

were working on, the numbers the ads we were implementing, and the daily changes we

were implementing.”

Conclusions and Implications

The Google Online Marketing Challenge presents an important means of understanding

student response to an experiential learning opportunity. With its Challenge, Google

allows students to learn about search advertising through running an actual campaign.

Equally important, the company’s ability to track student activity, its algorithm that rates

student use of its various online tools, and its grading rubrics permit educators to move

beyond student evaluation and client qualitative evaluation of the learning associated

Google’s Online Marketing Challenge Lavin, Marilyn

with an experiential activity, and to examine important factors related to learning

outcomes.

The current analysis confirms the importance of experiential learning. The opportunity to

do a real search advertising campaign for a real client as well as association with Google

prompted extraordinarily positive student response. And students undoubtedly gained

first-hand knowledge of the workings and analytics associated with AdWords that would

be either impossible or difficult to obtain in another manner.

But the present study also points to some limitations of experiential learning. Some

students were not able to articulate what they expected to learn from the Challenge;

others expected to learn, but did not put forth the level of effort needed to reach the goals.

In addition, the difficulties associated with teams on other types of projects were also

evident here, and some business clients did not provide students with the necessary

support needed to assure optimal outcomes.

These findings, both positive and negative, do not diminish the role of experiential

learning in the business curriculum. Major corporations like Google who provide such

opportunities do provide important means for many students to become better versed in

business practice. Similarly, companies that are willing to allow students to participate in

various tasks and projects offer students real insight about the workings of an actual

business. These are critical advantages that cannot be easily replicated with traditional

classroom exercises. But, in acknowledging the benefits of experiential learning, the

present research suggests that it is not a panacea nor is it likely to be associated with

equal learning outcomes for all students. Experiential learning, like all learning, requires

students to be motivated, work hard, cooperate with team members, and, where a client is

involved, be matched with a business willing to collaborate on the project. To this extent

that all of these factors are present, experiential learning is likely to produce highly

positive outcomes, but it must also be recognized that the experiential learning

opportunity itself is not likely to necessarily foster involvement, good work effort,

teamwork, or a positive client relationship.

Table 1

Comparison of Learning Expectations:

Teams Ranked “semi-finalist” vs. “needs improvement”

Ranking Learning Expectation

Semi-finalist (undergrad) Our group hoped not only to learn how to

successfully run an AdWords campaign, but also all

the aspects of search advertising and why it is so

successful.

Semi-finalist (undergrad) Our group was focused on learning the dynamics

that make Google and their AdWords program

successful.

Google’s Online Marketing Challenge Lavin, Marilyn

Semi-finalist (grad) Our overall learning objectives were to experience

firsthand the nuances of a web-based advertising

campaign. We were interested in learning how to

structure ad campaigns including ad copy, ad

placement, and campaign metrics as well as

developing client relationships.

Semi-finalist (grad) Our team’s goal was to learn how Google AdWords

functioned and how writing targeted paid search ads

can lead customers to a company’s website to

generate sales and brand identity. We wanted to

learn how to select effective keywords that were

related to the selected landing pages in effort to

promote the company

Needs improvement (undergrad) Our team hoped to learn how to effectively position

our ads and choose keywords to gain the highest

possible click through rate.

Needs Improvement (undergrad) Our team set out to learn as much as we could about

a real life Adwords campaign as well as grow the

presence of Chadwick's Surfaces International. We

wanted to learn, first-hand, just how effective an

internet campaign could be and see what the

potential was of such a basic Adwords campaign.

Needs Improvement (grad) We approached this competition hoping to learn

how to design a successful AdWords campaign and

to gain knowledge into how best to apply the

campaign to a real life company

Needs Improvement Our team wanted to learn how to set up and monitor

a campaign using Google’s Adwords tool. It was

important to know the various features available in

a campaign and how to use features such as

geographic targeting, keyword searching and

adjusting the bid on keywords. It was also

important to learn how to use the Adwords tools to

allocate a budget appropriately.

Needs Improvement Coming into this project our group wanted to learn

the general ins and outs of how to run an online ad

campaign, how to work with a group of people over

the internet, and how to try to predict how a target

market will act.

Google’s Online Marketing Challenge Lavin, Marilyn

Bibliography

Alsop, R. (2007). Recruiters’ top schools. Wall Street Journal, September 17, R5.

Gentry, James (1990). What is experiential learning? Guide to Business Gaming

and Experiential Learning. Retrieved from http://sbaweb.wayne.edu/~

absel/bkl/BG/BGa2.pdf.

Google Academic Guide. Retrieved from

http://www.google.com/onlinechallenge/professors_guides.html.

Google (2009). 2009 results. Retrieved from

http://www.google.com/onlinechallenge/2009results.html

Hart Research Associates (2008). How should colleges assess and improve student

learning? Retrieved from

http://www.aacu.org/LEAP/documents/2008_Business_Leader_Poll.pdf.

Hoover, D. and C. Whitehead (1974). An experiential-cognitive methodology in the first

course in management: Some preliminary results. Proceedings of the National

Academy of Management.

Munoz, C. and A. Huser (2008). Experiential and cooperative learning: Using a

situation analysis project in principles of marketing. Journal of Education

for Business. 83(4), 214-20.

Navarro, P. (2008). The MBA core curricula of top-ranked U.S. business schools: A

study in failure? Academy of Management Learning & Education. 7(1), 108.

Rosso, M et al. (2009). Using Google AdWords in the MBA MIS course. Journal of

Information Systems Education. 20(1), 41-50.

Stiller, E. and C. LeBlanc (2002). Effective software engineering pedagogy.

Journal of Computing Sciences in Colleges. 17(6), 124-34.

Wolfe, D. and E. Byrne (1975). Research on experiential learning: Enhancing the

Process. The Proceedings of the Second National ABSEL Conference,

325-26.

Biographies

Marilyn Lavin is a Professor of Marketing at the

University of Wisconsin-Whitewater. She focuses

her teaching and research primarily on Internet

Marketing and Retail Management. Her work has

appeared in such publications as the Journal of

Consumer Research, International Journal of Retail

and Distribution Management, and the Journal of

Retailing and Consumer Services. She is a member

of the Google Academic Panel.