1. INTRODUCTION

This paper is an outcome of an innovative education program designed to measure the impact of experimental pedagogy among management students. The main result of the paper goes further in the way we teach to our students in management and business schools. In this sense, many papers have analyzed learning process in managerial work. In particular, ‘natural learning (Burgoyne & Hodgson, 1983) differences among three levels of learning, keeping the higher level to individuals reflecting on and questioning other ways to find the managerial solution, included changes in behavior. Research in educational and learning management has become more prevalent over time and its use continues to increase. The seminal paper of Rousseau and McCarthy (2007) analyzes how evidence-based management promotes the best available scientific evidence to properly inform managerial decisions and organizational practices.

The recent inclusion of the experimental methodology in management education has increased in the last years. The goal of the experimental methodology is to approximate a specific economic context, managerial situation, or business process to the students using experimental methods in an applied way. In this sense, classroom experiments (Holt, 1999) include one or some theoretical aspects directly related with the teaching program of the course. In this particular case, this classroom experiment is based on the field of macroeconomics (see Duffy, 2010) and implements the Phillips Curve theory, based on the trade-off between inflation and unemployment levels. The experiments complement the current class in order to give students the possibility of implementing the theoretical concepts into a real situation, being part of a decision-making process, in which their decisions are linked with an expected payoff (subjects earned mark points for their responses to their final exam). Thus, students participating in the Phillips Curve experiment react differently to theoretical concepts and improve their acquired knowledge evidenced by their decisions. In particular, the classroom experiment consolidates the theoretical concepts using ‘active learning and increasing the students motivation toward the final subject (Emerson & Taylor, 2004). The main finding of the paper significantly suggests large scores in the final exam grades for the students under the classroom experiments. The manipulation is following the standard experimental procedure, one intervention group, one control group, controlling for a rigorous methodology. Regardless of the fact that students belong to the same class, the author runs the experiment according to the strict experimental procedures and protocol, keeping the privacy and anonymity of decisions and payoffs. The experiment was designed in order to follow the role of a classroom experiment, as well as giving some empirical evidence founded in it. The remainder of the paper is organized as follows.

The next section is devoted to an overview of related literature. After, the next section describes the design and methods. Furthermore, next section provides the main results and examines them. And the final section concludes including further research and study limitations.

2. LITERATURE REVIEW

The experiments have been traditionally used in natural sciences, such as biology, chemistry, and physics. Economics, on the other hand, has not been considered an experimental science. However, from the last decades, experiments in economics and management education have been substantially increased. In particular, some experiments have been implemented in class to analyze active learning methods for students in many different topics such as pedagogy, entrepreneurship, pharmacology or biology (DeTienne & Chandler, 2004; Gleason et al., 2011; Holt, 1999; Micheletto, 2011; Mora, 2017; Stoltzfus & Libarkin, 2016; Taraban et al., 2007).

Although a vast majority of pioneer experiments were done in classroom settings, their main purpose was not pedagogical. Instead, pioneer scientists in this area were trying to test theoretical claims to improve the science of economics. In 1948, Chamberlin conducted the first classroom market experimentation using economic theory students from Harvard University. The classroom experiment was designed to practice a monopolistic competence problem, which assigned buyer and seller roles to the students in order to understand how the supply and demand theory works. Following Chamberlin, the use of experimental and behavioral economics in the classroom has been progressively increased. The number and impact of laboratories in experimental and behavioral economics using incentivized students has been increased exponentially.

However, this paper focuses on the use classroom experiment as an experiment used in class (see Holt (1999) to grasp the origins and uses of classroom experiments). The main purpose of this kind of experiment is to show a theoretical approach through an experiment performed directly by the students. The pedagogy is based on active learning instead of chalk and talk. Kolb and Kolb (2005) describe how experimental learning can be applied in the educational learning processes. In this line, the goal of this paper is to analyze the effectiveness of classroom experiments as a tool of pedagogical innovation among management students.

To better understand the relation between classroom experiments and motivation, there are some papers using qualitative methods explaining the motivation of students involved in classroom experiments. Holt (1999) argues that classroom experiments enhance the effectiveness of Socratic methods as opposed to traditional lectures. Emerson and Taylor (2004) discuss how classroom experiments increase the positive attitude of students towards the study of economics. These papers support the assumption that classroom experiments motivate students to study the theoretical concepts involved.

Moreover, there exist several papers exploring the effectiveness of classroom experiments. Frank (1997) gives evidence about the effectiveness of a classroom experiment. In particular, students participating in the classroom experiment were more successful in answering a multiplechoice test on the tragedy of the commons than other students from the same courses not participating in the classroom experiment. However, there is limited evidence to suggest that classroom experiment participation improved test scores.

In the field of macroeconomics, Gremmen and Potters (1997) organize a multi-day international economic simulation game. Authors split their students in two groups, playing the game during the semester or following the traditional courses. Results confirm higher scores in the posttest for the group playing the game.

Additionally, Dickie (2006) examines whether the use of classroom experiments with the use of grade incentives affect learning of introductory microeconomics, measuring the partial effects of experiments independently of instructor quality and teaching methods using Test of Understanding in College Economics (TUCE) scores (Saunders, 1991). The author found positive and significant effects on student learning whereas grade incentives do not. Furthermore, Cardell et al. (1996) examine a much large sample of students (1,800 students) finding no statistically significant impact of the experimental approach on student achievement using TUCE scores too.

Other seminal papers follow the same question; Durham, McKinnon and Schulman (2007) provide evidence about the profit obtained by students using experimental methodology in their courses, both for microeconomics and macroeconomics. The size of the impact varies with preferred learning style. In particular, read-write learners get less profit than learning-by-doing students from classroom experiments. In the same line, Emerson and Taylor (2004) find that exposure to classrooms experiments positively influences student attitudes towards the study of economics, correlating the higher performance obtained in TUCE with certain students characteristics such as gender, major, and grade point average. Ball, Eckel and Rojas (2006) find a positive and significant impact of computerized classroom experiments in the academic performance of students, even more for women and first year students. Arroyo-Garcia and Zeledón-Ruiz (2014) used experimental techniques on students to construct meaningful learning by autonomous learning process.

On the other hand, Swarthout and Dufwenberg (2009) analyze how classroom experiments enhance students ability to analyze the game using game theory. Authors present two games teaching the concept of Nash Equilibrium. Despite the prior literature, authors found no differences in the fact that students played the games before answering an exam question about Nash Equilibrium.

Table 1 summarizes the literature review of quantitative studies analyzing the impact of classroom experiments.

Thus, the previous literature enforces the assumption of the classroom experiments as an innovative pedagogical tool to take it in consideration for being part of the pedagogical process. Nevertheless, the cost of applying classroom experiments will be offset by the benefits of the students as we have learned from the majority of studies mentioned above. Classroom experiments are an important educational complement for the theoretical lectures. The vast majority of results obtained at these studies provide information about the positive effects of this methodology for the students. In particular, these studies are dealing to improve the motivation and the academic performance of students participating in the experiments. The use of the experimental methodology as a teaching tool to facilitate learning processes arouses the interest of students towards the subject. Nevertheless, we should ask which elements make classroom experiments more productive, such as the role of assessment (Durham et al., 2007; Balkenborg & Kaplan, 2009; Cartwright & Stepanova, 2012). We propose the following hypothesis:

In summary, the design of this paper will provide an appropriate measure of the impact of classroom experiments in a macroeconomic field using a proper design. Finally, as the study is based on pure macroeconomics theory, and the majority of experiments are related to microeconomics concepts and less to macroeconomics, this study emphasizes this point as an additional originality.

Following the previous literature, this paper introduces two additional values for this purpose:

First, this paper proposes a clear environment to measure the impact of learning techniques using experimental economics on the academic performance of students, adding several crucial factors in the design to better measure the effects of classroom experiments in the academic performance of students. This clear design has different components, such as: groups were randomly assigned (no recruitment bias), the lecturer and the contents explained were exactly the same across groups (no pedagogy differences), and an external committee did the exam corrections to avoid the psychological bias in the process (no psychological bias in the corrections)¹ .

Second, the design originally presents a Phillips Curve experiment directly related with the contents of the macroeconomics course for management students. The purpose of the experiment was based on creating a link between the classroom experiment and the Phillips Curve theory through a simple. The contents of the game were directly related to one question of the exam. The students were specifically asked about the theory on the final exam with a total weight of 30% of the final mark.

3. DESIGN AND METHODS

The course was implemented at the University of Valencia, and the classroom experiment in the experimental laboratory of the University of Valencia (LINEEX) during the academic course 2011-2012. The experiments involved undergraduate students from business and management degree (third year), a total of 167 participated in the experiment, 49 students in the Intervention group and 118 students in the Control group. None of the participants had previous experience in this type of experiment and only participated once. Experiments were run in a computerized environment using Z-tree (Fischbacher, 2007), following the standard methodology in experimental economics. The experiment took less than 90 minutes to be run organized in two sessions, including the explanation of the instructions and results². The experimenter controlled all the experimental procedures, including strict anonymity and private payoffs through academic incentives.

The design of this study highlights a clear scheme proposed to better understand the relation between classroom experiments and the academic performance of students. For an overall picture, Figure 1 describes the design and experimental procedures.

Specifically, there are some highlights to consider to better understand the design:

The lecturer randomly divided the group in two parts, Control group and Intervention group, assuming the Intervention group as the group to study. More precisely, the lecturer randomly assigned 50 numbers from the whole list of students to participate in the laboratory experiment. Due to the physical restrictions of the experimental laboratory at that moment, the maximum acceptance number of participants was 50 participants per session. A total of 49 students out 167 were randomly recruited and participated in the experiment. Thus, the Control group is formed by 118 students. No initial differences in the academic performance between two groups can be observed³.

The lecturer of the group took the same time explaining the concepts in the blackboard for both groups. All the theoretical contents about the Phillips Curve were explained before the implementation of the experiment and the execution of the exercises. These two tasks (experiment and exercises) were time equally distributed and the contents were exactly the same for the two groups.

The contents before the experiment and the exercises for both groups were uniformly compensated: the characteristics and the timing invested in the two groups were identical. Additionally, to avoid a bias derived from an over-tasking process in the Intervention group, the lecturer proposed an extra task for the students allocated in the Control group, based on calculate the theoretical solution for each level of inflation in the game. In this sense, the same time that the Intervention group spent playing the experiment, the Control group was solving exercises about the Phillips Curve, and directly related with the contents included in the final exam. This fact allows us to avoid the experienced bias between both groups according to the time expended playing the experiment.

Furthermore, the purpose of the experiment is motivated on an educational target. In other words, the experiment was designed to practice the Phillips Curve theory in the class. In the experiment, students earned points, that is, the incentives in the game were based on academic incentives. According to the answers of the participants, the payoffs from the experiment were converted into direct points at the end of the experiment. In particular, the participants had the straight opportunity to increase until 0.4 points (out of 10) the marks from the final exam. To avoid a potential bias derived from this fact, the lecturer gave the same amount of points for the Control group to achieve the Phillips Curve exercises (calculate the theoretical solution for each level of inflation in the game). Note that in the two groups the final points were related to the relative performance of the tasks.

Finally, once the course finished and all the students answered the final exam, the exams were revised according to the current university regulation⁴. To avoid the unconscious psychological bias from the fact that the lecturer knows the purpose of this study, an external lecturer corrected the exams without any knowledge about the purpose of the study. Thus, the academic performance of the students measured through the final exam had no correction bias effect according with the average of other years and groups.

As we have observed before, the design of the experiment presents some highlights that reinforce a clean analysis of the academic performance of students using a classroom methodology. In this sense, we count with a clear environment to measure the real impact of classroom experiments on the acquisition of theoretical knowledge by students.

4. RESULTS

This section provides some statistics of the data already collected giving an explanation about the main research question presented in the paper. The main evidence is explained in Table 2, which resumes the academic performance of students in both groups.

Table 2 shows the academic performance of students in the Intervention group and the Control group showing the score obtained in the Phillips Curve question exam (from 0 to 3 points) (see Ver pdf: ANEXOS. The results are strongly different, displaying a remarkable score of 1.71 in the Intervention group and 1.19 for the Control group. The non-parametrical test (Mann-Whitney test) shows a significant result at 0.14% (p-value<0,0014) , this result is extremely robust according to the size of the sample (49 for the Intervention group and 118 for the Control group) and the nature of the non-parametrical test. The Hypothesis 1 is not rejected; the Intervention group has a positive impact in the academic performance of students, reaching higher points in the final exam.

To represent graphically these differences, a Kernel Density graph is presented in Figure 2. By simple inspection, we can identify that the distribution of mark points is reasonably different between groups. Thus, while the Control group displays higher density of distribution for the lower points segment, the density of distribution declines when the points increase. The opposite occurs for the Intervention group, displaying higher density of distribution in the right part of the portion, that is, for the best marks reached in the question exam.

The paper presents two secondary results differentiating by gender and experience of the students. Gender differences are commonly tested in education research. In the sample, the overall descriptive statistics present 47% (st.dev. 0.50) of females at intervention group, and 57% (st.dev. 0.50) at control group. Figure 3 presents differences between groups across gender. While we can observe positive and significant results for both males and females (p-values< 0.0213 and 0.0347 respectively) in the Intervention group, there are no significant differences by gender between Control and Intervention groups (p-values< 0.6115 and 0.8161) The experiment has no different effect by gender, both males and females reach better performance in the Intervention group by showing a similar level among groups.

To introduce a further step in our analysis, we test whether the experimental methodology has a different impact across different students. Despite the students being in the same course, the level of knowledge is not homogeneous. To keep the effect of classroom experiments among a heterogeneous sample, we can control students according to the individual experience within the theoretical topic, categorizing between ‘experienced and ‘non experienced students. In this sense, ‘non experienced students had no previous experience in the Phillips Curve theory, meaning that it is the first time attending a macroeconomics course with the Phillips Curve model included in the content of the course. However, ‘experienced students have learned this model in the past, that is, they have failed the exam or they never filled the exam in previous years. In particular, the overall descriptive statistics present 40% (st.dev. 0.49) of experienced students at intervention group, and 21% (st.dev. 0.41) at control group.

Academic data collected from the university reveals that students typically obtain better marks if they successfully pass the whole academic year, remaining as ‘low performance students those of them who failed the exam and repeated the subject with the experience obtained in the previous year. This analysis introduces a further step attending to the previous experience of subjects, and the effect of classroom experiments on different kind of students. Generally speaking, we can drive our categorization from ‘experienced and ‘non experienced, to subject ‘retakers and ‘non retakers.

Figure 4 shows the positive effect of the experimental methods in experienced students. In particular, while experienced students present marginally significant low marks than non- experienced students in the Control group (p-value< 0.1058) we obtain a different result in the Intervention group: both types of students obtained the same level of marks (p-value< 0.8950). Thus, the experiment has a positive effect on experienced students reaching high levels of performance. The performance increase is 33.1% for non-experienced students and 61.4% for experienced students⁵.

Furthermore, Figure 5 presents a kernel density graph representing the effectiveness of the experiment in experienced students. The experiment has an equity effect; independently of the students condition both types of students reach the same level of performance. Thus, classroom experiments are presented as an effective tool to reduce the gap between ‘non experienced and experienced students.

These secondary results are comparable with the paper of Ball, Eckel and Rojas (2006), which concludes that women and first year students increase the academic performance even more than men. Next section discusses the main findings.

5. CONCLUSION AND MANAGERIAL IMPLICATIONS

In this study, students participating in a classroom experiment about Phillips Curve principles achieved better academic performance in the final exam compared to a control group. The differences were statistically significant at 0.14% between the two groups (p-value< 0.0014). Additionally, the paper presents a clear design with some advantages to correctly respond to the research question, displaying an adequate design to avoid any psychological bias from the experimenter.

The main result of the paper is consistent with the previous literature about learning methodology. Students playing the game reinforce their positive attitude towards the study of the subject (Emerson & Taylor, 2004). Classroom experiments enhance learning-by-doing pedagogy and it is already presented and tested in the work paper; the results identify how classroom experiments, as innovative pedagogical techniques, are better to improve the academic performance of students. As many other experimental studies from the literature review, our findings support the importance of experimental methodology for higher education, by tackling complex conceptual topics in social sciences courses. The core of the paper is the positive impact of classroom experiments on academic performance. The secondary results about gender and ‘experienced students are different from the results obtained in Ball, Eckel and Rojas (2006). Our results show no gender differences, and a performance improvement of the ‘experienced‘ students.

The results suggest that students view in-classroom experiments more favorably than the traditional courses. Considering the increased academic performance showed by students in the experiment, educators should consider this pedagogical tool as a mandatory tool of any economic and management course when the contents can be implemented in the lab.

Further research in this area is needed considering that the students follow a kind of motivation when they are studying the topic or doing the homework. It seems that students can motivate themselves remembering their personal experience in the lab to believe that they can understand the question. At the university level, the topics related with the Phillips Curve are historically correlated with low performance, directly translating this fact into low marks levels according to the whole exam. For instance the classroom experiment enhances the motivation to study and to understand the topic involved. However, the way by which the students increase motivation toward the subject is not fully understood. Further research is needed to cover the individual source of motivation toward the topic.

Finally, the satisfaction of students and even teachers should be taken in consideration, in particular with the implementation of experiments, as innovative education management techniques, in comparison with more traditional methods. Further research needs to provide evidence between satisfaction and the use of innovative practices to explain academic results.

Like any other study, this research article presents limitations. A general limitation of the work appears when taking in consideration the nature of the experimental population formed by students.

There are many factors influencing the motivation of students, and it is very difficult to isolate them by implementing a simple experiment. The source of motivation comes from both intrinsic and extrinsic factors. We can easily imagine a student investing time and effort into an important piece of study because it was signaled as a crucial one to validate the exam. Or, on the other hand, the student could increase time and effort towards this piece of study because the experiment helped her to better understand the topic. The results of this study do not make any difference by different sources of motivation; however the results are clear by differentiating between experimental methodology and traditional methods when measuring academic performance.

References

Experimental Instructions (translated from Spanish)

The purpose of this experiment is to study how individuals make decisions in certain contexts. The instructions are simple and, if you follow them carefully, you can directly earn final marks at the end of the experiment. The final amount of marks will be showed in a confidentially way, no one will know the payments received by the other participants. If you have any question, please raise first your hand. Out of these questions, any communication between you is forbidden and subject to the immediate exclusion of the experiment.

Exam question about Phillips Curve (translated from Spanish)

APPENDIX II

In Question 3. (3 points)

Consider the following information for an economy:

Suppose now that in period t a new chief of the Central Bank with a tough anti-inflationist reputation is elected and he proposes π^e _t =0.8 π_t-1. According to this new situation, follow the exercise proposed in e) obtaining the new results in t+1 and t+2. Finally, compare them with previous ones.

Publication Dates

History