Disclaimer:  The data used in this report is fictitious and in no way represents a real situation.  This report was written solely as an academic exercise and in an educational research class at Dakota State University.

A Study of Drill-and-Practice Software and its Effect on Student Achievement

Introduction:

Computer use in the teaching and learning of mathematics is being encouraged and research shows that it is beneficial (Manoucherhri,1999). This does not mean that all of the math computer programs that are being used are worth the money or the time that students spend using the programs. Certain types of math computer programs have a greater effect on student achievement than other programs. This brings us to an important question: Does the use of drill-and-practice software, such as Geometry (Intelligent Tutor, 1997), have a positive effect on student achievement?

Review of the Literature:

In a yearlong study by Roberts and Stephans (1999), their results found that computer software is not beneficial when teaching certain topics in Geometry. According to Alan Ginsburg (1998), "Studies by the Education Testing Service conclude that students who use computers for drill-and-practice activities are not as likely to be good at math as students who use computers for other purposes, like simulations and applications." Ethan Bronner (1998) even sites that drill-and-practice software has a counterproductive effect on students who spend long periods of time using the drill-and-practice software. This is due to the fact that many teachers who use drill-and-practice software do not have adequate knowledge about how to best use computers in mathematical instruction (Manoucherhri, 1999). This is not to say that all software is bad, but drill-and-practice software is not as good as other types of software.

Hypothesis:

Geometry does not have a strong positive effect on student achievement.

Method:

-Subjects:

Seventy-two students were involved in the experiment. These students were from five small school districts in South Dakota. A small school district is defined in this experiment as less than 125 students in grades 9-12. School Districts that were involved include Baltic, Chester, Lake Preston, Montrose, and Rutland. Sixteen students were from Baltic, nineteen were from Chester, twelve were from Lake Preston, fourteen were from Montrose, and eleven were from Rutland. All students in the study were enrolled in Geometry, using the same textbook and curriculum.

-Instruments:

The students were given a written evaluation at the start of the experiment to assess prior knowledge in Geometry. At the end of the nine-week period of the study, each of the groups took a written evaluation to assess the knowledge gained. Both evaluations were written by a cooperative effort of the five instructors involved in the study. Both evaluations were designed with the exact same format, as to not make the test format a variable in the study.

-Design:

This was a quasi-experimental design, with nonequivalent control groups. The subjects received the treatment depending upon which school district that they attended, not by a random sampling of all the students in the study. The control and treatment groups will follow a pretest-posttest format. The three school districts that received the treatment were randomly selected from the five schools by drawing their names out of a hat. The students who attend Baltic, Lake Preston and Montrose were drawn as the treatment groups. The students who attend Chester and Rutland were drawn as the control group.

-Procedure:

All five groups used the same curriculum and textbooks, and were given the same pretest and posttest. The instructors were given training on how to deliver the material in a consistent manner across the groups. The students in the treatment groups used Geometry, in fifteen-minute blocks of time, between two and four class periods a week. The variations in the number of times used during a week depended upon factors such as testing, school activities, school vacations, etc… During the same fifteen-minute blocks of time, the control groups received independent practice time for their math homework activities. The instructors did not inform the students that they were part of a study and the instructors did not assist the students with any questions during either the pretest or posttest.

Data Analysis:

A "t" test for independent means was used to compare the treatment and the control groups. A two-sample "t" test was used because the subjects resided in different communities. The data from the tests is in the table below.

The test resulted in a t-value of –2.19, which means that the control group actually did better on the tests than the treatment group. The test also resulted in a p-value of 0.035, which means we are confident that the results are valid and not due to chance.

As the data shows, Geometry does not have a strong positive effect on student achievement. The data actually showed a negative effect in using the computer program versus independent practice time for math homework activities.

It is important to note that this study only tested Geometry, not any other software programs. Other computer software programs may have a different outcome under the same study. Another important factor that needs to be considered is the treatment subjects were not picked randomly over the total sample of the seventy-two subjects involved in the study. Socioeconomic factors for each particular community could have had an effect on the outcome. Even though the teachers were trained to teach in the same manner, individual teaching styles may have made a difference in the outcome.

This study may appear to reject the use of computers in the mathematics classroom, which is not the case at all. This study points out that some computer programs are ineffective and should not be used. This study needs to be expanded to other drill-and-practice computer programs and other types of math classroom related programs, such as simulation and application programs.

 References:

Bronner, E. (1998, September 30). Computers help math learning, study finds. New York Times,
            p. 20.

Ginsburg, A. (1998, October 21). Test scores and computer use: Longitudinal studies needed.
        Education Week, 18, 38.

Intelligent Tutor. Geometry. (Windows 95), Munster, Indiana: Intelligent Software, Inc., 1997.

Manoucherhri, A. (1999). Computers and school mathematics reform: Implications for mathematics
        teacher education. The Journal of Computers in Mathematics and Science Teaching, 18,
        31-48.

Roberts, D. L., & Stephens, L. J. (1999). The effect of the frequency of usage of computer
        software in high school geometry. The Journal of Computers in Mathematics and Science
        Teaching, 18, 23-30.