This presentation was delivered at the 2014 Conference on College Composition and Communication in Indianapolis, IN, March 24, 2014.

Automated essay scoring (AES) is the use of specialized computer programs to assign grades to essays written in an educational setting. It is a method of educational assessment and an application of natural language processing. Its objective is to classify a large set of textual entities into a small number of discrete categories, corresponding to the possible grades—for example, the numbers 1 to 6. Therefore, it can be considered a problem of statistical classification (Wikipedia, “Automated essay scoring”).

As we know, the conversation on machine grading isn’t new. It began in 1962 with Walter Reitman’s thoughts on the possibility of machine grading at a conference on needed research of teaching English. Most historical summaries of AES trace the origins of the field to the work of Ellis Batten Page. In 1966, Page argued for the possibility of scoring essays by computer, and in 1968 he published his successful work with a program called Project Essay Grade (PEG).

Multiple-choice exams have, of course, been graded by machine for a long time, but essays are another matter. When it comes to English composition, the question of whether computer programs can reliably assess student work remains sticky. Can a computer program accurately capture the depth, beauty, structure, relevance, creativity, etc., of an essay?

Douglas Hesse writes that when it comes to the resulting problems of cost and frustration associated with traditional writing assessment, he worries more about how computer grading will negatively impact “the transferability of skills and about the cost to how students perceive writing (and the resultant cost to human culture).”

Amid the pressure many composition teachers and their institutions face in regards to the possible adoption of computer-grading programs for assessing student writing, many teachers have strong opinions – most of them disagree with the employment of grading machine in their classroom. In this pressing time, I think it is important for teachers, as well as graduate students, to understand the impact on pedagogy, process, and product that grading machines may have if they replace human readers. Hence, by providing an overview on the appeals of computer grading programs and the ongoing debates around the adoption of automated essay scoring software for assessing writing, this presentation aims to illustrate how mechanized evaluation of writing may mechanize pedagogy as well as the process to create a written, machine-directed product. I will also offer suggestions for writing instructors in handling the adoption of essay-grading programs in their classrooms.

Read the full conference paper here.

View the conference presentation here.