Robert Shepherd says:
One cannot tell [how sophisticated the Xerox automated essay grader] is from the marketing literature, which is a concatenation of glittering vagaries. But even if one had a perfect system of this kind that almost perfectly correlated with scoring by human readers, it would still be the case that NO ONE was actually reading the student’s writing and attending to what he or she has to say and how it is said. The whole point of the enterprise of teaching kids how to write is for them to master a form of COMMUNICATION BETWEEN PERSONS, and one cannot eliminate the person who is the audience of the communication and have an authentic interchange.
via http://dianeravitch.net/2013/05/16/can-machines-grade-essays-should-they
Langdon Winner talks about looking at technology and thinking seriously about what happens when the technology works. That is, we spend a lot of time criticizing technology for its failures and not enough time criticizing technology when it works. In the case of automatic grading, we can spend our time thinking about technical imperfections (many of which will eventually be ironed out), but what are the costs when it works just like it is supposed to?