The release to the public last November of ChatGPT, followed by even more sophisticated artificial intelligence tools, has understandably been met with fascination and no small amount of dread. Will the robots come for all of our jobs? Will they kill us?
Some of the hype is undoubtedly just that. Prior technological innovations were also predicted to wipe out human employment but mostly just shifted it. The internal combustion engine greatly reduced the need for farriers but generated a large new market for auto mechanics. Reliable telephones and (eventually) internet-based email and phone chat all but eliminated the need for telegraph operators but created vast new categories of employment. For all we know, the discovery of fire caused consternation among those cave people who had theretofore earned their share of their tribe’s bounty by hugging others to keep them warm.
Even so, some of the fears seem justified. Autonomous weapons systems piloted by AI pose a potentially existential threat to life on Earth. Even if the robots do not turn against us as in The Terminator, The Matrix, or countless other dystopian science fiction visions, AI weapons could trigger alerts or miscalculations that lead fallible humans to launch nuclear weapons. True, just as autonomous cars could ultimately prove safer than human-driven ones, so too, AI soldiers might be more cautious than humans, but there are enough possible scenarios in which something goes wrong to worry that AI warfare will add to the total risk to our species.
Impacts on employment and the potential existential risks of AI remain speculative, at least in the short term. They are also beyond my claim to any expertise. Accordingly, in the balance of this essay, I focus on a field I know well: education. I have spent nearly my entire life as a student or (for the last 31 years) a teacher. Already we are seeing major shifts in reaction to the new AI tools.
Exams
I began full-time law teaching in 1992. In most years, I teach a seminar in which students write papers (more about that below) and two “doctrinal” classes—these days almost always constitutional law (which Cornell requires all JD students to take in their first semester of law school) and federal courts (an upper-class elective). Although there are some opportunities for students to practice their writing in these classes, I have almost always followed the law school tradition of basing grades entirely on a final examination (with the possibility of a ½-grade adjustment based on class participation).
Up until now, I have always given open-book take-home exams. I deliberately gave students plenty of time to read the problem, think about it, outline, write, and revise, all while taking breaks. For some years, I allowed students 24 hours in order to relieve any time pressure they might feel. However, I learned that too many of them ignored my instructions and spent nearly the entire 24 hours working—which probably did not improve their performance and was certainly bad for their physical and mental health. Thus, for a long time I have given students eight hours. As I typically explained, it’s not very often that they will be asked as lawyers to write a brief in three hours with no ability to do any research, but they will sometimes need to produce one in a single working day. My goal was to design an exam that would test (and thus encourage the learning of) skills that are useful for lawyers, not just those useful for law school exam takers.
I have reluctantly concluded that the availability of even moderate quality AI tools requires me to change my exam format. As my Cornell and Verdict colleague Joseph Margulies explained in a column a couple of months ago, with AI now capable of doing work good enough to earn a B (and likely to improve), the temptation could be too great for some students to resist.
To be sure, even before AI, students had the opportunity to cheat on my exams. They could consult with one another or even pay someone to take their exam for them. Doing so would of course violate the instructions and the law school’s honor code, but I had no fool-proof means of detecting such violations. Even so, my impression was that there was not widespread cheating. How do I know? It’s a hunch based partly on the fact that the students who did well on my exams were typically the same students who did well on the closed-book proctored exams administered by my colleagues.
So if I trusted that nearly all of my students would not cheat in the pre-AI world, why do I think it’s more likely that they will do so now? For one thing, old-school cheating required at least one confederate—who could then report a cheater. For another, I suspect that students will increasingly come to think of utilizing AI tools as not cheating—especially as those tools are integrated into search engines. In the early days of the internet, people who would never dream of walking into a brick-and-mortar store and stealing a music CD routinely used file-sharing services like Napster to accomplish the same—and also unlawful—thing.
Papers
Although hardly ideal, my solution to the AI problem with respect to exams is relatively straightforward: beginning in the coming semester, I shall administer “closed-network” exams. I will permit students to consult their notes and casebook but not AI tools. Thus, at least for my students, the immediate future of exams will look a lot like the pre-internet era when I was in law school.
Papers—the subject of the earlier essay on this site by Professor Margulies—present a thornier question. One can hardly require that proctors oversee students during every moment during which they work on their papers. Nor does it make any sense to forbid students from using online search engines as part of their research. Yet as AI becomes integrated into search tools (as Microsoft has already integrated it into Bing in chat mode), it will become increasingly difficult to prevent students from utilizing the large language models that drive AI without thereby also preventing them from doing effective research.
Nonetheless, for the coming semester at least, I plan to supervise papers chiefly in the way I always have done in the past. Students will need to submit topics, outlines, and drafts for feedback en route to crafting original research papers. I suppose it will eventually be possible to ask an AI model for each of the foregoing. Indeed, as an experiment, I asked ChatGPT for a list of ten potential paper topics for the seminar I’ll be teaching in the fall. It came up with a reasonable assortment. I then asked for a detailed outline of one of the papers. Again it performed reasonably well. However, when I asked for the paper itself, it produced wooden prose that mostly talked around the issue rather than engaging. I have little doubt that an AI detector would ring the alarm. I also thought the paper not very good.
Accordingly, and for now, I consider myself warranted in thinking that smart, well-motivated students will outperform AI in the coming semester. Thus, I am also comfortable using my old rules for my seminar.
Teaching and Practicing with AI
There will come a day—perhaps very soon—when AI tools will be capable of outperforming even excellent law students. At that point, further resistance to AI may prove futile, and not just because excellent AI-generated papers will be as good as any a law student can produce (but not so much better or different as to trigger alarms from AI-detection software). At that point, it might still be possible to devise means of preventing students from cheating by having AI write their papers. But even if it is possible, would it be sensible to continue to try to frustrate AI-assisted writing?
Consider how standardized tests of mathematics and some related fields addressed calculators. When personal calculators began to become widespread in the 1970s, test administrators initially forbade students from using them in proctored exams. By the 1990s, they had adjusted. For example, since 1994, the SAT has permitted students to use calculators on some portions of the math exam.
And with good reason. Although the ability to calculate a twenty percent tip in one’s head can be useful, it is hardly crucial, given that the smartphones carried by nearly everyone who can afford to pay for services in developed countries include a calculator app. Meanwhile, for any reasonably complicated calculation, a calculator saves time and effort while increasing accuracy. Given the ubiquity of calculators in “the real world,” continuing to forbid their use in all test-taking contexts would make as little sense as requiring motor vehicle drivers to demonstrate proficiency in horsemanship to obtain a driver’s license.
As AI becomes integrated into more and more of the productivity tools that lawyers and other professionals routinely use, banning its use in the taking of exams will make less and less sense. Eventually, we will need to learn how to evaluate students’ ability to use AI and other tools effectively. More importantly, we will need to figure out how to teach students to use AI tools effectively.
Before any of that can happen, however, we will need to see how lawyers use AI in practice. It’s sensible for professional education to aim at preparing students for the profession they’re entering. It’s challenging to do so when the nature of that profession is changing rapidly. AI-based tools for managing document discovery have been available to litigators for years, but how practitioners come to use the new generation of large language models and whatever comes after that remains to be seen.
In the meantime, legal educators will likely continue to regard AI with the same mix of fascination and dread that the general public feels. For my part, I expect to teach the same skills of critical thinking and precise expression that the legal profession has long valued. If nothing else, they could prove useful whenever the network goes down.