There have been several interesting conversations about legal scholarship in the past couple of days. They started on blogs, but I understand they have also generated 280 characters at a time of what, given that format, must surely be rich, thoughtful discussion elsewhere. Taken together, they raise some interesting questions. Although most of the posts are about scholarship in general, some of them focus on scholarship and AI, and AI arguably figures as a complicating factor in all of them. I hope to get around to all of them, but rather than try to do it all in one gulp, let me start with just one of those posts.
It comes from Larry Solum, whose post Wednesday at the Legal Theory Blog castigated social-media commentary on recent birthright citizenship scholarship for exhibiting “a very uncharitable attitude towards scholarship with which the poster disagree[s].” He continued:
My view of the role of Legal Theory Blog is premised on the idea that scholarship ought to be viewed as a search for truth and that scholarly debate should be civil and charitable. Attempts to shame or silence scholarship are always problematic and rarely justified. Statements to the effect that the author of a piece is insincere without any evidence should be condemned and have no place in responsible scholarship. What should matter is what Jürgen Habermas called “the unforced force of the better argument.”
I mostly want to focus on sincerity here, but let me say first that I’m not sure why attempts to “shame” scholarship are “always problematic and rarely justified.” Maybe Larry means that the attempts are problematic because they’re rarely justified. Scholars often overestimate the strength of their criticisms and/or the weakness of their adversaries’ views, and the error rate makes shaming problematic as a practice. I sympathize. But surely there is plenty of bad scholarship in any academic field–bad in intent, bad in design, bad in execution. The direct and indirect harms of bad scholarship are numerous. It enables error to burrow in; it muddies the waters and raises search costs; cumulatively, it devalues good and bad scholarship alike. Not least, because those who do bad scholarship “well” can end up in influential positions, it entrenches and encourages the same bad moves, especially but not exclusively by junior scholars. I would have thought, therefore, that shaming bad scholarship is often problematic but frequently justified. To take one of many possible examples, American law reviews cite plenty of work taken from other fields. Yet neither American journals nor American law professors universally run citations through Retraction Watch. I would think shaming is in order, both because this failure is literally shameful and because the shame might lend support to better individual and collective practices.
I sympathize, too, with Larry’s assertion that calling an author insincere “without any evidence” is wrong and irresponsible. Or at least I mostly sympathize with it. The problem with such accusations isn’t that they’re always wrong, and I don’t read Larry to suggest otherwise. It’s that they are thrown around too casually, given our lack of access to others’ minds.
Accusations of insincerity often rely on the fallacious proposition that one can arrive at a reliable conclusion about someone else’s motives by reasoning logically from a small number of facts and (contestable) premises–e.g., “It’s so sloppily done, or the author is so new to the discussion, that it must be insincere, conspiratorial, etc.” That sort of argument can be described as relying on “evidence.” But it’s weak evidence, and most of the real work in such an accusation is done by the questionable inferences, not the actual evidence itself. Confident accusations of insincerity often ignore mixed motives, and they ignore the reality that people are often imperfectly aware of their own motives. In practice, accusers are often cynical about the motives of their adversaries, but credulous or silent about their own motives or those of writers with whom they agree. This does not make the accuser’s charge of insincerity wrong, but it may affect the accuracy of the general picture of the debate and its participants.
One defense of accusations of insincerity is that they provide an important part of the context for understanding a debate–about scholarship, about politics, or what have you. On this view, if a prim insistence on “civility” demands the exclusion of such charges, then it is effectively legitimating and benefiting bad-faith actors and distorting reality. Against this, and even if one ignores the value of civility as such, is the possibility that the number of errors, faulty reasoning, and corrosive habits that one avoids by barring weakly evidenced charges of insincerity outweighs the epistemic value of permitting those accusations. Some bad actors will get away with their insincerity. But the gains outweigh the costs.
That’s the position I ultimately take and one that I try, with epic levels of imperfection, to apply. Nevertheless, after reading Larry’s post, I find myself perversely doubting this position. Larry confines himself to criticizing charges of insincerity made “without any evidence,” while leaving well-founded accusations of insincerity on the table. My sense is that most people, wherever they come out on the norm that should apply to speculative accusations of insincerity, would agree that criticizing insincere legal scholarship when one has strong evidence for that criticism is at least fine and perhaps obligatory. But despite my own general view, I have to wonder: Why should we care about insincerity at all? A charge of insincerity clearly carries weight in the world. If it didn’t, we wouldn’t bother to make it–or to insist that one not make it without strong evidence. But why should it matter?
Following Habermas, Larry argues, against allowing casual charges of insincerity, that all that matters is “the unforced force of the better argument.” If that’s so, why exactly should we care if an argument is made insincerely, even when there is strong evidence of this? Whether an argument is insincere is wholly distinct from the question whether it has been made shoddily, badly, illogically, without convincing evidence, and so on. Full many an honest and sincere actor has written scholarship that fails all those tests. And surely some number of rotten, corrupt, self-serving, office- or profit-seeking, cause- or ideology-driven, mercenary, and otherwise shitty individuals have written excellent articles for bad motives and without an ounce of sincerity–for fun, or politics, or profit, or tenure (which is the same thing). Whatever the writer’s motives or true beliefs, such an article has the unforced force of the better argument. The argument is not better or worse because its author happens to believe it.
The mercenary, dishonest, insincere writer of a good argument may have written herself one step close to damnation, and the pure-hearted writer of a flawed argument may have written herself one step closer to heaven. Given those high stakes, our concern for the souls of others, and the fact that scholarship and writing are–like everything else–a moral enterprise, we should take that seriously, and maybe we should say so more often. But if our ultimate lodestar is the unforced force of the better argument, shouldn’t we assent to the persuasive argument of the damned writer and reject the flawed argument of the saved writer, and read both arguments as if the truth and persuasion of the work are all that counts and authorial sincerity is irrelevant? If a persuasive article is prefaced with a frank statement from the author that she has written the article because its argument will undermine the government’s position in the birthright citizenship case, or because it will be good for her career, or out of simple perversity, and that nothing that follows should be understood to state her own view, why should that matter if our considered conclusion is that she has given the better argument?
I appreciate that by focusing on the one phrase quoted by Larry, I’m ignoring a lot of relevant background context from Habermas–including, to quote Michael Froomkin, a “good faith commitment to honest [and “nonstrategic[ ]”] debate.” Perhaps those missing ingredients, which I am too ignorant to do justice to here, supply the answer to my question. If they do, I would have thought that we should be more willing to police the requirements of honesty and good faith, and thus less inclined to follow Larry’s restrictive rule concerning accusations of insincerity. Maybe the sheer quantity of American legal scholarship that openly or covertly fails the test of engaging in non-strategic debate makes the whole realm so non-ideal that we should treat American legal scholarship as a plague zone, a failed experiment in honest deliberation and debate. Maybe we would then see an individual’s adoption of a strong assumption against insincerity partly as an effort to model ideal deliberation, regardless of what others are actually doing, and partly as a dogged or desperate attempt to preserve one’s own soul in a sinful world. I find those possibilities interesting and admirable. But in either case, the approach would have little or nothing to do with the actual state of affairs.
The possibility and reality of AI-generated legal scholarship, it seems to me, heightens the question. Roughly put, I understand the position of advocates in this area to be that if the use of AI, in part or even in whole, to create scholarly work results in more and better scholarship, then the search for truth has been advanced and we should treat this as a positive good. To the extent that this is right, and that the AI engine itself has no motives to be sincere or insincere about, why should we care whether the originating “author” is sincere or not? Imagine that the human originator writes, in the (machine-generated) preface to an otherwise convincing article, that he asked the AI engine to find a missing argument in some existing body of literature, choose the side of that argument that is most likely to appeal to the largest number of likely readers (with a weighted emphasis on appealing to lateral hiring committees), and write the best possible version of that argument, which follows. Does the truth of that argument suffer from the multiple layers of lack of genuine feeling or belief that went into producing it? If that human generates one of these every week, sending them into the world out of what he happily admits are purely mercenary motives and with a general indifference to the positions taken, does his sincerity or insincerity matter at all?
I would like to think it does. But I’m not sure I can justify that belief. I might reject the whole enterprise on other grounds, even if it results in more and better scholarship produced more rapidly. There are good reasons to avoid lightly accusing others of insincerity, even if we are certain that some number of academic authors are in fact insincere. But the overarching question whether we should care about scholarly insincerity at all seems like a tougher question to me, and one that will increasingly have little to do with the actual quality of the scholarly work.
