It's Time to Get Rid of Law Reviews

The Washington Beacon has published an absolutely jaw-dropping piece about the Harvard Law Review's article selection process, which allegedly gives substantial consideration not just to the assumed identity (race, gender, sexual orientation) of the author, but to the assumed identity of the authors of sources cited. (Who knew that I should have been indicating in every citation the race, gender, and orientation of the author?)

I haven't bothered submitting to HLR for some time, but if the allegations are true, it still leaves me dismayed that I have had my time wasted as an author and furious that I have had my time wasted doing outside reviews. Don't ask me to do free reviews when it's just for show. I'm just waiting for the class action...

It’s easy to dismiss the HLR fiasco as an example of woke gone wild, and that’s undoubtedly part of the problem, but the more fundamental problem is that student editors should have no business selecting articles. Indeed, as I will argue below, law reviews are a medium that has served its purpose and they should shut down—there’s a much better way to disseminate legal scholarship: connecting authors directly to legal research databases (direct-to-database publishing).

The Problems with Student-Edited Law Reviews

The criticisms of student edited law reviews are well-known: in what sane world would second year law students sit in judgment of the quality and originality and importance of scholarship, including on topics about which they have not even taken courses? The students, no matter how bright and hard-working, just do not have the expertise to evaluate the scholarship.

What’s worse, student-editing distorts legal scholarship: we have to work around a biannual article shopping cycle (Shrovetide and Hlamas), and we have to write for an audience of 2Ls. That means we are inclined to pick topics that will resonate with them, which favors big policy debates, rather than important technical issues. We are incentivized to overclaim to make our work sexy. We end up spending too much time explaining and citing for basic concepts, including sections that explain why our work is important, etc. The incentive, especially pre-tenure, is to optimize around over-selling crap to 2Ls. That’s nuts.

Peer-Review Is Poorly Suited for Legal Scholarship

But let’s not kid ourselves into thinking that peer review is a panacea. Peer review is a good mechanism for double checking on methodological issues, like whether an empirical study has the proper controls. It works well when there is a common methodology for an entire field, say, economics. But peer review is not reliable for substantive knowledge issues: a peer reviewer might or might not have the relevant substantive knowledge to evaluate an article. For example, an economist might be able to give a good review of the methodology of the OLS analysis in a paper on bankruptcy claims trading, but the economist (and frankly many law professors) would likely lack the bankruptcy knowledge to realize that most trades in bankruptcy claims are not visible in the claims register, and would thus miss the enormous selection bias problem with a paper that relies on an analysis of trades for which a Rule 3001(e) filing was made.

The even bigger problem with peer review, however, is that it is utterly unsuited for normative scholarship, and a lot of legal scholarship is normative. A originalist reviewer probably isn’t going to like a non-originalist con law analysis, no matter its quality. A pro-choice scholar probably isn’t going to like a work arguing against abortion, regardless of quality. And so on. I’ve seen this first-hand. I submitted my first article to the Antitrust Law Journal almost 20 years ago. One reviewer liked the piece. The other denied that there could possibly be antitrust problems in two-sided markets. iirc, I got a revise and resubmit. In this instance, I told the journal to kiss off and fortunately the students at UCLA didn’t have normative priors that affected their review: payment card antitrust was not something most students would have a view on, making them actually a better screen than professionals who might have a dog in the hunt, even if indirectly. (I note that a PLOS One model of only reviewing for methodology avoids the normative evaluation problem, but that doesn't work in a field without a methodology...)

What’s the Purpose of Law Reviews?

So where does that leave us? I think we should go back to first principles. What is the purpose of a law review? Law reviews originally began as a method of disseminating legal scholarship. 125 years ago or so, if you wanted to research scholarship on an issue, you had to go through lots of separate print volumes of law reviews. The knowledge went from the author to the reader through the law review (A—>LR—>R). That’s not how it works today. I do not know anyone who reads an actual print general law review. Readers consume articles electronically, generally through databases like Lexis, Westlaw, Hein, and SSRN, that compile law review articles. So knowledge goes from Author to the Law Review to the Database to the Reader (A—>LR—>D—>R). With this structure, one has to ask what value the law review is providing in the process.

As I see it, the law review performs two services in the scholarship process. The first is a vetting service. While the law review’s publication might not mean that the article is any good, it probably screens out the really crazy stuff. Second is an editing service. That’s hit-or-miss, however. I’ve had a few law review editors (I’m looking at you, Dan Hemel) really improve my pieces substantively. But generally, edits have either been minor stylistic improvements, added (and unnecessary) citations, or even stylistic debasements. All in all, I can’t say that my articles have really improved from the editing process, and the improvement certainly hasn’t been worth the effort put in by me and the editors. All of this is to say, law reviews are not adding very much value to the production of legal scholarship.

To be sure, law review also provides an educational service, in that it exposes a subset of students to legal scholarship, and helps them become better at the critical skill of Bluebooking. And it’s a credential that enables greater differentiation of students for clerkship hiring than 1L grades alone. So, basically, law reviews add little value to scholarship and education, but provide a slightly helpful marker for judges who are overwhelmed by the volume of clerkship applications.

Get Rid of the Law Reviews

Accordingly, here’s my modest proposal: let’s cut out the middle man and get rid of law reviews. Legal scholarship should just be published direct-to-database. Authors would submit directly to databases. (A—>D—>R).

Here’s how it could work: Each law school should contract directly with the relevant scholarship databases to publish its faculty’s work. For example, Georgetown Law could contract with Lexis and Westlaw to have the “Georgetown Law Review” that would consist of articles written by Georgetown Law faculty. Georgetown would have its own internal vetting process (that could be as little as “we trust our faculty” or involve an actual review process of sorts). Georgetown could also have its own editing process (either by students or by staff). Either way, the school would do the gatekeeping function. Articles would go up to the database on a rolling basis and would have a citation of vol. Geo. L. Rev. pin (year). But the article would never appear in print unless a faculty member wanted to pay for a vanity offprint. Schools would be the guarantors of their own faculty’s scholarship quality.

We already have something very close to this: SSRN. Lots of working papers get published to SSRN. What I envision is basically the same thing, but with the working papers also going into Lexis, Westlaw, and Hein, and with a more standardized citation format. How much, if any vetting and editing would be up to the particular school, but being a faculty member (including a VAP or fellow) should be vetting enough, and hopefully we can all write good.

Now there is a small amount of legal scholarship published by people not on faculties. I can imagine a small number of traditional law reviews, generally speciality journals existing for their work, but I’m looking for a solution for the 99%, not the 1% here, and I think direct-to-database does the trick of getting scholarship disseminated without the insanity of student article selection and editing or the inappropriateness of peer-review for a frequently normative field.

Comments

[Sam]

Top tier English, European, Australian and Singaporean legal scholarship is of a much higher standard than what you find in the U.S. This is because those countries have peer review (it may also be because legal realism, politicisation and psuedo-interdisciplinary method utterly robbed the US academy of merit).

The notion that scholars would be too biased to provide worthwhile review is laughable to me personally but I accept that claim is likely true for the American academy.

[Adam Levitin]

I am not well versed in non-American legal scholarship, but my impression is that it tends to be much more practical/technical, and less theoretical. A lot of the pieces that I have seen would end up in practitioner journals in the US. If that’s right, then reviewer bias is much less of a concern. (Fwiw, I’ve thought of Australian scholarship as a bit closer to the US, and Israeli scholarship even more so.)

But one of the real virtues in my opinion of American legal scholarship is how deeply it has imbibed legal realism. There are exceptions, of course—law and econ scholarship is sometimes unduly formal—and it can also go overboard when it morphs into its critical offspring. Still, the move away from formalism is both what makes peer review a bad fit and what makes American legal scholarship meaningfully engaged with policy issues.

[Ed Boltz]

Your criticisms and suggestions give a slight nod, when mentioning how actual bankruptcy knowledge is required to understand the claims process (although I suspect you're focus on Ch. 11 claims), to the problem that law review articles and legal scholarship can seem to pay little mind to the actual practice of law by actual practicing lawyers.

But then, I read and comment on all of the bankruptcy papers by getting them from SSRN and Digital Commons.

[Adam Levitin]

I was referring to Chapter 11 claims. Not a lot of action in post-petition consumer claims.

But here's a consumer bankruptcy example: suppose a paper were looking at whether there was racial bias in consumer bankruptcy case dismissals. An economist reviewer might be able to spot issues with Bayesian surname geocoding (basically trying to figure out someone's race by name/location), but wouldn't realize that there's a distinction between dismissals by the judge (discretionary) or by the clerk of the court for failure to file schedules (not discretionary).

[Jonathan]

FYI, I actually read the Washington Free Beacon article and the HLS rubric they cited, and it is basically a nothingburger. It seems they have their own preformed opinions about Harvard based on whatever Trump has said, and went looking for evidence. The rubric that they believe is a smoking gun, and was the heart of the linked article, mentions “author diversity” as a small part of one of eight categories for grading, named “Diversity”, which is mostly aiming to encourage diversity in topics, institutions and younger authors. You might be sending people to a pretty far-right website for no reason there.

[Adam Levitin]

Yes, Washington Free Beacon is a conservative publication. But I don't know what the relevance of that observation is other than ad hominem. The WFB piece is not at all a nothingburger. It's pretty damning.

True, the WFB hasn't shown that any particular article was accepted or rejected because of diversity concerns. But the evidence put forth by the WFB wasn't just a grading rubric including a diversity category (which is NOT about topics, institutions, and younger authors, as there is a strong presumption against younger authors like current law students). The WFB also has a lot of quotations from actual HLR article evaluations that suggest at least some HLR editors are more interested in promoting a social engineering agenda than scholarship. When editors are evaluating the identity of the authors of sources cited, that's not just a casual nod at diversity. Consider this:

"From quick searches, the author primarily cites T-14, male scholars who do not appear to be from underrepresented groups," one editor wrote of an article on banking law. "Financial regulation is not known for its diversity…"

That strikes close to the bone. Who should the author have been citing? There aren't a lot of financial regulation scholars out there and they are disproportionately at T-14 schools. (I am far from certain, however, that finreg is a particularly pale-male-yale field. It certainly isn't compared with bankruptcy! I suspect this reflects an unconscious bias of the editor about finance...)

Maybe the most disturbing bit in the WFB piece, however, was this description of the HLR process:

'The system means that the most politically engaged editors are often overrepresented in the final tally, according to one former editor, who described O-Reads as a face-off between "the radical left and the Federalist Society."'

If that's true, it's a huge problem for any scholar whose work doesn't fit the orthodox positions or topical interests of the "radical left" or the "Federalist Society." Any heterodox or non-ideological scholarship is just an orphan in such a world; non-partisan topics like commercial law just fall through the cracks.

[Ed Boltz]

Continuing, a practicing consumer bankruptcy attorney would know that "by the clerk of the court for failure to file schedules" would know that such dismissals are not always automatic and subject to a large amount of discretion. Contrariwise some dismissals by a judge, for example where there was no credit counselling course prior to bankruptcy, are completely nondiscretionary.

But 2nd year law student certainly wouldn't know that nor do they tend to care about dirty, dirty consumer bankruptcy, because neither do the "Radical Left", the Federalist Society, most of their bankruptcy professors or especially the TBLs hiring them off of their federal clerkships