AI Models on Law School Exams

The question of how well AI can do on law school exams is one that interests me, since I give exams and want them to be a measure of how much my students have learned (as opposed to their skills at using AI -- although I want them to learn those too). Others appear to be interested too -- just see the ssrn downloads for papers on this topic.  Caveat: I can't pretend that I have more than the shallowest of understandings of AI models.  But this cool new paper I came across might be of interest to folks.  

The paper is from a set of scholars at ETH in Zurich (a place long known for its excellent research).  As I understand the draft (and, to repeat, I don't understand a lot of this stuff), the paper finds that the large language models (LLMs) don't do that great when you increase the level of reasoning required on the exam.  I was also intrigued to read (I think) that LLMs are not necessarily better on multiple choice exams than essay type ones.  From the abstract, here is a sentence that stood out: "Our evaluation on both open-ended and multiple-choice questions present significant challenges for current LLMs; in particular, they notably struggle with open questions that require structured, multi-step legal reasoning".  

The paper is "LEXam: Benchmarking Legal Reasoning on 340 Legal Exams" 

Among the other cool things about this paper to me is how collaborative it is -- students, professors, and even judges.  To me, it reflects well on the culture of the institution that has such a degree of collaboration. 

Sharing Risk

Credit Slips readers will want to check out a brand-new book from Pat McCoy, the Liberty Mutual Insurance Professor at Boston College Law School (and a past guest blogger here). Sharing Risk offers both a diagnosis and prescription for the financial precarity of American households. Because over half of Americans do not make enough to meet basic needs, they often turn to borrowing to make ends meet. McCoy proposes expended risk-sharing arrangements about income security, housing, health insurance, and college education. McCoy's proposals seek to enable American families to flourish and secured their economic well-being.

The book is available directly from the University of California Press. McCoy also passed along that she is now blogging at a new substack. 

Fannie and Freddie Are Now Explicitly Guarantied

When will Donald ever learn to run his tweets by counsel before posting them? He consistently shoots his legal position in the foot. The latest is about the implicit government guaranties of Fannie Mae and Freddie Mac: 

I am working on TAKING THESE AMAZING COMPANIES PUBLIC, but I want to be clear, the US Government will keep its implicit GUARANTEES, and I will stay strong in my position on overseeing them as President.

Pro tip: it's not an "implicit" guaranty if you say it out loud. Once you do, it's explicit. 🤦

That's actually potentially a huge problem for federal accounting purposes. The whole reason that Fannie and Freddie's enormous book of debt is not on the federal balance sheet, blowing through the debt limit, is that the guaranty has always been implicit: it's about a wink and a nod. With this tweet, I am not sure that it is possible for Fannie/Freddie to come off the federal balance sheet even if privatized because of the now "explicit" guaranty. (Or as a fallback, there's a promissory estoppel argument.) As far as I can tell, because of an over-eagerness to tweet, Fannie and Freddie's obligations now bear the eagle. Maybe the CBO will view this differently, but all that comes to mind right now is the timeless words of Napoleon Dynamite:

https://imgur.com/sOgB6ZF

 

The Supreme Court Is Just Making Stuff Up About the Fed

The Supreme Court is welcome to have its opinions. But it is not welcome to have its own facts. Fact-finding is at the core of the judicial enterprise, and once the Court starts simply making things up, it loses its legitimacy. 

The Court took a dangerous step in that direction today with its opinion granting the President's order for a stay of the District Court's injunction of the President's removal of a member of the National Labor Relations Board and of the Merit Systems Protection Board.

Forcing Bank Deposits to Subsidize Stablecoins: the GENIUS Act

The Senate is set to take up a vote on this Thursday on the GENIUS Act, the legislation to create a regulatory framework for stablecoins. Whatever else one might think about stablecoins or the GENIUS Act, its insolvency provisions are an absolute mess, both conceptually and in drafting. If the GENIUS Act becomes the law, we're in for a FUBAR situation when a stablecoin issuer ends up insolvent. Even more concerning, if a bank custodian for a stablecoin issuer's reserves ends up insolvent, the claims of the stablecoin investors will come ahead of the bank depositors. That's right. Crypto comes ahead of ma-and-pa. 

The effect: stablecoins are being subsidized by bank deposits. Now that's GENIUS.

Pete Rose and Investment Markets

Hopefully readers will indulge me in a bit of a tangent for this post, which is about Pete Rose and his gambling, which often seems to be just another word for investing, particularly when investing in tokens that have no underlying fundamentals.

Rose, the all-time baseball hit leader and one of the most fun players to watch, was famously banned from the Baseball Hall of Fame for gambling on baseball, including on games in which he played or managed. There's now a push to posthumously rehabilitate him, with the President of the United States serving as one of his chief advocates. The President's argument is that Rose didn't do anything so bad because he never bet against his team. 

That's just incorrect. Rose claimed that he never bet against his team. We don't know if he did; he wasn’t the most credible character, given that he initially denied gambling on baseball at all. But let’s assume that he told the truth. Even if Rose never placed a bet directly against his team, he was absolutely betting against them because he wasn't placing one-off bets. He was a serial gambler with repeat relationships with a number of bookies. In that situation, gambling only on the Reds to win on certain days translates into gambling on the Reds to lose on other days.

The Form Knows Best (Does it Really?)

A few weeks ago, I got to give a talk at the University of Miami.  The focus: six myths about contracts, as many of us teach contract law (or at least as I teach it).  The talk was a blast -- the faculty at Miami were wonderful and generous, but poked all sorts of big holes in my claims.  The broader topic, in a little essay titled "The Form Knows Best" with two superb students (Tara Chowdhury and Faith Chudkowski), was the gravitational pull of the standard form on contract drafting practices. Our abstract:

Law students learn that contracts are carefully negotiated, precisely drafted, and shaped by doctrine. But lawyers tell a different story. This article compares six pillars of contract law with what we heard in over 170 interviews with senior transactional lawyers across M&A, sovereign bonds, and leveraged loans. The result is a gap between the Official Story taught in classrooms and the Unofficial Story told by practitioners—where boilerplate dominates, case law is rarely consulted, and market custom often prevails over rational design. We suggest that many contracts are better understood as historical artifacts—products of inherited forms and production pressure. Or, as one lawyer put it, more Mars Bar than masterpiece. That gap may matter—especially when courts often interpret form as if it reflects intent.

Deliberately Polluting the Death Master File Violates the Fair Credit Reporting Act

The Trump administration seems to be walking straight into a pair of Fair Credit Reporting Act violations by placing immigrants whom it knows to be alive on the Social Security Administration’s Death Master File. The Death Master File is a list compiled by the Social Security Administration of people Social Security believes to be dead (generally based on the filing of death benefit claims with Social Security, so it is not at all a complete list of who is actually dead). Creditors and other users of consumer reports regularly use the Death Master File, either directly or through a consumer reporting agency, as part of credit granting, employment, or insurance decisions—you don't want to be doing business with someone who is dead (and that might indicate that the living person with whom you are dealing isn't who they say they are). So, Death Master File issues end up being consumer reporting issues and fall under the Fair Credit Reporting Act, violations of which can not only create substantial private civil liability, but they can also be enjoined in a suit by a state attorney general.  

 

Fair Lending Deception by the CFPB

The Trump CFPB is seeking to vacate the consent order it entered into Townstone Mortgage for alleged violations of the Equal Credit Opportunity Act (ECOA). According to the Trump CFPB, CFPB staff engaged in misconduct by bamboozling former CFPB Director Kathy Kraninger about the legal strength of the case because they were woke warriors who took the position that “disparity automatically equaled discrimination,” and “wanted a de-facto mortgage quota, a policy aligned with the views of radical DEI proponents like Robin DiAngelo and Ibram X Kendi.” That view is hard to reconcile with the total number of discriminatory lending suits the CFPB has brought over the past eight years:  all of seven cases. Yup, that sure sounds like an out-of-control agency. 

 

If the Trump-controlled CFPB wants a consent order vacated "in the interests of justice," the district court should require it to prove both that there was in fact misconduct and that the misconduct harmed the defendant. The only "evidence" of this supposed misconduct is a self-serving, hearsay declaration by Dan Bishop, the Deputy Director of OMB (deputized to CFPB) reporting on his own alleged investigation. That's not "evidence." (And that's putting aside whether Bishop has been legally appointed to a CFPB position...)

 

But even if Bishop's story is credited entirely, there's still a problem. The supposed misconduct related to disparate impact liability and the reason the CFPB served a Civil Investigatory Demand on Townstone in the first place, but the defendant was sued for facial discrimination based discouragement of credit applications based on public statements its CEO made on a radio show named after the company. The Bureau could have brought the case without the information from the Civil Investigatory Demand. There’s no nexus between the supposed misconduct and the CFPB’s lawsuit, so there cannot be prejudice to the defendant. Accordingly, there's no reason to vacate the judgment in the interests of justice.

23andMe Bankruptcy

Quite a bankruptcy week. First there was Forever21's gone forever 22, and now we have 23andMe. Kudos to the Slips' own Melissa Jacoby and her co-authors Sara Gerke and Glenn Cohen for having the most timely publication ever regarding the 23andMe bankruptcy filing. But there are some weird things about this case, namely the debtor acquiescing in a massive stay violation and the St. Louis, Missouri, venue.

The Supreme Court Invites Bank Fraud Sophistry

The Supreme Court, in a decision that will surely be beloved of law professors, held that the bank fraud statute applicable to loan applications covers only actually false statements, not merely misleading statements. The Court got to flex its "oh look at how smart we are" muscle with clever illustrations of the difference between false and misleading statements:

If a tennis player says she “won the championship” when her opponent forfeited, her statement—even if true—might be misleading because it could lead people to think she had won a contested match. The Government also agreed at oral argument with another example: If a doctor tells a patient, “I’ve done a hundred of these surgeries,”
when 99 of those patients died, the statement—even if true—would be misleading because it might lead people to think those surgeries were successful.

And not to be outdone, Justice Alito adds in his own precocious observation in a concurrence:

After noticing that a plate of 12 fresh-baked cookies has only crumbs remaining, a mother asks her daughter, “Did you eat all the cookies?” If the child says “I ate three” when she actually had all 12, her words would be literally true in isolation but false in context. The child did eat three cookies (then nine more). In context, however, the child is implicitly saying that she ate only three cookies, and that is false.

Come on. Would common sense possibly suggest that Congress intended to allow misleading, but not literally false statements on bank loan applications? The result is absurd. Nothing in the legislative history would suggest that Congress wanted such a constrained view of the statute; indeed the legislative history isn't discussed, but there's lots of dictionary definition discussed. Apparently we are ruled by Merriam-Websters, rather than common sense. (And if that's the case, lets just have an AI judge and avoid all the SCOTUS nomination strum und drang). The Court's ruling is an invitation to the Holmesian bad man to go right up to the line of what is false. It all but invites Clintonian sophistry.

But given the Court's casuistry, let me pose my own:  if a bank's credit application included a question "Have you made any misleading statements on this application?" and the answer was false, would that trigger a violation of 18 U.S.C. § 1014?

I think the answer is yes. That suggests a simple regulatory fix to this bad decision: bank regulators should insist that bank loan applications include a declaration that the applicant has not made any misleading statements in connection with the application.

 

Is Greenpeace Heading for Bankruptcy?

[Updated 3/26/25:  I jumped the gun here; it's an occupational hazard of blogging.

 

It turns out that the North Dakota Rules of Civil Procedure are not the only North Dakota law on supersedeas bond requirements. Tucked away among the Century Code's provisions about the technical mechanics of execution and levy by sheriffs is a provision in the North Dakota Century Code that places a dollar limit on the supersedeas bond requirement. It limit the aggregate supersedes bond requirement for all defendants in a case to $25 million. That seems like a much more achievable figure for Greenpeace. as far as I can tell, the bonding limit came out of tort reform efforts. Who would have expected it to benefit an environmental group?

 

Assuming that the North Dakota courts follow the $25 million bond limit, I would expect Greenpeace to be able to post the bond, in which case bankruptcy would not be needed.]

 

It appears that the terminus of the Dakota Access pipeline is...Chapter 11. That's where I believe Energy Transfer's $660 million trespass and defamation verdict against Greenpeace in North Dakota state court is going to end up. Although Greenpeace is vowing to appeal the verdict, that's just a brave face. Greenpeace won't be able to post the supersedeas bond, and its US entities, at least, will likely end up filing for bankruptcy.  

 

Where Have I Heard This Before?

James Nani posted a story at Bloomberg Law about Tara Twomey's dismissal as executive director of the U.S. Trustee Program. It's worth a read and not just because of the shout out to my earlier blog post here on Credit Slips. 

Nani notes Twomey "isn't without critics." Fair enough, although that could be said for any effective governmental official. The critic in the article was Lawrence A. Friedman, himself a former executive director of the program from twenty years ago. From the article: "'Tara Twomey had no business being appointed to that job,' Friedman said. 'It was a political appointment at the behest of Liz Warren and others in the bankruptcy system.'" By "Liz Warren," I am fairly confident he meant Senator Warren.

The idea that Twomey had "no business being appointed" is appalling. Twomey had years of experience in consumer cases and business cases. Notably, she served as special consumer counsel in the chapter 11 of Ditech Holdings, a bankrupt mortgage servicer. She authored amicus briefs in consumer bankruptcy cases on behalf of the National Bankruptcy Rights Center. I am told her amicus briefs were cited more frequently in Supreme Court cases than any party except the solicitor general. She has taught courses at Stanford, Harvard, and Boston College. She was a member of the American Bankruptcy Institute's Commission on Consumer Bankruptcy, which has a full bio as of 2019 detailing her many accomplishments. At the time of her appointment, Twomey was a member of both the American College of Bankruptcy and the National Bankruptcy Conference. (Friedman is a member of neither.) These are prestigious, invitation-only organizations of bankruptcy professionals, although she had to resign from the NBC upon her appointment given that it takes substantive positions on bankruptcy policy issues.

Despite this record, Twomey had "no business being appointed?" Where have I heard that before?

Virtual Access to Event in Memory of Juliet Moringiello March 20 2025

Widener University Commonwealth Law School will hold an event honoring Professor Juliet Moringiello on March 20, 2025 at 1pm ET. Friends and fans of Juliet are welcome and encouraged to join virtually. The Zoom link is embedded in the image accompanying this post as well as accessible here. I will repeat what Widener says in the bottom of the image about Juliet: "Professor Moringiello was a beloved scholar, professor, mentor, author, administrator, colleague, and friend whose impact on our students and institution was profound and will never be forgotten. We gather to celebrate her contributions to the legal field, share memories, and find comfort in one another."

Making the Bankruptcy System Less Great

News reports this morning are confirming the rumors that went around the bankruptcy community last night. The Trump Administration has fired Tara Twomey as executive director of the Office of U.S. Trustees. This is a short-sighted and likely illegal decision.

The executive directorship of the US Trustee Program is a nonpolitical position. Twomey's predecessor served under both Republican and Democratic administrations. One report said the termination notice for another DOJ official cited Article II of the Constitution, meaning the Trump Administration must be relying on the wacky and ahistorical "unitary executive theory" where the president has unchecked power. That some judges and law professors have signed up for this idea does not make it any less wacky and ahistorical. The action against Twomey demonstrates that the only thing that matters now is loyalty to the president. Ability does not count. Twomey was a most able leader of the UST Program.

The Trump Organization’s Shake Down of Capital One

The Trump Organization is trying to shake down Capital One. And they’ll probably succeed. The Trump Organization has sued Capital One for closing its accounts in January 2021, allegedly because of Donald Trump’s political views. (Or, put differently, Capital One decided that it was not good business to continue being associated with an entity connected to the January 6 insurrection.)

As a legal matter, the Trump Organization's complaint is risible; Capital One should be able to easily get the case dismissed. But that might not matter because the Trump Organization has them over a barrel: if Capital One doesn’t pay up, the implicit threat is that the Trump administration will move to block the Capital One-Discover merger and generally make life unpleasant for Capital One. (Of course, if the Trump administration were really clever, they wouldn’t have dropped the CFPB suit so fast, but that’s probably just that the right hand didn’t talk to the left.) That’s gangster capitalism and underscores the incredible conflicts of interest that continue to exist for Trump.

Juliet Moringiello - One of the Greats

Juliet Moringiello was an amazing person. Her alchemy of brain and spirit and energy and heart and common sense made a positive difference for so many people, across disparate places and professions. She could teach you how to navigate a commercial law and to downhill ski.

Testaments from Widener University Commonwealth Law School and professional organizations illustrate how Juliet served academic and legal communities with distinction. Examples include the Uniform Law Commission (including an instrumental role in the development of the 2022 amendments to the Uniform Commercial Code), American Law Institute projects, and as a scholar-in-residence for the American Bankruptcy Institute. Juliet did these things while also serving in critical leadership roles at Widener and offering engaged and committed classroom teaching, including first-year property law and an array of upper level classes and seminars. 

Chris Odinet's memorial captures beautifully Juliet's commitment to helping others and building communities. As reflected in the mentoring award she recently received from the Commercial and Consumer Law Section of the Association of American Law Schools, Juliet did so much behind the scenes to lift up others and to help them improve their research and analysis. 

Juliet was ideally positioned for mentoring because her own scholarship was creative and wide-ranging and yet reflected care and attention to detail. She offered important insights on municipal bankruptcy and related state law procedures. Whereas scholars and jurists long have referred to the "Butner principle" in the abstract, Juliet closely studied the case for which the principle is named, which turned out not to match how it was remembered. She explored poorly drafted statutory language that since 2005 has affected the treatment of car loans in Chapter 13 repayment plans for individuals and proposed an analytical framework accordingly. These are just a few of the examples of her writings in which a reader can find careful and sustained attention to the relationship between state and federal law. 

With respect to state secured transactions law, Juliet comfortably traversed the border between real property and personal property. The problems dwelling from the tangible-intangible divide of personal property particularly attracted her attention. She explored puzzles that arise, for example, when one tries to apply fundamental concepts such as possession to remotely controlled activities.

And those projects dovetailed with Juliet's longstanding interest in understanding emerging technologies, and her ability to demystify how foundational commercial law concepts can be squared with innovation - from software licensing agreements and electronic contracting, to cyberspace and domain names and Second Life, to non-fungible tokens. As popular subjects for scholarship, writings on hot tech topics risk ephemerality. Juliet's work is built to last. She made these issues accessible while demonstrating how they could and should be situated in broader legal frameworks.

Of course, these professional interests were part of a rich multi-faceted life of family and friends, of appreciating the sights and nature in Pennsylvania, in Quebec, and anywhere and everywhere she traveled. When there wasn't enough snow for skiis, you might find her on a hike. Or on a bike. Or a paddleboard. 

Juliet Moringiello offers inspiration to do impactful work, to help others, and to spend time on the the things you love. Deepest condolences to her family.