« Live-Blogging the Big-Bankruptcy Empirical Research Agenda (2): Defining Terms | Main | MERS Can’t Transfer Mortgages »

Big-Bankruptcy Empirical Research Post-Op (3): Jack-knife Fights and Pencils in Zimbabwe

posted by Jonathan Lipson

If you have followed me this far--and it's understandable if you haven't--you might be curious to know what ultimately came of LoPucki's Big-Bankruptcy Empirical Research Conference, which I "live-blogged" (is that a verb?) yesterday.

The short answer:  It's all about jack-knifing and pencils in Zimbabwe.


Background:  Nothing gets academics’ dander up like debates about methodology.  For legal academics, this often breaks into two related clashes.  (1) Whether to be an “empiricist” or not; and (2) if so, how to do it.  

The folks at LoPucki’s conference mostly drink the empiricism Kool Aid, so answer the first question “yes.”  After all, they included some of the nation’s leading business bankruptcy empiricists, among others Ken Ayotte (Northwestern), Joe Doherty (UCLA), Ted Eisenberg (Cornell), Bob Lawless (Illinois), Adam Levitin (Georgetown), Steve Lubben (Seton Hall), Ed Morrison (Columbia), Bill Whitford (Wisconsin), Sarah Woo (NYU) and, of course, LoPucki himself.

Rather, the real knife fight was over how to do this work.  Must it only be quantitative (and guided by a scientifically legitimate—falsifiable—hypothesis)? Or could (should) it also include (arguably less rigorous) qualitative methods?  Does it have to be social science?  Or is “good enough for law” good enough?

This may sound like mere wonkage.  But it matters for two reasons.  

First, traditional social scientific analysis—statistics with a hypothesis—has an undeniable allure.  If the numbers say it, we think, it must be true. Many lawyers and legal academics found themselves in law because they were not strong quantitatively.  We envy the “math macho” of the certainty in “hard” numbers.  We are exhorted to observe the “rules of [statistical] inference” even if we were not trained in them.  In other words, we are trying to practice a kind of social science without a license.

But this is not just ego talking.  It is no accident that many would claim that the 2005 amendments to the Bankruptcy Code are flawed at least in part because they had little empirical basis:  they reflected the will of lobbyists, who cajoled and threatened Congress, not the reality of what was (and was not) happening on the ground.  I have argued elsewhere that one reason to be suspicious of Dodd-Frank is that important aspects of the resolution authority are not based in any empirical analysis.

Second, and more important, it can be dangerous to rely too heavily on numbers.  As my UW colleague Bill Whitford observed, “the word empirical has come to mean quantitative research, but the truth is that empirical research is also qualitative.”  He’s right.  Qualitative research—meaning interviews and other “anecdotal observations”—may spell the difference between analysis and nonsense.

Ed Morrison—a very strong quant by legal academic standards—recounted that an economics undergrad he knew apparently wrote a paper claiming to find a strong statistical relationship between demand for no. 2 pencils in Zimbabwe and the price of mail bags (or something comparably bizarre).  This seemed to confirm LoPucki’s observation (and I paraphrase):  “Empiricism is about reality; statistics is about manipulating numbers.”  

The truth, of course, is that these methods--counting and talking (or listening)--shouldn’t be mutually exclusive.  Indeed, LoPucki’s database was originally developed using interviews and other qualititative techniques, as well as pure numbers.  In response to a claim that good empirical work requires a quantifiable hypothesis, LoPucki asked (perhaps rhetorically): “Is it enough if I just jack-knife through all the numbers?” (jack-knifing is a statistical testing technique used to estimate the bias and variance of a statistic). 

Unless we think the price of mail bags determines the demand for no. 2 pencils in Zimbabwe (or vice versa), the answer must be—at least for legal academics—no:  You should still talk to people to find out what is actually happening in the real world.

For those of you who care about qualitative empirical work, a final word:  Whitford not only used these techniques to make central contributions to business bankruptcy knowledge (i.e., the LoPucki database).  He has also very generously organized a conference scheduled for this fall honoring the work of Wisconsin's Stewart Macaulay, one of the parents of modern qualitative empirical legal research.  Although Macaulay is a contracts—not a bankruptcy—scholar, his exhortations to talk to people on the ground—to learn the "law in action”—have every bit as much salience as fancy statistical analysis.



Really? Are legal academics not aware that correlation doesn't equal causation?

I think a lot of researchers eschew interviews for two reasons, 1: personality traits of some academics lead them to prefer working with texts and data rather than people; and 2: interviews are hard to categorize easily so they mess up the elegant insight one wants to espouse. But I think they are essential to good research about human beings. LoPucki's venue studies, for instance, would benefit from asking the people involved why they chose a venue.

The comments to this entry are closed.


Current Guests

Follow Us On Twitter

Like Us on Facebook

  • Like Us on Facebook

    By "Liking" us on Facebook, you will receive excerpts of our posts in your Facebook news feed. (If you change your mind, you can undo it later.) Note that this is different than "Liking" our Facebook page, although a "Like" in either place will get you Credit Slips post on your Facebook news feed.



  • As a public service, the University of Illinois College of Law operates Bankr-L, an e-mail list on which bankruptcy professionals can exchange information. Bankr-L is administered by one of the Credit Slips bloggers, Professor Robert M. Lawless of the University of Illinois. Although Bankr-L is a free service, membership is limited only to persons with a professional connection to the bankruptcy field (e.g., lawyer, accountant, academic, judge). To request a subscription on Bankr-L, click here to visit the page for the list and then click on the link for "Subscribe." After completing the information there, please also send an e-mail to Professor Lawless ([email protected]) with a short description of your professional connection to bankruptcy. A link to a URL with a professional bio or other identifying information would be great.