A couple of weeks ago I wrote a short critique of one piece of a long study written by David Evans and Joshua Wright about the Consumer Financial Protection Agency and funded by the American Bankers Association. The related blog post is here. Evans and Wright have responded.
There's a lot that I thought was objectionable or questionable in Evans and Wrights study, but most of it was well within the bounds of reasonable argument. I have no problem intellectually with arguments that any particular regulation could impose costs that outweigh its benefits. Instead, I was was moved to write because Evans and Wright were making precise numerical claims about the cost impact of the CFPA, and that these claims were based on either (1) a highly questionable comparison to dissimilar regulation or (2) pure conjecture.
In their reply, Evans and Wright spend a good deal of time arguing about things that are really beside the point to my critique. For example, Evans and Wright emphasize that I have not proved the affirmative case for the CFPA's positive impact (a passing point I made to show that the economic impact of regulation is susceptible to multiple predictions) and that I have "disputed virtually none of [their] findings that the CFPA Act would impose high costs on lenders and ultimately result in denying borrowers choice." Let's be clear. My critique was about three spurious numbers. I didn't set out to prove a positive case in the critique and don't need to do so to make my central point. And to imply a concession from silence about other issues is ridiculous in this context. This sort of logical move is, however, consistent with the problems with Evans and Wright's statistical claims.
But let's get to the heart of the matter. My issue with Evans and Wright is about the numbers, not about their priors regarding regulation. There are three numerical claims in Evans and Wright's piece with which I took issue. First, Evans and Wright claim that a CFPA would result in a 160 basis point increase in the cost of credit and a derivative 2.1% decrease in credit demand. These assertions were based on a comparison with a study of non-analogous regulations that have been found to have an 80 basis point impact. Evans and Wright argue that even though the regulations are different, they are less invasive, so therefore at least twice the impact would be the lower bound. Why twice? Just because. Evans and Wright still have no justifiable basis for doubling, as opposed to tripling the number, etc. It is not as if 160 basis points is within some statistical confidence interval or the like. While a 160 basis point number appears to have the imprimatur of social science, it is just conjecture, or, to be charitable, a very rough guesstimate.
In a cost-benefit analysis, however, precision matters. A CFPA might be worthwhile at 120 basis points, but not at 160 basis points, for example. The problem with Evans and Wright's methodology is that they can no better defend a 160 basis point number than a 120 basis point number or a 700 basis point number. Evans and Wright emphasize that there were merely setting a lower bound, but that hardly makes their number more defensible. Evans and Wright simply do not and cannot know the impact, including what the lower bound would be. Of course, precision is beside the point if the goal is to produce a scare statistic, rather than a rigorous cost-benefit analysis.
The third spurious statistic in Evans and Wright is a claim that a CFPA would result in 4.3% slower job creation. They achieved this number by taking a statistic about the role of small startups in job creation and then "supposing" that a CFPA would inhibit five percent of this. I noted there were problems with their job creation statistic (namely that it failed to account for the spectacular failure rate of small startups after their first year, when they result in net job loss, not creation). But that was a side point. The critical problem was that they "supposed" a impact number without any basis whatsoever for their supposition.
Evans and Wright take issue with my statement that "The key point here, however, is the impact of the legislation is speculative and certainly not susceptible to precise statistical predictions.” They state, "That is a nihilistic approach." Actually, it is an intellectually honest approach. A debate that is poisoned by spurious statistical claims, rather than their debunking, are what will engender nihilism. It'd be great to have an empirically informed policy debate. But that's not license to make up numbers. Policy debates have to function within our epistemological limitations. There's a constructive debate to be had about the CFPA. But constructive doesn't mean making things up.