1st Circuit reversal rate not what it appeared to be

Of the more than 460 cases heard by the U.S. Supreme Court during the first six terms of the Roberts court (2005-2010), 13 were heard on direct review from the 1st U.S. Circuit Court of Appeals. When one takes into account cases presenting circuit splits, however, the Supreme Court actually reviewed 69 decisions involving cases from the 1st Circuit.

This article examines the Roberts court’s affirmances and reversals of all of these 1st Circuit decisions, concluding that the 1st Circuit has a significantly higher reversal rate than previously understood.

The traditional method for calculating a court of appeals “track record” is a simple score card measure: For any given year, divide the ratio of the number of cases in which the court reversed the circuit by the total number of cases decided by the Supreme Court from that circuit.

The method is outcome driven, focusing on affirmances and reversals rather than the substance of the court’s reasoning. It also only accounts for those cases on direct review from the circuit court below.

We think that the traditional measure is incomplete and can be misleading. A better measure — the “full” reversal rate — examines a broader category of cases and focuses on a slightly different metric.

In any given term, the Supreme Court considers not just a circuit court’s cases on direct review, but also “shadow decisions,” i.e., those circuit decisions identified by the court as part of a circuit split.

Including shadow decisions into a circuit court’s reversal rate uses more information than the traditional one and in that way more accurately assesses the frequency with which the Supreme Court agrees with that circuit’s approach.

Moreover, because the full approach generates more observations about each court of appeals, this approach also means that the track record of a court of appeals in any one year is more robust.

With only 70 to 90 merits cases a year on its docket, the Supreme Court will likely hear only a handful of cases from any given circuit. For example, in 2010, the Supreme Court reviewed only two cases from the 1st Circuit.

That number increases to 16 when shadow decisions are included. Including how a court’s approach fares in shadow decisions therefore creates a larger and more reliable dataset.

Further, unlike the traditional approach, which measures just the outcome of case, the full approach focuses on the Supreme Court’s reasoning and that of the courts of appeals.

To be clear, for the purposes of this article, we characterize “success” as the Supreme Court’s acceptance of a 1st Circuit decision, whether on direct review or through review of a circuit split case from a sister circuit.

In that way, the court of appeals is correct (or successful) only in the sense that it accurately predicted the way the Supreme Court would rule on the case or an issue, not whether in some other sense the court of appeals was correct in its interpretation and the Supreme Court was not.

A full description of our methodology and more detailed data tables are available on our website, http://www.hangley.com/Supreme_Court_Project/. We conclude the following from our analysis of the 1st Circuit’s success before the Roberts court to date.

How 1st Circuit stacks up

Using the traditional method, assessing the 1st Circuit on a term-by-term basis creates an incomplete, and skewed, picture of the circuit court’s level of success, possibly because the data set is so small.

Table 1, which can be viewed here, shows the 1st Circuit’s traditional and full reversal rate, by year and for the period 2005 to 2010. For two of the six years of the Roberts court (2006 and 2010), the Supreme Court reviewed a total of three cases, affirming each time. On that basis, the 1st Circuit enjoyed a traditional reversal rate of 0.00% for those terms.

In comparison, when shadow decisions are taken into account, the Supreme Court reviewed 11 1st Circuit decisions in 2006 and 12 1st Circuit decisions in 2010. Of those, the Supreme Court reversed six cases in 2006 and four in 2010, resulting in reversal rates of 54 percent and 33 percent, respectively. Those reversal rates are significantly higher than the reversal rates under the traditional method.

If you aggregate the Supreme Court’s direct review of the 13 decisions the court heard during the entire period, you would conclude that the Supreme Court reversed the 1st Circuit only six times for a traditional reversal rate of 46.2 percent.

That success rate was the best in the country by a wide margin; no other circuit court was reversed less than the 1st Circuit, with the 10th Circuit (57.1 percent) and 3rd Circuit (62.5 percent) coming closest under the traditional method.

The full methodology — which includes valuable information from the shadow decisions considered in circuit split cases — presents a very different picture. Instead of just 16 reviewed decisions, the Supreme Court actually considered 69 1st Circuit decisions between 2005 and 2010, over five times more than the traditional method. Of those, the Supreme Court rejected the 1st Circuit’s standard 40 times, for a reversal rate of 58.0 percent.

Strikingly, that reversal rate shifts the 1st Circuit from one of the least reversed circuits in the country to the lower end of the pack. In other words, since 2005, rather than being the least reversed circuit in the country, the 1st Circuit ranks in the bottom third of the circuits for reversal rate. Only the Federal Circuit (63.3 percent), the 6th Circuit (66.3 percent) and the 9th Circuit (68.9 percent) were reversed more often under the full methodology.

1st Circuit’s closest ‘sister circuits’

Our full method is also helpful in illuminating the extent to which the 1st Circuit is aligned with other circuits on the issues on which the Supreme Court has granted cert.

Much as court watchers create a concordance table identifying the frequencies with which each justice votes with every other justice, we have created an analogous table showing the degree to which the circuits agreed with each other on the cases heard by the Roberts court.

Specifically, the agreement percentages are calculated as the number of agreements (both circuits affirmed or both reversed) divided by the sum of agreements and disagreements between the two circuits.

These data show that the 1st Circuit has the greatest level of concordance with the 2nd Circuit (73.5 percent), the 7th Circuit (71.8 percent) and the 3rd Circuit (71.0 percent). Interestingly, all three of these circuits have lower reversal rates than the 1st Circuit, with the 3rd Circuit and the 7th Circuit having two of the three lowest reversal rates in the country between 2005 and 2010.

In contrast, the 1st Circuit is least aligned with the 11th Circuit (46.9 percent) and the 9th Circuit (52.8 percent).