At all events, there is no doubt that Finnell’s estimated rates of overblocking, which are based on the filtering companies’ own category definitions, significantly understate the rate of overblocking with respect to CIPA’s category definitions for filtering for adults.  The filters used in the Tacoma, Westerville, and Greenville libraries were configured to block, among other things, images of full nudity and sexually explicit materials.  There is no dispute, however, that these categories are far broader than CIPA’s categories of visual depictions that are obscene, or child pornography, the two categories of material that libraries subject to CIPA must certify that they filter during adults’ use of the Internet.  Finnell’s study also calculated underblocking rates with respect to the Westerville and Greenville Libraries (both of which logged not only their blocked sites, but all sites visited by their patrons), by taking random samples of URLs from the list of sites that were not blocked.  The study used a sample of 159 sites that were accessed by Westerville patrons and determined that only one of them should have been blocked under the software’s category definitions, yielding an underblocking rate of 0.6%.  Given the size of the sample, the 95% confidence interval is 0% to 1.86%.  The study examined a sample of 254 Web sites accessed by patrons in Greenville and found that three of them should have been blocked under the filtering software’s category definitions.  This results in an estimated underblocking rate of 1.2% with a 95% confidence interval ranging from 0% to 2.51%.

We do not credit Finnell’s estimates of the rates of underblocking in the Westerville and Greenville public libraries for several reasons.  First, Finnell’s estimates likely understate the actual rate of underblocking because patrons, who knew that filtering programs were operating in the Greenville and Westerville Libraries, may have refrained from attempting to access sites with sexually explicit materials, or other contents that they knew would probably meet a filtering program’s blocked categories.  Second, and most importantly, we think that the formula that Finnell used to calculate the rate of underblocking in these two libraries is not as meaningful as the formula that information scientists typically use to calculate a rate of recall, which we describe above in Subsection ii.E.3.  As Dr. Nunberg explained, the standard method that information scientists use to calculate a rate of recall is to sort a set of items into two groups, those that fall into a particular category (e.g., those that should have been blocked by a filter) and those that do not.  The rate of recall is then calculated by dividing the number of items that the system correctly identified as belonging to the category by the total number of items in the category.