In the example above, we discussed a database that contained 1000 photographs.  Assume that 200 of these photographs were pictures of dogs.  If, for example, a classification system designed to identify pictures of dogs identified 80 of the dog pictures and failed to identify 120, it would have performed with a recall rate of 40%.  This would be analogous to a filter that underblocked at a rate of 60%.  To calculate the recall rate of the filters in the Westerville and Greenville public libraries in accordance with the standard method described above, Finnell should have taken a sample of sites from the libraries’ Internet use logs (including both sites that were blocked and sites that were not), and divided the number of sites in the sample that the filter incorrectly failed to block by the total number of sites in the sample that should have been blocked.  What Finnell did instead was to take a sample of sites that were not blocked, and divide the total number of sites in this sample by the number of sites in the sample that should have been blocked.  This made the denominator that Finnell used much larger than it would have been had he used the standard method for calculating recall, consequently making the underblocking rate that he calculated much lower than it would have been under the standard method.

Moreover, despite the relatively low rates of underblocking that Finnell’s study found, librarians from several of the libraries proffered by defendants that use blocking products, including Greenville, Tacoma, and Westerville, testified that there are instances of underblocking in their libraries.  No quantitative evidence was presented comparing the effectiveness of filters and other alternative methods used by libraries to prevent patrons from accessing visual depictions that are obscene, child pornography, or in the case of minors, harmful to minors.  Biek undertook a similar study of the overblocking rates that result from the Tacoma Library’s use of the Cyber Patrol software.  He began with the 3,733 individual blocks that occurred in the Tacoma Library in October 2000 and drew from this data set a random sample of 786 URLs.  He calculated two rates of overblocking, one with respect to the Tacoma Library’s policy on Internet use – that the pictorial content of the site may not include “graphic materials depicting full nudity and sexual acts which are portrayed obviously and exclusively for sensational or pornographic purposes” – and the other with respect to Cyber Patrol’s own category definitions.  He estimated that Cyber Patrol overblocked 4% of all Web pages in October 2000 with respect to the definitions of the Tacoma Library’s Internet Policy and 2% of all pages with respect to Cyber Patrol’s own category definitions.