pages as they existed either at the point that a patron in one of the three libraries was denied access or when Finnell and his team reviewed the pages.  It is therefore impossible for anyone to check the accuracy and consistency of Finnell’s review team, or to know whether the pages contained the same content when the block occurred as they did when Finnell’s team reviewed them.  This is a key flaw, because the results of the study depend on individual determinations as to overblocking and underblocking, in which Finnell and his team were required to compare what they saw on the Web pages that they reviewed with standard definitions provided by the filtering company.

Tacoma library’s Cyber Patrol software blocked 836 unique Web sites during the month of August.  Finnell determined that 783 of those blocks were accurate and that 53 were inaccurate.  The error rate for Cyber Patrol was therefore estimated to be 6.34%, and the true error rate was estimated with 95% confidence to lie within the range of 4.69% to 7.99%.  Finnell and his team reviewed 185 unique Web sites that were blocked by Westerville Library’s Websense filter during the logged period and determined that 158 of them were accurate and that 27 of them were inaccurate.  He therefore estimated the Websense filter’s overblocking rate at 14.59% with a 95% confidence interval of 9.51% to 19.68%.  Additionally, Finnell examined 1,674 unique Web sites that were blocked by the Greenville Library’s N2H2 filter during the relevant period and determined that 1,520 were accurate and that 87 were inaccurate.  This yields an estimated overblocking rate of 5.41% and a 95% confidence interval of 4.33% to 6.55%.  Finnell’s methodology was materially flawed in that it understates the rate of overblocking for the following reasons.  First, patrons from the three libraries knew that the filters were operating, and may have been deterred from attempting to access Web sites that they perceived to be “borderline” sites, i.e., those that may or may not have been appropriately filtered according to the filtering companies’ category definitions.  Second, in their cross-examination of Finnell, the plaintiffs offered screen shots of a number of Web sites that, according to Finnell, had been appropriately blocked, but that Finnell admitted contained only benign materials.  Finnell’s explanation was that the Web sites must have changed between the time when he conducted the study and the time of the trial, but because he did not archive the images as they existed when his team reviewed them for the study, there is no way to verify this.  Third, because of the way in which Finnell counted blocked Web sites – i.e., if separate patrons attempted to reach the same Web site, or one or more patrons attempted to access more than one page on a single Web site, Finnell counted these attempts as a single block, see supra note 10 – his results necessarily understate the number of times that patrons were erroneously denied access to information.