It is difficult to determine how reliable Biek’s conclusions are, because he did not keep records of the raw data that he used in his study; nor did he archive images of the Web pages as they looked when he made the determination whether they were properly classified by the Cyber Patrol program.  Without this information, it is impossible to verify his conclusions (or to undermine them).  And Biek’s study certainly understates Cyber Patrol’s overblocking rate for some of the same reasons that Finnell’s study likely understates the true rates of overblocking used in the libraries that he studied.  We also note that Finnell’s study, which analyzed a set of Internet logs from the Tacoma Library during which the same filtering program was operating with the same set of blocking categories enabled, found a significantly higher rate of overblocking than the Biek study did.  Biek found a rate of overblocking of approximately 2% while the Finnell study estimated a 6.34% rate of overblocking.  At all events, the category definitions employed by CIPA, at least with respect to adult use – visual depictions that are obscene or child pornography – are narrower than the materials prohibited by the Tacoma Library policy, and therefore Biek’s study understates the rate of overblocking with respect to CIPA’s definitions for adults.  In sum, we think that Finnell’s study, while we do not credit its estimates of underblocking, is useful because it states lower bounds with respect to the rates of overblocking that occurred when the Cyber Patrol, Websense, and N2H2 filters were operating in public libraries.  While these rates are substantial – between nearly 6% and 15% – we think, for the reasons stated above, that they greatly understate the actual rates of overblocking that occurs, and therefore cannot be considered as anything more than minimum estimates of the rates of overblocking that happens in all filtering programs. 5.  Methods of Obtaining Examples of Erroneously Blocked Web Sites

The plaintiffs assembled a list of several thousand Web sites that they contend were, at the time of the study, likely to have been erroneously blocked by one or more of four major commercial filtering programs:  SurfControl Cyber Patrol 6.0.1.47, N2H2 Internet Filtering 2.0, Secure Computing SmartFilter 3.0.0.01, and Websense Enterprise 4.3.0.  They compiled this list using a two-step process.  First, Benjamin Edelman, an expert witness who testified before us, compiled a list of more than 500,000 URLs and devised a program to feed them through all four filtering programs in order to compile a list of URLs that might have been erroneously blocked by one or more of the programs.  Second, Edelman forwarded subsets of the list that he compiled to librarians and professors of library science whom the plaintiffs had hired to review the blocked sites for suitability in the public library context.  Edelman assembled the list of URLs by compiling Web pages that were blocked by the following categories in the four programs:  Cyber Patrol:  Adult/Sexually Explicit; N2H2:  Adults Only, Nudity, Pornography, and Sex, with “exceptions” engaged in the categories of Education, For Kids, History, Medical, Moderated, and Text/Spoken Only; SmartFilter:  Sex, Nudity, Mature, and Extreme; Websense:  Adult Content, Nudity, and Sex.