University of Washington’s Information School, to help him identify which sites were appropriate for library use.  We describe the process that he used in the margin.  Due to the inability of a member of Janes’s review team to complete the reviewing process, Janes had to cut 157 Web sites out of the sample, but because the Web sites were randomly assigned to reviewers, it is unlikely that these sites differed significantly from the rest of the sample.  That left the sample size at 699, which widened the 95% confidence interval to plus or minus 2.8%.

Of the total 699 sites reviewed, Janes’s team concluded that 165 of them, or 23.6% percent of the sample, were not of any value in the library context (i.e., no librarian would, consistent with professional standards, refer a patron to these sites as a source of information).  They were unable to find 60 of the Web sites, or 8.6% of the sample.  Therefore, they concluded that the remaining 474 Web sites, or 67.8% of the sample, were examples of overblocking with respect to materials that are appropriate sources of information in public libraries.  Applying a 95% confidence interval of plus or minus 2.8%, the study concluded that we can be 95% confident that the actual percentage of sites in the list of 6,775 sites that are appropriate for use in public libraries is somewhere between 65.0% and 70.6%.  In other words, we can be 95% certain that the actual number of sites out of the 6,775 that Edelman forwarded to Janes that are appropriate for use in public libraries (under Janes’s standard) is somewhere between 4,403 and 4,783.

The government raised some valid criticisms of Janes’s methodology, attacking in particular the fact that, while sites that received two “yes” votes in the first round of voting were determined to be of sufficient interest in a library context to be removed from further analysis, sites receiving one or two “no” votes were sent to the next round.  The government also correctly points out that results of Janes’s study can be generalized only to the population of 6,775 sites that Edelman forwarded to Janes.  Even taking these criticisms into account, and discounting Janes’s numbers appropriately, we credit Janes’s study as confirming that Edelman’s set of 6,775 Web sites contains at least a few thousand URLs that were erroneously blocked by one or more of the four filtering programs that he used, whether judged against CIPA’s definitions, the filters’ own category criteria, or against the standard that the Janes study used.  Edelman tested only 500,000 unique URLs out of the 4000 times that many, or two billion, that are estimated to exist in the indexable Web.  Even assuming that Edelman chose the URLs that were most likely to be erroneously blocked by commercial filtering programs, we conclude that many times the number of pages that Edelman identified are erroneously blocked by one or more of the filtering programs that he tested.  Edelman’s and Janes’s studies provide numerous