categorized sites and pages.  Typically, a filtering software vendor’s previous categorization of a Web site is not re-reviewed for accuracy when new pages are added to the Web site.  To the extent the Web site was previously categorized as a whole, the new pages added to the site usually share the categorization assigned by the blocking product vendor.  This necessarily results in both over- and underblocking, because, as noted above, the content of Web pages and Web sites changes relatively rapidly.

In addition to the content on Web sites or pages changing rapidly, Web sites themselves may disappear and be replaced by sites with entirely different content.  If an Ip address associated with a particular Web site is blocked under a particular category and the Web site goes out of existence, then the Ip address likely would be reassigned to a different Web site, either by an Internet service provider or by a registration organization, such as the American Registry for Internet Numbers, see http://www.arin.net.  In that case, the site that received the reassigned Ip address would likely be miscategorized.  Because filtering companies do not engage in systematic re-review of their category lists, such a site would likely remain miscategorized unless someone submitted it to the filtering company for re-review, increasing the incidence of over- and underblocking.  This failure to re-review Web pages primarily increases a filtering company’s rate of overblocking.  However, if a filtering company does not re-review Web pages after it determines that they do not fall into any of its blocking categories, then that would result in underblocking (because, for example, a page might add sexually explicit content). 3.  The Inherent Tradeoff Between Overblocking and Underblocking

There is an inherent tradeoff between any filter’s rate of overblocking (which information scientists also call “precision”) and its rate of underblocking (which is also referred to as “recall").  The rate of overblocking or precision is measured by the proportion of the things a classification system assigns to a certain category that are appropriately classified.  The plaintiffs’ expert, Dr. Nunberg, provided the hypothetical example of a classification system that is asked to pick out pictures of dogs from a database consisting of 1000 pictures of animals, of which 80 were actually dogs.  If it returned 100 hits, of which 80 were in fact pictures of dogs, and the remaining 20 were pictures of cats, horses, and deer, we would say that the system identified dog pictures with a precision of 80%.  This would be analogous to a filter that overblocked at a rate of 20%.  The recall measure involves determining what proportion of the actual members of a category the classification system has been able to identify.  For example, if the hypothetical animal-picture database contained a total of 200 pictures of dogs, and the system identified 80 of them and failed to