Who is the antagonist in Silas Marner by George Eliot?

Dunstan Cass is visibly the most prominant antagonist in Silas Marner, although many consider Silas an antagonist, as well. Silas' mistrust of other people, as well as his inexperience, serve to make him his own worst enemy on more then one occasion.