Many people pass Dimmesdale by in this chapter; why are so many people up so late and walking about?

Many of the people who walk past Dimmesdale are on their way to or from the old Governor's. The governor is lying close to death, and people are paying their respects, which is why they are out and about so late at night. These people include Mistress Hibbins, the governor's sister and another minister, among Hester and others.