In The Scarlet Letter, towards the end (around chapters 18 and 19) he and Hester are discussing running away together and starting a new life with Pearl, where no one knows them. As they both watch Pearl playing down by the brook, Dimmsdale confesses that he has often been afraid of Pearl. He is afraid that his 'parentage' would be seen in her face...in other words that she would bear some resemblance to him.
