Why is Charlie so understanding about the hostile Attitude that his friends at the bakery have towards him as his intelligence is raised?

I don't thing Charlie is as understanding of their actions as he is about understanding their true intent. Charlie never understood that the people he believed were his friends were actually make fun of him..... after his operation, he was ashamed at having been fooled.