Of what "irresponsibility" are sleepwalkers and dreamers accused in the novel, Invisible Man?

Sleepwalkers and dreamers are accused of the irresponsibility of failing to recognize both their own effect on society and the needs and contributions of others around them.