Willy is envious of both of them. Charley, the next door neighbor, is the realized version of Willy and Willy understands that on some levels. This is also compounded by the fact that Willy has borrowed money from Charley, which he can never repay. Charley offers Willy a job but Willy is too proud to take it. So, Willy hates him even more.
Charley has a son named, Bernard, whom Willy and his boys used to tease and make fun of when they were younger. Now, of course, Bernard is a big time lawyer, making loads of money, and more succssful than either of Willy's sons will ever be.