At the beginning of the book, Tom's brother is ill with the measles. To protect him from catching the disease, his parents send Tom to stay with his aunt and uncle in their small flat in the city.

This is a drastic change for Tom, who is used to spending hours in the garden behind his family's large house. His aunt and uncle do not have a garden; in fact, there is not even a yard. Several scenes take place inside the flat or elsewhere in the spacious old house that has been subdivided into smaller flats, but most of the action takes place in the mysterious garden that appears after dark. This elusive garden, Tom eventually learns, actually existed some fifty years earlier but it has since been paved over and is used to store garbage cans.