You Can't Say You Can't Play is the story of Kindergarten teacher Vivian Paley's attempt to teach her class not to treat some children as inferior to others by systematically excluding them from playing with others. Paley sees this process of exclusion hardening as children age, and continuing long into adulthood. She believes that if children can be taught to treat one another as equals no matter what, this lesson could follow them throughout their lives.

In Chapter 1, Paley outlines the problem. The phrase "You can't play" is the beginning of the formation of a social hierarchy, and makes some children feel awful about themselves and sometimes leaves them isolated. Paley writes a story to teach the pain of rejection to her students that feature a variety of characters, which initially includes a magpie named Magpie, a young princess named Annabella and her father, a prince named Kareem. She reads the children the story throughout the book, writing it day by day alongside her attempts to root out social hierarchy and social rejection among her students.

The story is used to teach the children about fairness, rejection, how to treat others as equals and include them, and the good things that result from practicing these ideals. Paley comes up with an idea for a social rule, which is also the title of the book - "You can't say you can't play." In other words, that little phrase "You can't play" or any of its variants may not be used. And all children must accept all others. She realizes, however, that this rule will come as a shock to her students and so she embarks on two tasks. First, she discusses the rule with older students, first, second, third, fourth and fifth graders. Second, she prepares her students for the rule with her story and discussions of their own.

In Chapter 2, Paley conducts the inquiry among the older students, taking notes on the attitudes of these students towards the kindergarten experiment that she is proposing. They are skeptical about having the rule applied to themselves, believing that it will insulate them from harsh reality, interfere with friendship, and fail due to natural character flaws. They, however, acknowledge that it is fair. They think it might work for the Kindergartners. Alongside the interviews, Paley unfolds her story about Magpie, Annabella, Kareem and later Alexandra, Beatrix, Raymond, the flower people, the golden eagle, the ship captain and his crew, the dragons and Raymond's father Corporal Thomas (the dragons and Raymond's father do not arise until Chapter 3).

In Chapter 3, Paley readies her class for the experiment and in Chapter 4, she implements it. Paley finds that the rule has interesting consequences. After a short while, the rule becomes self-reinforcing. The children and Paley must work out the application of the rule in a case by case fashion but the children often invest creative methods of applying the rule. The rule sometimes faces serious challenges, but for the entirety of Chapter 4, it perseveres. Paley sees some costs but believes implementing the rule has brought wonderful improvements.