How does Samuel sneak Virginia out of James Town in Chapter 26?

Samuel takes the baby from John Laydon with the pretext of taking her for a walk, then he takes her a canoe where he casually announces to some women nearby that he is taking her to Point Comfort, expecting that Ann will hear this and follow him there.