The origin of the quote is unclear.
A well-known scientist (some say it was Bertrand Russell) once gave a public lecture on astronomy. He described how the earth orbits around the sun and how the sun, in turn, orbits around the center of a vast collection of stars called our galaxy.
At the end of the lecture, a little old lady at the back of the room got up and said: "What you have told us is rubbish. The world is really a flat plate supported on the back of a giant tortoise."
The scientist gave a superior smile before replying, "What is the tortoise standing on?"
"You're very clever, young man, very clever," said the old lady. "But it's turtles all the way down!"
As a Dicworld fan, it seems the obvious answer would've been that the turtle swims. That's what turtles do!