Michael Phelps has been the dominant figure in swimming for the past three Olympiads, and Usain Bolt has been the dominant figure in track and field for the past two.
Phelps had his first taste of the Olympics at age 15, in 2000. After that, his goals obviously centered around world records, world championships, and, mostly, future Olympics.
Phelps turned pro as a high schooler, never competed collegiately, and thus never really focused on competitions conducted in 25 yard pools, which are actually the most common in this country. He competed in yards meets a few times, mostly in Baltimore, and set a couple of American records at them, seemingly almost by accident. But he never took a serious run at the yards record book.
Bolt also competed in the 2004 Olympics, but did not do well. He became a household name in 2008 at the Beijing Olympics, furthered his reputation with his remarkable world records at the world championships in Berlin in 2009, and cemented his legacy as the greatest sprinter in history at the London Games.
Bolt has never bothered to compete at any of the major indoor meets. I've never seen a time listed for him for the 60 meter dash, or the 300.
There's a certain telling symmetry here. If you're that big, why bother to compete on a smaller stage? Leave that to the mortals, who have to scramble for glory where they can find it.