BEIJING—"People are very complimentary. They say thanks for all the memories," he said. "To me, it's a form of flattery."
If he starts signing autographs on the street, a crowd might form, but most people are happy with a quick exchange, a handshake or perhaps a photo.
He said privacy is harder for athletes in some cities. In Boston, a Red Sox star can't wander to a hotel checkout desk without being swarmed. But Baltimore stars have a long history of sliding comfortably into the community.
"Our models were the old Colts," Palmer said. "They used to tell us, 'If you're going to live here, we want you to be a part of it.' That might mean speaking or working for a charitable cause. ... It was easy to integrate into the society here."
It will be a difficult juggling act for Phelps, living the life of a 23-year-old while balancing his responsibilities as a corporate pitchman. Early speculation predicts he'll be worth $100 million over his lifetime to companies with products to push. But those companies expect him to steer clear of embarrassing situations.
He stumbled the first time he tried it, getting arrested for driving while under the influence shortly after the Athens Games; he was sentenced to 18 months' probation. There will be, once again, temptation at every turn. He has many high school friends who have expressed excitement over his return to town, ones who have been shooting him text messages throughout the week, teasing him and keeping him humble.
"It's going to be a cool time to catch up with all of them," Phelps said.
When a reporter said jokingly last week that Phelps was about to start living the real-life version of the HBO series Entourage, he laughed and acknowledged that it might not be far from the truth.
"It would be pretty cool to have, like, a Turtle and a Johnny Drama and those guys," Phelps said, referring to characters on the show. "That would be pretty neat. That's one of my favorite shows. It would be pretty cool to have someone who just drives you around all the time and plays video games. That would be fun."
His personal life will also be subject to a new level of scrutiny - an unavoidable trade-off that comes with fame. Already, the Internet has been flooded with rumors about whom he is dating, and being tailed by the occasional pack of paparazzi might be something he has to adjust to.
His Facebook page has exploded - he's closing in on 1.3 million fans and is just behind the No. 1 page of Barack Obama.
"I think it's going to be fine in Baltimore," Phelps said. "I don't think I'll get hassled. I think I'll be able to live my life how I want to live it. And people will respect that. That's the kind of town that it is. There are amazing people, there are so many amazing athletes that have come out of that city, and everyone respects everybody."
In Fells Point, residents and people who work there are hoping Phelps is given his space.
"People are dying to see him," said Catherine McGrain, 19, a dockhand at Henderson's Wharf Marina. "I got pretty excited when I heard about it. Maybe I could show him around town."
"I'm as happy as I could be. He sounds like a swell kid," said Lew Diuguid, 73, a Fell Street resident. Diuguid edits The Fells Pointer, a monthly newsletter.
"If he likes Fells Point, then we like him," Diuguid said. "But we'll also be looking for his help. For instance, I belong to the Baltimore Harbor Watershed Association, which works to clean up the environment - and, in Phelps, we've got a waterman if ever there was one."
But some are worried about sightseers.
"Somebody from the Olympics is moving to Fells Point? It'll probably get crowded down here," said Mike Bryan, assistant superintendant of maintenance of The Condominiums at Henderson's Wharf, who was hosing down the sidewalk in front.
Phelps will try to lie low, at least as much as possible. He'll be reunited with Herman, his English bulldog, and some evenings it will be just the two of them, adjusting to their new life. With Phelps so frequently on the road, Herman has been living with a family friend for several months.