He and two other African-American students desegregated their junior high school in the early 1960s. He remembers the catcalls and having to live with two African-American teammates in a $100-a-month hotel, rather than a host family, in Bluefield, W.Va., while playing Rookie ball in 1967.
"When I got to the Majors with the Orioles, we had a lot of African-American players on the same team," Baylor said. "I remember Brooks Robinson used to say, 'Now there are 13 of you guys. You're outnumbering us. I've gotta make a move. We've got to get the majority back.' We had guys who never blinked an eye. You showered with guys. You played cards with guys. You ate with guys. You just ran with whoever, black or white.
"It's hard for young players to understand. You walk out on the field and no one wants to play catch with you. Or no one wants to stand in the shower with you. Wow."
In a sense, the day was strange. Everyone who came to bat had the same number. To know who was warming up or pinch-hitting, one had to recognize the player's face or, from a distance, his gait or his swing or his motion. Yet, it wasn't all that strange, because players recognized the importance of remembering the player who helped made baseball represent what the country was supposed to represent.
"It's different," said Rockies shortstop Troy Tulowitzki, who went 1-for-4 and scored a run in the Rockies' 5-2 victory over the Cubs. "You're used to seeing guys with their number on. But when everybody has 42 on, it's just a reminder that he meant a lot to the game.
"It's quite an honor to wear that number. He was a great player. A lot of people looked up to him. He really changed the way people viewed the game. And he played the game the right way. I respect him."
For Dexter Fowler, when he knocked a pinch-hit, RBI double in the eighth, he was wearing a reversal of his normal number, 24, which he wears 24 because he grew up a fan of Ken Griffey Jr.
It didn't take making it to the Majors for him to have awareness of what Robinson means. Fowler, a rookie, is the Rockies' only African-American player. They have African-American coaches, Baylor and first base coach Glenallen Hill.
Fowler said he called former Rockies pitcher LaTroy Hawkins, now with the Astros, when he saw Hawkins wearing it with Team USA during the World Baseball Classic.
Robinson's importance was taught to him by his parents, John and Trudy Fowler, when he was growing up in the Atlanta area.
"The book, the movie ... we had this little documentary," Fowler said. "We sat and watched it as a family. It's a special day. It means the world. Without him, we wouldn't be out there playing baseball."
Baylor was a young Major Leaguer in 1972. Robinson had boycotted going to games because of the lack of coaching, managing and off-field opportunities for minorities at the time, but agreed to throw out the first pitch during the World Series on Oct. 15, 1972. In a televised interview, he said, "I'd like to live to see a black manager. I'd like to live to see the day when there's a black man coaching third base."
Robinson, however, died of a heart attack nine days later at just 53. It wasn't until the Indians hired Frank Robinson as player-manager in 1975 that Baylor said he thought of the possibility of advancing in the game. Baylor eventually became the Rockies' original manager from 1993-98 and managed the Cubs from 2000-02.
Baylor said he met Olympic gold medalist Jesse Owens once during Spring Training but never met Jackie Robinson. But he appreciated all that he did for him.
"You play with Frank Robinson and you're around Vada Pinson and Bob Gibson, guys that had gone through tougher times than we did at the Minor League level," Baylor said. "I can't imagine what Jackie Robinson felt like, and he still had his dignity and everything else about him."
Thomas Harding is a reporter for MLB.com. This story was not subject to the approval of Major League Baseball or its clubs.