Mobley was born in Eastman, Georgia, but was raised in Elizabeth, New Jersey, near Newark. When he was 16, an illness kept him in the house for several months. His uncle thought of buying a saxophone to help him occupy his time, and it was then that Mobley began to play. He tried to enter a music school in Newark, but couldn't, since he was not a resident, so he kept studying through books at home. At 19, he started to play with local bands and, months later, worked for the first time with musicians like Dizzy Gillespie and Max Roach. He took part in one of the earliest hard bop sessions, alongside Art Blakey, Horace Silver, Doug Watkins and trumpeter Kenny Dorham. The results of these sessions were released as Horace Silver and the Jazz Messengers.