His first gigs were playing the banjo with the Julian Arthur Orchestra at the Hay Theater in Philadelphia in 1925. In the 30s he played with many bands in New York City, including Claude Hopkins, Chick Webb and King Oliver. Later Hilton played with McKinney's Cotton Pickers, Benny Carter and Fletcher Henderson. In the 40s he was in the bands of Duke Ellington, Ella Fitzgerald and Cab Calloway.