Many musicians were inspired by Oliver's great talent because he could get so many different sounds out of a trombone. When it came to playing the cornet, his fellow musicians learned about the use of mutes because Oliver was called the master of the various ways to use mutes. He left New Orleans when all the musicians were going up to Chicago to make recordings. For awhile, he joined the band of Bill Johnson at the Dreamland Ballroom.

He eventually formed his own band and played at the Lincoln Gardens, and then he bought former student Louis Armstrong (who called him "Papa Joe") up to join the band. Lil Hardin was playing piano and that was where Louis met and married her. "King" Oliver's Creole Jazz Band made some recordings at that time. Oliver said he liked keeping Louis with him playing second trumpet because that way, he was still "The King."

Musicians said the "vocalized" trumpet sound heard in Duke Ellington's Orchestra had been brought in by musicians who had played with Oliver. When Louis Armstrong left Oliver to play with Fletcher Henderson's orchestra, "The King" soon followed him to New York.