Decoding MP3 file then merging with other PCM streams

I am looking to merge PCM streams containing microphone data with sound in a mp3 file. the main idea behind this is to cut bandwidth down.

Now my problem is how do I decode a given MP3 file chunk by chunk to merge with my other PCM sources for streaming.

my theoretical solution: I was thinking to first pre-decode the mp3 file on my server and save it as PCM data. (Storage is cheaper than bandwidth). Only when the server is ready to play the audio I then take chunks of the pcm data(pre-decoded mp3) and merge/mix it with the other PCM sources.

I am not sure if this would would work and would like a point into the right direction.