No, it's not limited to only your voice. It's that everything you hear sounds hollow and booming, including your own voice.
Normally, when you speak, some of the sound travels from your voice box to the cochlea directly through soft tissue and bone, but the majority of it are air waves that travel from your mouth and bounce off of different external objects and enter your ear canal and propagate down to the eardrum, which in turn moves the middle ear bones, which in turn sets the inner ear fluids in motion, which in turn causes the hair cells to move and release chemicals that create action potentials which essentially creates electrical signals that travel from the ear to the brain and you recognize this as your own voice.
How your own voice sounds to you depends on this balance between air conduction and bone conduction. When you have ear muffs on, you are blocking external sound vibrations which is the whole purpose of ear muffs, but this also includes your own voice. Such that more of the sound is conducted through the bone vibrations. This is why we often sound funny when we listen to our own voice recording, as if it's not our own voice.
But this does not only relate to our own voice, occlusion effect is evident for other sounds as well. Another person speaking to you while you have your ear muffs on will sound different than when you have the ear muffs off. Same goes for other sounds, like the sound of a bird singing. Basically all the sounds that have some high pitches will sound more "booming" when you have ear muffs on.