How did it work out for you?
I am, or was, in a similar situation. I was wearing earmuffs far too often, and it only made me more and more sensitive.
Do you use both earmuffs and earplugs, or just earmuffs?
How sensitive are you? For me, I had to start by removing them and simply walking...