It's most likely the other way around. When hearing damage occurs, the brain is getting fed less sensory input. In the long term, parts of the brain receiving less stimulation wither and die leading to potential cognitive problems.
The brains needs constant information to function properly.