Well do you have stereo when playing a stereo track? It might sound as a dumb question but perhaps something is set to mono.
If you have stereo with things like youtube then it is a software problem in reaper.
Again i think it will be more like a routing error than reaper it self.
If not stereo with youtube it can be a routing error in the softmix.
Little hums can come from everything or nothing.
Disconnect everything turn off every daw and connect only your speakers. Do you still have a hum? Then the problem can be bad shielded cable or a broken wire ( hot or cold pin)
Please let us know... From there we can help you further...