Thread Small portable recording device

I am new to sound recording and wanted a portable device to record family gatherings (storytelling), friends playing musical instruments and singing, children talking and various environments while travelling. I just recieved an Olympus WS310M digital recorder and ME51S microphone that I ordered from an online shop. Maybe it is just me or I am expecting too much, but the recording results were terrible! The background hiss that is apparent when recording in a silent room is horrendous, and there are faint little electronic noises (artifacts?) apparent in the background. This is at the highest quality recording settings. I can't believe the engineers at Olympus allowed this device to be sold in its present form. I swear my Dad's 20 year old portable tape recorder had less static hiss than this thing. Needles to say I am attempting to return these devices.
Can anyone recommend a device for me that won't create recordings that make a silent forest sound like a beach onto which waves are constantly crashing? I realize you get what you pay for and will sacrifice digital interoperability for quality if need be. I would spend around $US200 (I am in Australia). I wanted to go digital for archiving and editing purposes. Perhaps a little handheld tape recorder with a decent mic might be apt?