Tape has always been the standby for field recording. So I would look for a high quality portable cassette or DAT recorder.
--OR--
I don't know if there is a memory stick type of thing that would work, but that would simplify getting sounds into the computer.
What will really limit you is the microphone and how close you can get to the source. GIGO! Buy a good directional or shotgun mic for this. Outdoors a windscreen is not optional its required.
WHAT YOUR EARS HEAR IS NOT WHAT THE MIC HEARS. So you will need headphones to monitor.