Well, let's see....
Your best bet for an amp would be a small combo amp (30 to 60 watts or so- big enough to play with someone else in a low volume environment). You'll need a strap, too. Picks are optional- most bass players play with their fingers, although some use picks. Some do both depending on what music they're playing.
If you get a combo amp with a line output built in, you can run that to a small mixer and then into the line input on your computer soundcard. You can record your bass that way.
But I'd start with the bass, a small combo amp, a strap, and some lessons.