Once you have a a GPS logger set it to record at an appropriate interval. If you set them to record too often on a long trip, you will run out of memory. But the more often they log the more accurate the geocoding process is. For a new GPS logger, logging every second of a weekend trip will work find. You can use many to record every 10 or 100 meters so you save memory when you are not moving. There are lots of settings to play with.
Once set up and recording, make sure the GPS has a clear view of the sky, place it in the top of your pack, avoid burying in or down the side. Consider parting with the cash to buy a dedicated GPS logger (http://shop.ebay.com.au/?_nkw=gps+logger) they are generally much better than the other options mentioned.
Taking your photos
Take your masterpieces as you usually would, just make sure you have your GPS with you and that it is logging. One handy trick is to set the camera's clock close to the time on the GPS clock, and don't change it. It is more important that the time is consistant than accurate. So don't go changing the camera clock half way through a trip.
Working out the time differance
To make sure the photos align well, we want to note the time between the camera and GPS. Most syncing software has an inbuilt calculator to do this. But to know the time stamp differance take a photos of the GPS clock, or another accurate UTC clock (time.gov). Then you can see the official time in the photo and the EXIF data stores the camera's time.
Syncing the photos and GPS data
Follow your camera and GPS instructions to upload your images and GPS data to your computer. There are many programs that can be used for geotagging your photos, and many are free. My favourtie is gpicsync It works on most operating systems, is free and opensource. Download it and get started. It is fairly easy to follow but if you get stuck, try yelling at the computer and if that does not work then try reading the instructions.
Where in the world are my photos?
Now the photos know where they were taken, so what next?. Gpicsync creates a 'KML' file that you can open in Google Earth and see your photos and route, it is pretty cool. To manage a larger number of trips you can load your photo into Picasa (free). Picasa is a cute photo manager that allows you to do many basic functions like crop and rotate, but also allows you to view you photos on Google Earth.
You can also share your geotagged photos with the wider world through services like Picasa web, Panoramio, and flickr.
If you have money to spend and think this is pretty cool stuff then you can get a camera with built in or clip on GPS such as a Canon or Nikon digital SLR. This way the photos are geotagged when you press the shutter release, no need to do this syncing process. Also you can get a GPS with a digital compass so you can even tell which direction the camera was looking.
If you want to geotag photos that you do not have GPS data for, then you can use Picasa and Google Earth. Picasa makes this process much less painful. You select the photos, and it opens Google Earth, then you point to where the photo was taken, and Picasa updates the EXIF data.
Some bonus Tips