I have a book of stamps that I possibly purchased as far back as the 37-cent stamp (but it may even be a 39 cent). However, the stamp does not have a denomination on it - it only says "First Class." Is this a forever stamp? Can I use it as a 41 cent stamp?

The most reliable way to find out would be to check with your local post office...

It's a pain in the butt to get to the post office, so I guess I'll just have to wait.

look online at the USPS website and find the stamp that matches yours and that will tell you what it is and how much it is worth

Thanks, Leslie. I had trouble finding the page at first, but found my answer: http://pe.usps.gov/text/qsg300/Q604a.htm

There should be a tiny tiny year stamped on the bottom corner of the stamp. Your stamp will be worth whatever rate hike took effect in that year.

The USPS website has a page which shows pictures of all of the stamps without the cost printed.  I had to use that while back for some stamps I had with just flags on them.  It was kinda fun looking at the other pictures too.