Well, if you buy a Mac, make sure it's one with an Intel chip - you'll be able to run Windows too.
Anyway, I could not recommend crappy Mac hardware to anyone, regardless of how much money you have to spend. The problem with the Mac platform is the hardware, not the software. When I can license OS X to run on a beige box (or in my case, a black box) I'll probably make the "switch." Until then, stick with Windows XP and a generic computer.
I would NOT recommend you build your own PC if you know nothing about them, or are unwilling to devote a lot of time to learning. You can damage your computer if you don't set voltages and memory timings correctly.
Better to stick with an off-the-shelf system.
I HIGHLY recommend HP's support. You can get an extended warranty from them where if something goes wrong they'll overnight you a box, someone will come and pick up that box the next day, and within a few days after that it'll show up on your doorstep fixed. They cover pretty much everything and opening it to add memory and stuff doesn't void the warranty (like with some other companies).
So pretty much what Flip said is correct. Get an AMD processor. Turion is pretty much made to be a laptop chip, but the Athlon 64 systems are pretty cheap now. I'd avoid the Sempron, as you get much less bang for your buck, and while it may serve you fine now, do you really want to have to upgrade again in a year when you've outgrown your PC?
Get as much RAM as you can afford, up to 2Gb. XP won't even run with less than 512mb (well, won't run to the point where you can *use* your computer) but 1Gb is pretty much essential now. If the system you're buying only comes with 512mb, just make sure there are at least two empty slots for adding more later. If you buy it in a store like FutureShop they'll likely even be able to add more memory for you right in the store.
DVD burners are cheaper than stand-alone CD burners, so you'll probably HAVE to be getting a DVD burner. They will also burn CDs, so you won't lose any functionality.
Hard drives are important, but you'll likely get a crappy one with your pre-built system. I wouldn't worry about this unless you're a power user. Any drive that ships with a PC today will be plenty fast enough for your needs.
Software-wise, I'd just go with an academic version of Office 2003 - it has Word (word processing), Excel (spreadsheets), and Publisher (desktop publishing) as well as a few other things, maybe, like Powerpoint (presentations) or Access (toy database application) depending on which variety you get.
No chip maker lists megahertz anymore, as they're trying to actually make slower chips (that also run cooler and use less energy) that do more work per clock tick. In other words, faster is not necessarily better.
For the AMD chips, the higher number is better. So basically if you see there's an "Athlon 64 3500+" and another with an "Athlon 64 4800+" the higher number is better. Those numbers represent the relative speed of the chips when compared to benchmarks of the old x86 megahertz scale. So the 4800+ is roughly equivalent to something that would be 4.8Ghz.
Yeah, it's fast.
I have an AMD Athlon 64 X2, which is the dual-core version of the chip, but I hope to make use of its dual-core capabilities in Photoshop and Illustrator, doing video editing, etc. If you can afford the dual-core, get it. You won't need to upgrade for years.
Otherwise nothing else really matters. Media Center is a nice add-on, but not a selling feature. How many USB ports it has in the front doesn't matter, as you probably won't use more than one of them anyway.
Pretty much any pre-built system will have built-in video out. So you can ignore this spec and add-on another video card later if you want to play games on it (if you do plan on maybe playing games, make sure your new computer has a PCI Express slot inside, then, rather than the older AGP).
The honest truth is that for web browsing, email, and publishing, pretty much any off-the-shelf system priced in the midrange will be fine.