I guess I lived in a different part of Taoyuan than you lot then (and did for over a year), as it's not a pit, nor is it most of what you lot have described it as.
Sure, it's not Taipei or Kaoshiung (Gaoxiong), but it's a decent enough place and seems to be getting better.
Yes, there aren't loads of night clubs or party places, but there are other things you can do there on a weekend.
If you're into sports and outdoors stuff, it's a decent place, there's even a Go-Kart track there, although I never got around to trying it out, as they just finished it off as we moved back to Taipei. And no, we didn't leave because we didn't like it there, my girlfriend works in Taipei and the fuel prices went up like crazy at the time, so we decided to move.
Anyhow, there are plenty of good places to eat there and enough places to shop western things to survive if you can't make the shift to live off the local stuff.
I think the best Singaporean restaurant in Taiwan is in Taoyuan, although can't say I've seen a lot of other Singaporean restaurants on the island.
There's also a fantastic fruit market there with a great selection and good prices.
Public swimming pools are also easy to get to and you could get a bicycle, as the place isn't quite as crowded in terms of traffic as Taipei. There are also some nice parks there (at least in the area where I lived) and there's a local IKEA (which was a boon for me).
I'm sure there are some nasty parts of the town too, but I didn't go to those places.
For your reference, I was living in western Taoyuan City, not too far from the number 1 highway.
To get to Taipei you can either take the slow poke train from the main train station, or the high-speed rail, but that train station is out in the sticks, although there's supposedly a shuttle bus that goes there, but it didn't go past where I lived. You're stuck with using bus otherwise, although most of the big supermarkets have free shuttle buses, which might come in handy and Taxi isn't that expensive in Taiwan.
Some Canadian guy opened up a Pizza place there a little while ago (haven't visited it, as it too happened around the time we left) and Debbie's is a great little diner for weekend brunch.
Yes, it's not Taipei and you're going to want a mate or two that lets you use their sofa on the weekends if you want to party in Taipei, as that's the biggest downer in Taoyuan. You'll also want to head into Taipei from time to time to get some stuff you can't get in Taoyuan.
Also, one good thing is that you can either get a bigger place for the same rent money, or a cheaper place compared to Taipei.
Ok, the part about the beetle nut girls is actually true...