You can find directions for some great Taipei day rides here
. There are also descriptions of longer rides.
The east coast is a good place for a multiday tour if you like very rural areas. Personally, I find staying on the main highways (the 9 and the 11) rather monotonous, and I'm not familiar enough with the area to suggest lesser-known routes. Also, if you want to get to the east coast you'll have to take a train--something to keep in mind. If you really want to get off the beaten path, go along the southwest coast from Taidong to Kending. That's as far away from Taipei as you can get, but the scenery is spectacular.
On the west side, my favorite big road is the 3, which runs beside the foothills of the Central Mountain Range. The section in Miaoli County is particularly lovely. Further south, there's a 1000-meter climb between Jiayi and Tainan that will take you past the Zengwen/Tsengwen Reservoir, another beautiful area.
If you want to climb, the Central Cross-Island Highway ascends higher than any other road to Taiwan, reaching 3275 meters at Wuling. At this time of year it shouldn't be too terribly cold at the top as long as the sun is out. The Northern Cross (highway 7) only climbs to about 1200 meters, and while you could probably do the whole thing in a day, I'd recommend staying a night at Baling or Mingchi. The Southern Cross, unfortunately, hasn't been open since Typhoon Morakot.
Of course, if you want, you could throw planning out the window, take a train to whatever part of the country you feel like exploring, and wander around for a couple days. I did that in southern Taiwan a couple years ago and had a great time. The south is so warm that you'd be comfortable riding around in a tank top. The north might get a bit chilly depending on the weather.
Have fun! And when you get back, tell all your cycling friends how awesome Taiwan is.