Public elementary school is "free". Admin fees of about $1200 are payable per semester, but you can apply for a grant if you are a low-income family. You have to pay for the lunch, in our case $400/month for a lunch cooked on campus.
Public kindergarten is free for kids over five but again you pay for the food and some materials fees, which was about $4,000 for 4 1/2 months this time round for us (2 snacks and 1 meal per day, care from 7:40 am till 3:30pm or so).
I wrote a couple of articles about our experiences so far on my blog if you are interested:
http://kidzone-tw.blogspot.com/search/label/Taiwan%20Education