It depends on the port. On intake ports, you usually leave the ports quite rough - with a final sandpaper roll grit of around 60-80 or so (for the reasons that 38 Coupe mentioned).
On the exhaust ports, we tend to polish them to a much higher degree - to achieve a more mirror-like finish. I usually go to about a 400 grit.
In combustion chamber areas, we tend to use the finer grits (like the exhaust ports) - somewhat depending on whether or not you're using ceramic coatings and other such fun.
On ceramic heat/barrier type coatings, you usually prepare the surface with a sand blast grit (not glass-beads) of around 120 or so - as the coatings need the surface prep to achieve a good adhesion.