take this from somebody that drilled dozens of holes every day for a living. Your standard two flute twist drill will not drill a round hole. Nature of the design. A four flute is much better, and an uncommon six flute is that much better. You don’t see it as much with a pin vise, but the regular drill bits also tend to wonder unless your using a guide bushing. Not uncommon to see them move about ten percent of the diameter or even more. This is why you always drill a hole in steps going from a smaller drill bit to larger ones. For a round hole, you really want to finish it out with a reamer that removes about .010".
The above is why you almost always try to punch the hole with a two flute end mill in a milling machine. Then ream it to size. Another issue you get into with a pin vise is the surface speed of the cutting edges verses the material your cutting. A typical # 60 bit in steel wants a minimum of 3000 rpm, and with plastics your lookinh at 4800rpm. Never going to get there with a pin vise! Never liked carbide all that much, and there are different grades of carbide. They break too easally and are a real bugger to remove.
The strait flute carbide drills are far better, but also a little more noney. Still they are harder than your ex wife’s breath in court.
gary