They generally didn’t bore their handgun barrels nor did they always ensure the bore was straight.
In the 18th century, most gunsmiths would not have bored their barrels. They were forged around a mandrel leaving a rough bore straight out of the forge. This bore would be slowly expanded by reaming with increasingly larger bits.
Having a barrel with a straight bore was very much a matter of forging a good blank and then taking a lot of care with reaming to keep it straight. Regular references to ‘ill-bored’ pieces indicate that straight bores were not guaranteed. Lower-quality muskets made for soldiers or the African trade weren’t necessarily the best. On the other hand, gunsmiths would put a lot more effort and care into manufacturing the fowling-pieces of aristocrats.
By the late 18th century cannons were bored. They were cast solid and then send to be bored rather than reamed. Typically the process started with a small pilot hole made by a small boring bit, if they got this straight subsequent enlargement by reaming would also be quite precise. Another difference is that 18th-century practice shifted to turning the piece rather than the boring bit. Don’t quote me on it but from what I gather the centripetal forces exerted in this manner ensure the boring bit centers itself more easily.
The Dutch painter and engineer Jan Verbruggen left us some paintings of the Woolwich foundry that show the process.