The center bolt is the culprit that holds the balancer and pulles. It could be pinned and never re torqued the center bolt. Which is what Chris is hoping happened.
Lok tite
the hole you drilled really are on the seam between the balancer and the crank. so the hole is half in the balancer and half in the crank. then a hardened steel roll pin is tapped into the holes to help lock the balancer to the crank. Kind of crude but effective for a little while. not really a long time solution in my opinion. keying the crank and installing an ATI balancer is a must do when the crank is out. If you are supercharging a stock engine, the crank will come out .... sooner or later!