The Sky Blues started brightly and threatened through James Hood and Joshua Archer before taking the lead when Archer released Hood down the right.
Hood did well to keep the ball in play and deliver across goal as Alex Henderson came out to meet him, leaving Rutowski with an open goal to chest the ball into the net.
The complexion of the contest changed minutes later when Whiteside was shown a straight red card for pulling down Junior Uzokwe around 35 yards from goal.
Dungannon began to make their numerical advantage count and David Walsh produced a diving save to keep out Tiernan Kelly's powerful effort, but the resulting corner brought the equaliser on 41 minutes.
Ballymena struggled to clear at the far post before Wallace bundled the ball over the line from close range.
The Swifts controlled much of the second half as they searched for a winner, with Jamie Glackin curling narrowly wide and Uzokwe heading over from Corey Smith's cross.
Walsh was also forced to save an effort from substitute Andrew Mitchell, while Ballymena defended resolutely as Dungannon continued to send crosses into the area.
The hosts came closest to completing the turnaround late on when Sean McAllister's volley deflected off Ballymena captain Fuad Sule and crashed against the crossbar.
Ballymena survived further late pressure to secure a point, while Dungannon ended a run of two successive league defeats but were unable to find a winner.