The Latics were trailing when the on-loan Birmingham midfielder struck just before the interval with a 25-yard thunderbolt after latching on to a loose ball.
Doncaster Rovers had looked in control until then with big striker Mark McCammon celebrating his return to the side by firing them in front in 26th minute.
James O'Connor began the move and Lewis Guy produced a superb cross from the right for McCammon to tap home a volley from close range at the near post.
McCammon headed home following a corner a minute later, but had his effort disallowed for pushing.
It was mostly Doncaster at that stage but Oldham gradually worked their way into the game and it needed a fine save by Neil Sullivan to prevent Gary McDonald scoring with a fierce shot which was tipped over the bar in the 40th minute.
But Sullivan was given no chance when Kilkenny struck on the stroke of half-time to transform the game.
Oldham came out with all guns blazing at the start of the second half and snatched the lead in the 49th minute following a slip by Rovers defender O'Connor.
Neal Eardley crossed from the right and when O'Connor slipped out he tried to clear the ball, McDonald pounced to lash it into the net.
Eardley dribbled through to hit the post in the 61st minute and not even the sending-off of key defender Neal Trotman could break the visitors' resolve.
Trotman was dismissed for a second bookable offence six minutes from time, but Oldham deservedly held on to earn themselves a home tie against Crewe in the second round on Saturday.