Oldham started brightly in their yellow shirts, particularly Chris Taylor. Within three minutes he had side-footed narrowly wide and forced a decent save from Owain Fon Williams.
Stockport finally got a grip and Anthony Pilkington was unlucky when he couldn't quite direct a flying volley on target, but at the other end the visitors replied when Andy Liddell saw a shot blocked. The Oldham midfielder picked up the rebound, only to blaze a second shot high and wide.
Liddell wasn't to be denied on 24 minutes though, thanks to a fortuitous opening goal. His in-swinging corner-kick flew straight in at the far post after Fon Williams had missed an attempted punch.
The goal led to a hold-up in the game after some Oldham fans appeared to have infiltrated one of the Stockport enclosures and fighting spilled onto the pitch, but order was swiftly restored and Craig Davies fired a low shot inches wide for the Latics.
Pilkington headed inches over for Stockport just before the break.
Stockport levelled matters less than two minutes after the restart. Peter Thompson flicked Tommy Rowe's cross into Paul Turnbull's path, and he smashed the ball home on the volley from just six yards out for his first senior goal.
The hosts were buoyed by the goal and Pilkington's shot forced veteran keeper Mark Crossley to make a save, only for Oldham's Danny Whitaker to produce a lovely bit of skill as he lifted a cute chip inches over the crossbar.
After a pulsating second half, Stockport went in front on 68 minutes. A long James Tunnicliffe throw-in was flicked on by Raynes, and there was Thompson who lashed home another volley - the Northern Ireland striker's second goal in as many games.