Forest defender Danny Collins was sent off after an hour at the City Ground as Oldham struck three times in the second half, twice from Robbie Simpson, to give Paul Dickov's outfit a deserved victory.
Oldham striker Matt Smith put through his own goal from just six yards in the first half before the Latics struck back with three quick goals in the secons half, while Billy Sharp's added-time effort for Forest was too little, too late.
Luton recorded an FA Cup giantkilling to extend Wolves' misery and pile the pressure on under-fire boss Stale Solbakken, whose position must now be under serious threat following the 1-0 loss.
Alex Lawless was the matchwinner with a well-taken volley within seconds of the restart after half-time, ensuring the non-league side will be in the hat for tomorrow's fourth-round draw.
Bournemouth had to settle for a replay against Wigan after a 1-1 draw at the DW Stadium
The League One side took the lead four minutes before the break thanks to a fine goal by Eunan O'Kane, but the Premier League side levelled through a fortuitous penalty with Jordi Gomez's spot-kick parried by the keeper but the midfielder pounced on the rebound to slot home with 20 minutes left.
A stoppage-time goal by Nick Proschwitz saw Hull scrape a 1-1 draw against Leyton Orient. The League One side went ahead with 12 minutes to go when Dean Cox's cross was met with a bullet header by David Mooney, but Proschwitz's downward header earned a replay for the Championship high-flyers.
Fulham needed an equaliser 10 minutes from time to earn a 1-1 draw at home to Blackpool. Ludovic Silvestre put the Seasiders ahead with half an hour to go when he fired home a 25-yard shot.
However, Robert Harris' clearance from Bryan Ruiz's corner could only find Giorgos Karagounis on the edge of the box and he looped a volley into the net.
Source: PA
Source: PA