It was a mixed afternoon for the O's striker, who put his side ahead on 13 minutes before he saw red for picking up two yellow cards.
But Orient, battling to beat the drop, grabbed a second goal on 70 minutes through debutant Gary Hooper before Oldham came storming back to level.
First Chris Taylor and then Andy Liddell with an 80th minute penalty levelled up this thriller.
Orient had started well and Wayne Corden's free-kick flew inches wide before the home side were forced into an early reshuffle when on-loan striker Hooper replaced the injured Shane Tudor.
But the disruption did little to dent Orient's enthusiasm and, after some clever approach play down the right, Alexander fired home from a tight angle past the bewildered Les Pogliacomi.
Oldham hit back and Paul Warne tested Glyn Garner in the home goal before Orient skipper John Mackie should have stretched his side's advantage but sent a tame header straight at the Oldham goalkeeper.
Oldham fought their way back into the encounter and Sean Gregan headed a good chance over before Warne powered a 25-yard drive against the bar.
The game exploded into life in the second half as on 49 minutes Alexander was dismissed for a challenge on Will Haining and then Oldham threw everything at the home side but could not find a way past the impressive Garner as he made an excellent save from Neal Eardley before Craig Rocastle sent an effort against the woodwork.
But Orient broke away and Hooper lobbed the home side into a two-goal lead on 70 minutes.
Oldham came storming back and Taylor cut the deficit when he headed home Liddell's right-wing cross.
Leon Clarke and Warne missed great chances before Liddell levelled from the spot after Brian Saah had tripped Richie Wellens.