The home side were 2-0 down with just 12 minutes to go and trailed again in stoppage time after getting back to 2-2 but Andy Liddell scored from the penalty spot to ensure a six-goal thriller ended all-square.
It was Orient's first away point since they won promotion from League Two in May but their defensive problems came back to haunt them.
Chris Porter bagged two for Oldham to match visiting striker Gary Alexander who marked his recall to the side with a fine performance.
Alexander set up the opener for Orient on nine minutes, although Oldham were convinced he was offside when a defensive header was deflected into his path.
Alexander crossed from the right and Jabo Ibehre scored from close range to open his account.
The visitors stayed on top and Michael Simpson went close from well outside the box before Oldham keeper Les Pogliacomi made a fantastic double save.
The Australian kept out Alexander's header with his left hand before scrambling across the line to deny Simpson.
Oldham livened up after the break and Porter and Paul Warne both wasted half-chances. But Alexander scored on 59 minutes as he found space on the right and hit his shot into the far corner from 12 yards.
Oldham began their comeback when Richie Wellens found Porter at the far post for a neat header but Orient had already squandered a great chance to go three up when Alexander hit the bar.
Oldham were level on 82 minutes when Porter produced a fine header from 10 yards after Liddell fired over the cross.
Orient responded by regaining the lead when Alexander hit a superb drive into the far corner while Oldham's defence went to sleep.
But in stoppage time Wellens was dragged to the ground by Justin Miller and Liddell sent Glyn Garner the wrong way.