But England's dirttrackers hauled themselves back into the games thanks to late tries from Dan Ward-Smith and Matt Banahan which gave them a three-point lead with four minutes to go.
Berrick Barnes levelled the scores with a penalty but then missed a last-gasp kick to win the match in injury time as his effort drifted just wide.
Centre Olly Barkley kicked two penalties to give England an early lead but soon Australia were in the ascendancy with Barnes pulling the strings.
O'Connor grabbed his first try when he slipped through Barkley and winger Ugo Monye's attempted tackles to score before adding a penalty.
Playing on his home ground, O'Connor again took advantage of uncertain tackling from Lee Mears to squeeze through for his second which he converted.
The Bath hooker did redeem himself with a try after a good break from Mathew Tait but O'Connor's second penalty right on the stroke of halftime made it 18-13 to the hosts.
It continued to be a one-man show after the break as O'Connor grabbed his hat-trick try when he dotted down Josh Valentine's grubber before converting from the touchline.
Fortunately for England that would be his last involvement before being withdrawn while the boot of Barkley kept the visitors in touch.
And while the Barbarians' attacking game was far superior, England had more power in the forwards and they would use that to their advantage as Dan Ward-Smith, making his first appearance in a Red Rose shirt, barged his way over from close range.
The comeback was complete when Banahan went over in the corner after a well-worked move but crucially Barkley missed the conversion to leave England just three points ahead.
But Barnes quickly levelled the scores and in the final moments was just off target as the clock went red to give Martin Johnson plenty to ponder ahead of the first Test against Australia on Saturday.