Wales centre Gavin Henson started his first game for six months - but the Ospreys almost made a spectacular crash-landing at Liberty Stadium.
The EDF Energy Cup holders began their trophy defence in stuttering fashion, edging an under-strength Harlequins 24-23.
Quins were without most of their Guinness Premiership players, including England quartet Danny Care, David Strettle, Mike Brown and Nick Easter, but they came within seconds of winning.
Ospreys substitute Tommy Bowe scored a breakaway try three minutes into injury-time, that fly-half James Hook converted in nerveless fashion from the touchline.
Hook earlier booted four penalties, giving him 14 points, while wing Shane Williams scored from the game's best move.
Henson, confined to a brief substitute's appearance since suffering an ankle injury against Heineken Cup quarter-final opponents Saracens last season, made little impact.
But he was not alone, as Ospreys' huge contingent of international players were pushed to the limit by old-fashioned bravery and determination.
Quins nearly delivered the surprise result of this season's Anglo-Welsh tournament so far thanks to second-half tries from centre De Wet Barry and hooker Tani Fuga.
Fijian fly-half Waisea Luveniyali added three penalties and two conversions, yet Quins could ultimately only reflect on a heart-breaking defeat.
Henson featured in a star-studded Ospreys team determined to reach its third successive Anglo-Welsh final at Twickenham.
Wing Jonny Vaughton and scrum-half Rhys Webb were the only uncapped players among Henson's colleagues, but Quins predictably showed 14 changes following a punishing Premiership outing at Worcester three days ago.
Flanker Chris Robshaw was the solitary survivor in a team captained by England's 2007 World Cup scrum-half Andy Gomarsall.
But Quins were not daunted by their task, despite almost conceding a first-minute try when Williams threatened to cash in on weak defence.
Hook booted two penalties for a 6-0 lead after 25 minutes, with Quins only opening their account through a Luveniyali penalty on a first excursion into Ospreys' 22.
The Welsh were side were kept in check through a combination of their own over-eagerness and Quins' admirable commitment, but they conjured a memorable try as half-time approached.
Hook's defence-splitting pass sent lock Alun Wyn Jones galloping clear, before some slick passing saw Henson put Williams over in the corner.
It was a move that would have unlocked most defences, and Quins knew they had to keep things together as the interval approached.
But they trooped off one player down after referee Tim Wigglesworth had little option other than to sin-bin Quins' powerhouse Tongan wing Epi Taione.
Taione's illegal tackle on Hook saw Wigglesworth reach for a yellow card, and the Ospreys were close to taking charge at 14-3 ahead.
Quins though, dominated the early second-half flurries after Ospreys' former Ulster wing Bowe replaced Sonny Parker alongside Henson in midfield.
Wing Charlie Amesbury attacked the Ospreys from deep, and although Robshaw was denied a try from an ensuing ruck, Tosh Masson sent his centre partner Barry over for a well-crafted score.
Luveniyali added the extras, giving Ospreys a piercing wake-up call after they had done little to support many pre-match predictions from outside the camp of posting a landslide win.
And matters deteriorated for the home side after 56 minutes when the Quins forwards again produced irresistible momentum, and Fuga touched down under a pile of bodies.
Referee Wigglesworth required confirmation from video official Tony Rowlands before the try was awarded, before Luveniyali's conversion put Quins 17-14 in front.
Luveniyali then kicked a 45-metre penalty, which was quickly cancelled out by Hook's short-range strike, before an intriguing Ospreys tactical switch saw Williams move to scrum-half.
But the Ospreys were their own worst enemies, with Jones being sin-binned for a technical infringement and handing Luveniyali an easy penalty chance that he gratefully accepted.
Jones and the Ospreys though, were saved by Bowe's dramatic late score that was created by full-back Lee Byrne's break and precision kick into space.
Peter O'Mahony was named man of the match against England last month
21 April 2017, 7:22pm
Peter O'Mahony's performance in the RBS 6 Nations finale for Ireland against England was crucial to his selection for the British & Irish Lions on Wednesday, Ireland and Lions defence coach Andy Farrell has revealed.