Ulster threw away an 11-point first-half lead against Bourgoin to slip to a second successive defeat in Pool Two of the Heineken Cup, going down 24-17.
The Irish province, under the guidance of acting coach Steve Williams following the resignation of Mark McCall this week, had a great opening 30 minutes, scoring converted tries through Kieron Dawson and Simon Danielli to build a comfortable cushion.
But the French hosts, who began the season well domestically, found their feet and gradually inched their way back into an intense clash at the Stade Pierre Rajon.
Back in touch thanks to the boot of fly-half Benjamin Boyet, they then found a cutting edge with prop Arnaud Tchougong and winger Jean-Francois Coux going over to help Bourgoin to victory.
Ulster picked up a defensive bonus point to get off the mark in the pool but their Heineken Cup hopes already look doomed after just two matches, having lost heavily to Gloucester at Ravenhill last week.
Bourgoin, who were defeated 22-15 to the Ospreys in the opening round, started brightly, Boyet just short with a long-range drop-goal.
Ulster hit back and with their first meaningful attack, they opened the scoring.
Tommy Bowe made the initial rapier-like break through the middle and fed inside to Paddy Wallace.
With just full-back Florian Denos to beat, the fly-half drew in his man before passing to his right for Dawson to ghost over.
Boyet reduced the arrears with a simple penalty in the 14th minute, but Ulster remained potent going forward and had soon enough scored a second try.
Danielli pierced the home defence by running a stunning angle onto Wallace's feed and, 45 yards out, the Scot skirted the attempted challenge of Boyet to go over in the left corner. Wallace converted for the second time for 14-3.
Boyet and Wallace exchanged penalties around the half-hour mark while the French fly-half added another in the 38th minute.
The returning Paul Steinmetz had a chance to regain the Irish province's 11-point lead before the interval whistle, but he was short and wide with a penalty in injury time.
After the break, the hosts picked up where they left off and were immediately on the front foot, testing Ulster's defence in the left corner.
The visitors braved the initial onslaught, but were caught on the hop in the 56th minute.
Bourgoin captain Julien Frier made the break, powering through flimsy tackles by Wallace and Mark Bartholomeusz, before offloading to Tchougong, who trundled over from 10 yards.
The crowd came to life and they were given further reason to be cheerful when Wallace was short with a penalty.
It reached a crescendo in the 63rd minute when Boyet broke through another powder-puff tackle by Wallace, and fired a long pass out for winger Coux to collect and bundle through Bowe's grapple to ground.
Boyet missed the conversion, leaving it a four-point game, but the Frenchmen continued to hold the whip hand and a late Boyet penalty clinched the win.