The Blues, fresh from their EDF Trophy success over Gloucester, drew first blood on a damp night at Ravenhill after only three minutes with a Ceri Sweeney penalty.
Ulster then lost Paddy Wallace after the Ireland international centre was involved in a heavy tackle in midfield but levelled the scores through Niall O'Connor's drop goal.
Clinton Schifcofske was then wide with a kickable penalty before making amends to give Ulster the lead on 23 minutes.
The home side were then unlucky to miss out on the opening try after David Pollock's charge down nearly saw Darren Cave receive a scoring pass but, two minutes before the half hour, some neat handling from the Blues allowed full-back Loxton to score.
Gareth Thomas produced a neat offload with Jamie Robinson getting the assist after a deft chip over the Ulster line.
Sweeney's conversion brushed wide off the upright but he made no mistake with a penalty Ulster conceded under their posts right at the end of the half after a big scrum from the Blues.
Ulster started the second half in determined mood but a long-range drop goal from O'Connor fell short and then Pollock was held up short after a neat lineout move on the Blues' line.
Ulster stuck at it and were rewarded with an O'Connor penalty seven minutes before the hour to reduce the Blues' lead to two points.
After that there was little of any note, with O'Connor's chip just proving too long for Simon Danielli and props Brendon Botha and Rhys Gill both being yellow carded in the closing minutes.