The win takes the Ulstermen away from the foot of the table, but a mere three points ahead of now bottom side Connacht who play Leinster on Sunday and they now sit one point behind Andy Robinson's Edinburgh who managed a losing bonus and have now just won once in five outings.
A first-half try from David Pollock along with a conversion and two second half penalties from Clinton Schifcofske gave the Ravenhill side that all too rare winning feeling.
After a fairly insipid opening 27 minutes, during which the game degenerated into aerial ping-pong contest only broken up by Schifcofske's awful 22nd-minute penalty miss, Ulster suddenly exploded into life.
After the visitors lost the ball in their own half, scrum-half Cillian Willis was able to launch a scrappy counter which saw flanker Pollock break through and touchdown near the posts.
Ulster's relief was palpable as the entire team embraced Pollock after his efforts which also saw Schifcofske nail the extras.
It resulted in a sustained period of Ulster dominance - with the home scrum's Brendon Botha-fuelled power being particularly evident - which lasted until the break, but brought no change to the scoreboard with the home side spurning a number of opportunities.
Stephen Ferris came close to crossing the line on two occasions and Darren Cave threw a speculative pass which Bryn Cunningham could not gather as he came steaming up on the outside.
Then Schifcofske was short with a 45-metre penalty effort on 41 minutes before, with the last play of the half, the referee reversed a more kickable penalty chance after having words with Botha for use of the elbow.
The second half opened with a quickfire response from the visitors and only some last-ditch Ulster tackling prevented Hugo Southwell from grabbing a try.
However, Ulster were penalised twice in this opening period of pressure with outside-half David Blair kicking two penalties in the 44th and 46th minutes to reduce Ulster's lead to a single point.
The home side needed to respond and did so with a straightforward Schifcofske penalty after 52 minutes when Allan Jacobson was penalised.
He did manage another effort, his third from six attempts, to give Ulster some breathing space after 62 minutes, but replacement out half Phil Godman tucked his side into Ulster slipstream with a late three pointer just to ensure a jittery closing few minutes.