Les Bleus, who won just two Test matches throughout 2013, romped into a 16-3 lead thanks to two tries from winger Yoann Huget.
Slowly but surely, England chipped away however and thanks to scores from Mike Brown and Luther Burrell, as well as a Danny Care drop goal and an Alex Goode penalty, the visitors found themselves 24-19 to the good going into the final minutes.
Replacement centre Gael Fickou, 19, raced clear down the left with three minutes remaining however and Maxime Machenaud's conversion handed France, who finished bottom of the RBS 6 Nations pile last year, a 26-24 victory and a winning start to 2014.
"We have to remember this victory, the smiles in the changing room, this is very important in 2014," said Saint-Andre.
"Once we could get our hands on the ball, we found some new solutions and that try gives hope to this young team.
"The order of day was to not let anything slip for the full 80 minutes. We had a first half of real quality, scoring two tries and posing problems for the English.
"But in the second half we found ourselves in some difficulties. England attacked the 9-10 channel and made a lot of offloads.
"There has been an evolution from last year, we had the team together for 15 days before the tournament, for that we have to thank the union and the league.
"We began to lose a bit of patience and committed a few errors, we were missing touch with penalties, and did not come up fast enough allowing England to play.
"They controlled the tempo of the match well and in the second half we began to suffer physically.
"But in the last quarter of an hour the English lost a bit of that aggression."