Also encouraging for Ottawa was the play of disgruntled forward Alex Kovalev.
The Russian veteran expressed his displeasure earlier in the day to reporters about being demoted to the fourth line. He played just over 18 minutes and was on the ice with the top line during the final minutes of the game, finishing with two shots on goal and a plus-1 rating.
"I thought his first couple of shifts were real good," said Senators coach Cory Clouston. "It was something we were looking at doing. I thought he played well."
Pascal Leclaire made 29 saves to help Ottawa (13-16-2) snap a four-game home-ice losing streak
The loss left the New Jersey Devils (8-18-2) feeling a new sense of desperation as they see any chance of reaching the post-season slip further away. The Devils are mired in a four-game losing streak and have now lost seven straight on the road.
Not even the return of franchise goaltender Martin Brodeur could turn their fortunes around. Brodeur faced 29 shots in his first game back since November 18. Patrik Elias and Danius Zubrus scored for the Devils.
"If we played well right from the get-go it would have been a lot different," said Brodeur. "You create your own breaks and right now we're not."
The Senators got off to a good start taking a 1-0 lead after the first period as Karlsson's shot from the point got through traffic and beat Brodeur glove side.
Elias tied the game with a power-play goal early in the second, as his shot deflected off Leclaire's glove and trickled across the goal line.
The Senators regained the lead midway through the period as Michalek, with his first goal in 10 games, beat Brodeur low stick side.
Just over two minutes later Ruutu, from behind the net, made it 3-1 as he banked a shot off Brodeur.
However, the Devils cut the lead back to one with their second power-play goal of the game when Zubrus took a great pass from Travis Zajaz and beat Leclaire with a one-timer from just above the hash marks.
After blowing a one-goal lead against the New York Rangers a night earlier, the Senators looked desperate to eliminate any chance of a repeat performance.
"It's a big win for us," said Leclaire. "We'll start from there. We did a lot of good things. Getting some results tonight was good for everybody."
Clouston admitted the win would definitely be good for the team's confidence considering how fragile it's been of late.
"We've been in the games," said Clouston. "Fortunately tonight we finished it off and got the two points and we've got to build on that."
Leclaire made a huge save on Jason Arnott midway through the final period to preserve the lead and was lucky when Henrik Tallinder rang a shot off the post late in the game.
The Senators caught a huge break with 3:36 remaining when David Clarkson's goal was called off after it was ruled he kicked it in.
Clarkson admitted he hadn't seen the replay, but didn't believe he had kicked the puck in. But considering the Devils luck he wasn't that surprised to have the goal called back.
"That's been our season so far," said Clarkson. "We're not winning and it's frustrating. We'll hear how we're playing well, but obviously not well enough because we're not winning. It's tough to swallow."