The Coatbridge side must enjoy their visits to the Angus seaside as their only other victory this season came 12 miles up the road at Montrose.
The win at Arbroath was particularly remarkable as it was achieved despite having to play the final 24 minutes with 10 men following the dismissal of Danny Black after receiving a second yellow card.
Following an uninspiring opening the game burst into life in the 28th minute when Arbroath's Jay Stein delicately curled the ball high over Rovers keeper Jamie Ewings for the opener.
This spurred Albion into action and they equalised seven minutes later. Arbroath's Steven Cook blocked a George Wallace shot with his hand and Scott Friel converted the resultant penalty. It was no less than Friel deserved as he had come close with a couple of earlier efforts.
Arbroath retaliated and had a couple of penalty claims of their own turned down before the interval.
The home side dominated the early stages of the second half, and Albion's hopes seemed doomed when Black was sent off aftert 66 minutes. But two minutes later Wallace latched on to a long through ball and shot low past Mark Peat for what turned out to be the winner.
Arbroath pressed for an equaliser but Rovers held out. " We had three or four decent chances but didn't take any of them," said home manager John McGlashan.