Derek Colquhoun pounced six minutes from time to give East Fife their first win in a month.
The Fifers drafted in new signings Steven Nicholas and Paul Gaughan, and both new boys added quality to the injury hit squad.
Rovers weren't here to sit back and admire though, and Ricky Patrick tested the keeper with a well-struck half volley after seven minutes.
Nicholas, in his second spell at Bayview Stadium, picked out Kris Brash and the full back's riving shot gave Mark Peat an anxious moment.
Lively James Herkes almost beat the Albion keeper after 19minutes when he cut inside and sent in a low drive which had Peat scrambling across his goal. Then Gaughan's header back to Scott Morrison nearly embarrassed the big keeper, but he redeemed himself with a close-range block from Scott Crabbe's effort.
A loose clearance from Albion's Jered Stirling gave Leslie Byle a sight of goal after 48 minutes and he hammered a a shot destined for the top corner until Peat intervened.
East Fife old boy Paul McManus almost buried his former team on the hour mark, but Michael Renwick hacked his header off the line.
Nicholas then raced to the other end and Peat somehow blocked his close range drive as both keepers continued to excel.
East Fife could have won it after 72 minutes when Gary Kelly raced through, but his chip beat the keeper but dropped wide.
Luckily for him and with just six minutes left on the clock, Colquhoun collected a loose ball and checked inside before applying a cool finish.