Ciaran Donnelly could have put Albion ahead on nine minutes after Jamie Doyle's corner picked him out, but he sent his free header wide.
The Fifers produced some pretty passing and Joe Savage teed up Greig McDonald, but his half-volley was finger-tipped over the bar.
But it was a Donnelly mistake which allowed East Fife to go ahead on 23 minutes when he fluffed Savage's cross and McDonald stabbed the ball home.
Doyle was pulling the strings for Rovers and he sent a stinging effort just too high on the half-time whistle.
John Dodds spared Craig Lumsden's blushes when he scooped the defender's header out of the top corner on 49 minutes.
Stephen Donachy blew a golden chance on 54 minutes when the Rovers man ballooned over the bar after Scott Chaplain had bust a gut to keep the ball in play.
Fife defender Andy Campbell then had to race back to deny George Wallace on the goal-line after Dodds had missed a cross.
Rovers were denied again on 73 minutes, when Stephen Reid's downward header was blocked on the line by Gary Kelly.
The Fifers lived dangerously but survived, despite Dodds spilling Donachy's low drive with 11 minutes left.