Rovers got off to a positive start and should have took the lead after nine minutes as Scott Chaplain put Joe Savage through on goal, but the striker's shot was well saved by Kieron Renton.
A minute later, though, the home side did take the lead when Ian Chisholm latched on to a Pat Walker in-swinging corner to powerfully head home.
Albion then doubled their lead five minutes later when Savage controlled a Chaplain cross on his chest before drilling a shot into the top corner.
Elgin were looking nervous at the back, and Albion should have killed the game off after 25 minutes when Chisholm robbed Hugh Dickson to go through on goal but blazed his shot just wide.
Then a minute later Steven Mackay became the culprit as he was caught in possession on the edge of the box, but his keeper saved his blushes as he made an excellent save to deny Savage.
Elgin came back into the match just before half-time and should have pulled one back when Adam Moffat put Martin Johnston through on goal, but the striker's shot drifted just past the post.
The visitors did pull one back just after the break when Johnston made amends for his earlier miss by firing a shot past the advancing Jamie Ewiings.
Albion did grab the vital third goal after 63 minutes when man-of-the-match Chaplain turned his marker on the edge of the box before curling a shot into the bottom corner.