Bajrami pulverised the previous Euros record for the quickest goal of 67 seconds by Dmitri Kirichenko of Russia against Greece in 2004.
Albania also qualified on Friday, drawing 1-1 in Moldova to ensure they would finish in the top two in Group E.