Keldorfer graduated from the Teacher Training College and the Mozarteum in his hometown. In 1892, he went to Vienna. He was the director of the ''Wiener Männergesang-Verein'' from 1902 to 1921 and also its choir master commencing in 1910. At the Vienna ''Schubertbund'' he was choir master from 1922 to 1938 and from 1945 to 1952. Fom 1938 to 1945 he was banned from working.
In 1928, he conducted choir associations of several tens of thousands of singers at the same time at the 10th German Singers' Association Festival in the ''Vienna Prater''.
Keldorfer was a founding member of the Society of Authors, Composers and Music Publishers and from 1921 also its director.
In 1946, he was commissioned to compose the piano and choral settings for the Federal Anthem of Austria (composed presumably by Paul Wranitzky), although Keldorfer still erroneously assumed that the melody was composed by Wolfgang Amadeus Mozart.