He became famous in 1531 when a hymnal titled “Ein Neues Gesangbuchlein” appeared. There were 157 German hymns in the book, of which 137 were either written or revised by Weiß. Most melodies derived from the Czech tradition of the Bohemian Brotherhood. It was the first German hymnal drawing on the Bohemian tradition and was reprinted several times, catalyzing additional works of that type.

In 1544 Johann Horn reprinted Weiß’ hymnal. The new version contained 181 hymns, 149 from the first edition and 32 new songs of uncertain authorship. It was printed in Nuremburg.
Weiß' hymns are still sung today. In the “Evangelischen Gesangbuch”, the hymnal of Germany’s official Protestant church, there are nine of his songs.