Cohen was born on the Romanian border near Iași He emigrated to Palestine with his parents in 1878. Beginning in 1886 he lived in Rischon LeZion where he studied viticulture.

He played the violin. In 1888 he composed the most important verses of the poem Tikwatenu (Hebrew תיקותנו "Our Hope") by Naphtali Herz Imber, which the latter had written down in 1877 in Iași. The ten-stropic text of this poem was published in 1886 in ''Imber's Announcement'' of Dawn in Jerusalem. The music for the "Hatikwa" was published with text by C. G. Röder in 1895 in Leipzig, Germany, under the title ''Vier Lieder als Arrangement'' von S. T. Friedland as "Yearning".

Cohen was probably inspired for his melody by the Moldovan folk song Carul cu boi (bullock cart), known in Romania as "Cantec de mai" (May Song) or "Cucuruz cu frunza-n sus" (corn with high leaves). Based on the German musicologist Peter Immanuel Gradenwitz (1910-2001), Cohen used a motif from the liturgical chants of the cantor Nissan Belzer Spivak (1824-1906) from Chișinău.