The duo were children of a businessman who migrated from Kundrodi Village in Kutch to 'Bombay' (now Mumbai) to start a grocery and provision store. The brothers began to learn music from a music teacher, who taught them in lieu of paying his bills to their father. One of their great-grandparents was a folk musician of some eminence.
Kalyanji started his career as a musician, with a new electronic instrument called the clavioline. which was used for the famous "Nagin Been," used in the film Nagin (1954) which had the music of Hemant Kumar. Kalyanji then, with his brother Anandji, started an orchestral group called Kalyanji Virji and Party which organised musical shows in Mumbai and outside. This was the first attempt made for holding live musical shows in India.
Kalyanji Anandji's arrival in the Bombay film industry as music composers was a turning point. When big music directors like S.D. Burman, Hemant Kumar, Madan Mohan, Naushad, Shankar Jaikishan and Ravi were ruling the Hindi film music world – and it was a golden period of film music – it was very tough to make a place amongst them. They still managed to attain success amongst the competition.
The Bharat Bhushan–Nirupa Roy hit Samrat Chandragupta (1959) was his first film as Kalyanji Virji Shah. Songs like "Chahe Paas Ho" (Lata–Rafi) that are remembered to this day were what made the movie a commercial success. This was followed by his composing music scores for more films like Post Box 999 before Anandji who was assisting him, joined him officially to form the Kalyanji Anandji duo in Satta Bazar and Madari (1959). Chalia (1961) was their earliest major hit. In 1965, two decisive scores, Himalay Ki God Mein and Jab Jab Phool Khile, established them as composers to reckon with.
Both Kalyanji and Anandji worked as music composers for over 250 films, 17 of which were golden jubilees and 39 silver. They organised many charitable concerts for NGOs and several charitable institutions in India and abroad with some of the biggest names in Bollywood, like Dilip Kumar, Amitabh Bachchan, Anil Kapoor, Vinod Khanna, Rekha and Sridevi.
Manhar Udhas, Kumar Sanu, Anuradha Paudwal, Alka Yagnik, Sadhana Sargam, Sapna Mukherjee, Udit Narayan, Sunidhi Chauhan, now very popular names, were nurtured as singers and got their first breaks from Kalyanji Anandji. Laxmikant Pyarelal worked as music assistants to Kalyanji Anandji before composing Music independently for Hindi films. They helped introduce or gave career defining breaks to lyricists like Qamar Jalalabadi, Anand Bakshi, Gulshan Bawra, Anjaan, Verma Malik and M G Hashmat.
In the late 1990s and early 2000s, their work was introduced to a young Western audience by three albums. Bombay the Hard Way: Guns, Cars and Sitars was a mix album put together by US DJ Dan The Automator; Bollywood Funk was an Out caste compilation album put together by Sutrasonic DJs Harv and Sunni; while The Beginners Guide To Bollywood was compiled by John Lewis from Time Out magazine. All three of these records concentrated on Kalyanji Anandji tracks from 1970s films that featured funk break beats, wah-wah guitars and Motown-style orchestrations. In 2005, The Black Eyed Peas's "Don't Phunk with My Heart" used music pieces from two of their songs: "Ae Naujawan" from the 1972 film Apradh and "Yeh Mera Dil" from the 1978 film Don, which won the American hip-hop group a Grammy Award.
On 24 August 2000, Kalyanji breathed his last.
The composers composed some popular songs rendered by Kishore Kumar such as "Jeevan se bhari teri aankhe", "Zindagi ka safar hai ye kaisa Safar", "Pal Pal dilke paas", "Neele Neele Ambar Par". Their composition called "Pal bharke liye" was used in an episode of The Simpsons. Remixed version of the song "Apni Toh Jaise Taise," originally from the 1981 film Laawaris was used in 2010 Bollywood film Housefull. The Calcutta High Court restrained producer Sajid Nadiadwala from the cinematic use of the song.
They used playback of Kishore Kumar for Dilip Kumar in "Bairaag " and for Shammi Kapur in "Vidhata "