He was born as Sa'aadat Hussain in undivided Punjab in Rahon, a town in Nawanshahr District which was then the Nawanshahr tehsil of Jalandhar District. As a boy Khayyam ran away to Delhi to learn music but was forced to return to complete his education. Khayyam then went to Lahore to learn music from the famous Baba Chishti.
He was never interested in studies and was always fascinated by the music of Indian cinema. He was inclined to music since a very young age. He often escaped to the city to watch movies. Khayyam soon ran away to his uncle's house in Delhi, in the hope of becoming an actor. Khayyam's uncle enrolled him in a school, but when he saw his passion for films, he allowed him to learn music, which was actually a step forward in realizing his ambitions. He got his training in music under Pandit Amar Nath.
He went to Lahore looking for roles in films. There he also learned music from famous Baba Chishti who was a famous Panjabi music director. One day Khayyam went to meet Baba Chishti, and after listening to a composition by Chishti sung to him its first part. Impressed, Chishti gave him an offer of joining him as an assistant. Khayyam assisted Chishti for six months and came to Ludhiana in 1943. He was only 17 then.
After a stint in the Army in the Second World War, Khayyam went to Bombay to fulfill his dream. He made his debut as Sharmaji of the Sharmaji-Varmaji composer duo with the movie Heer Ranjha in 1948. He went solo after his co-composer Rahman Varma went to the newly created Pakistan post partition. One of his earliest breaks was in the film Biwi in which the song "Akele mein wo ghabrate to Honge" sung by Mohammed Rafi was a huge hit. But he gained greater recognition from the film Phir Subha Hogi starring Raj Kapoor and Mala Sinha, in which songs written by Sahir Ludhyanvi and sung by Mukesh and Asha Bhonsle were set to tune by Khayyam, notable amongst them "Wo Subha Kabhi to Aayegi", "Aasman Pe hai Khuda aur Zameen pe Hum" and "Chin-o-Arab Humara".
The film Shola Aur Shabnam established Khayyam's reputation as a great composer. The songs written by Kaifi Azmi are some of the greats of the Hindi Film Industry. From Shola aur Shabnam "Jaane kya dhoondti rehti hein ye aanken mujhmen" sung by Rafi and "Jeet Hi lenge Baazi Hum tum" sung by Lata and Rafi, from the Chetan Anand directed Aakhri Khat "Baharon mera jeevan bhi sawaron" by Lata and "Aur kuch der theher, aur kuch der na ja" by Rafi. Also, noticeable are songs from the film Shagun which had Khayyam's wife Jagjit Kaur sing "Tum apna ranj-o-gham" and "Tum chali Jaogi". From the film Lala Rukh "Hai Kali Kali ke lab par" sung by Rafi and from Footpath "Shyam-e-Gham ki Kasam" sung by Talat Mehmood. From Mohhabat Isko Kehte hein "Theheriye hosh mein aa lun to chaley jaiyega" by Rafi and Suman Kalyanpur.
The 1970s saw Khayyam team up with Sahir Ludhyanvi once again to work in the Yash Chopra-directed "Kabhi Kabhi". The songs were super hits and showed Khayyam's versatility with huge hits like "Kabhi Kabhi mere dil mein Khayal ata hai" (Sung by Mukesh), "Tere chehre se nazar Nahin hatti" (Kishore & Lata) and "Mein Pal do pal ka Shayar Hun" (Mukesh). The film Shankar Hussain had Rafi sing "Kahin ek Masum Nazuk si Lardki" and Lata render "Aaap yun Faaslon Se", both classic Khayyam songs.
Khayyam's gave memorable music to the late-1970s and early-1980s was from Trishul, Thodi Si Bewafaai, Bazaar and Noorie. Again pairing with Sahir in Trishul he produced great songs like "Mohabbat bare kaam ki cheez hai" and "Janeman tum kamal karti ho" and the light "Gapuchi Gapuchi gam gam". "Hazar rahen" from Thodi si Bewafai and "Aaja re o mere dilbar Aaja" from Noorie and "Dikahyi Diye" from Bazaar are some noticeable songs. Khayyam also created music for the Kamal Amrohi directed "Razia Sultan" and his "Aye dil-e-nadan" sung by Lata capturing the mood beautifully. One of popular song is Cho lene do.
Khayyam was still to deliver his best and the opportunity came in Muzaffar Ali's Umrao Jaan in 1981. He made Asha Bhonsle sing songs which are indisputably her best. "In Aankhon ki masti ke", "Ye Kya Jagah hai doston" and "Dil Cheez kya hai" are evergreen.
Khayyam also composed non film songs noticeably "Paaon padun tore Shyam, Brij mein laut chalo" and "Ghazab kiya tere Vaade pe Aitbaar kiya". He also composed for Meena Kumari's album, 'I Write, I Recite' featuring nazms of Meena Kumari sung by her.
On his 89th birthday, Khayyam announced the formation of a charitable trust — Khayyam Jagjeet Kaur KPG Charitable Trust — and decided to donate his entire wealth to the trust to support budding artists and technicians in India.