The fort is quite ancient. Remnants of Microlithic man have been discovered here. The various Puranas (ancient scriptures) like Matsyapurana, Agnipurana and Skandapurana include many references for Harishchandragad. Its origin is said to have been in the 6th century, during the rule of the Kalchuri dynasty. The citadel was built during this era. The various caves probably have been carved out in the 11th century. In these caves, idols of Lord Vishnu were carved. Though the cliffs are named Taramati and Rohidas, they are not related to Ayodhya.

Great sage Changdev (one who created the epic “Tatvasaar”), used to meditate here in the 14th century. The caves are from the same period. The various constructions on the fort and those existing in the surrounding region point to the existence of diverse cultures here. The carvings on the temples of Nageshwar (in Khireshwar village), in the Harishchandreshwar temple and in the cave of Kedareshwar indicate that the fort belongs to the medieval period since it is related to Shaiva, Shakta or Naath. Later the fort was under the control of Moguls. The Marathas captured it in 1747.