Singh had his first brush with politics when he campaigned for Hari Krishna Shastri, the son of late prime minister Lal Bahadur Shastri, who was contesting from the Allahabad Lok Sabha seat in 1971.

In the Janata Party wave after the lifting of Emergency in 1977, he won for the second time from Karchana. In the first Janata Party government led by Ram Naresh Yadav, he was appointed as minister of state for Small Industries.

A close associate of former PM V P Singh, Singh got the Irrigation portfolio in the first Janata Dal government in UP formed after the 1989 Assembly elections. Mulayam was the chief minister. When the BJP withdrew support to the V P Singh government following the arrest of L K Advani in October 1990 in Bihar during the Ayodhya Rath Yatra, Mulayam’s government was also reduced to a minority. Singh split the party walking away with 92 Janata Dal MLAs. Mulayam however saved his government with the outside support of Congress MLAs.

After the mid-term Assembly polls in 1991, Singh became the Leader of the Opposition. But in the next mid-term polls in 1993, he lost from Karchana. However, three years later, before the state saw another mid-term elections, he had joined hands with Mulayam again. Since then he has been in the same party.