In the first place, it is to be stated that the main criterion for acts of worship to be accepted is that they are offered by or on behalf of a Muslim. This means that if a non-Muslim offers prayer or prayer is offe
red on his behalf, it will not be accepted since he lacks the main pillar of faith, which is testifying that there is no god but Allah and that Muhammad is the Messenger of Allah. Being interested to know about Islam is not enough to say that a certain person is a Muslim.
The prominent Muslim scholar Sheikh `Atiyyah Saqr, former head of Al-Azhar Fatwa Committee, states: In fact, the acts of worship are not accepted from a non-Muslim or on his behalf because he lacks the intention and Islam. The basic criterion for acts of worship being accepted is that the one offering the act be a Muslim and have the intention to perform that certain act.
Elaborating more on this issue, the late SheikhIbn `Uthaymeen, a prominent Saudi Muslim scholar, stated: Muslim scholars unanimously agree that a non-Muslim is addressed with Islam, that is, commanded to believe in Islam. But what about the rulings of Islamic Shari`ah and the acts of worship and the like: Is he addressed with them? This issue needs explanation. If you mean by address that he is commanded to do them, the answer is “no” because he has to accept Islam first and then do such acts. For this reason, the Prophet (peace and blessings be upon him) sent Mu`adh to Yemen and said to him, “Call them to testify that there is no God but Allah and that Muhammad is the Messenger of Allah. If they accept that, then tell them that Allah has ordained five prayers (salah) to be performed every day and night.”
Thus, non-Muslims are not commanded to perform Prayer (salah), and if they have performed Prayer without accepting Islam, their Prayer is null and unaccepted, for they have to embrace Islam first. But if you mean by address that they will be punished for not performing it, then the answer is “yes”, for they will be punished for not performing acts of worship even if they die while still non-Muslims. This is clear from the verse of Surat Al-Muddathir that tells us about the non-Muslims who will be asked the reason for their being among the residents of the Hellfire. They will say, (We were not of those who prayed) (Al-Muddathir 74: 43). This means that they will be held accountable for not performing such acts of worship because they have essentially rejected Islam and thus will be punished for the general rejection of Islam and the rejection of every act of worship.
Based on this understanding, we conclude that you cannot perform Hajj on behalf of your deceased non-Muslim father, simply because he doesn’t meet the main criterion for having the `ibadah (acts of worship) accepted.