Dr. MuzzamilSiddiqi, former president of the Islamic Society of North America (ISNA), states the following: The rule of the Shari`ah is that in the case of doubt, one should follow that which is certain or more probable. In Sahih Muslim, the Prophet, peace and blessings be upon him, is reported to have said, “If a person among you has a doubt about his prayer and does not know whether he has prayed three or four, then he should discard the doubt and follow that which is certain. Then he should make two prostrations before the final Salam. If the person has prayed five Rak`ahs, (Prayer units) then the two prostrations would make even his prayer. But if he has prayed four, then these prostrations would disgrace the Shaytan.” (Reported by Muslim)
In another Hadith in Sahih Al-Bukhari, it is mentioned that if one has doubt concerning his prayer, then he should do that which is most likely or probable and then at the end he should make two Sajdahs for Sahw (prostration for forgetfulness). (See Hadith no. 386)
So, if a person doubts whether he has prayed three or four prayer units, then three is a certain number. In this case he should add one more prayer unit to complete four and then at the end make two Sajdahs for Sahw. But if his mind inclines more to four units, then he should not add another Rak`ah, he should only make two Sajdahs for Sahw at the end of the prayer.