Sheikh Sayyed Sabiq, states in his well-known book, Fiqh As-Sunnah, in this regard: Busrah bint Safwan narrated that the Prophet (peace and blessings be upon him) said, “Whoever touches his sexual organ cannot pray until he performs ablution.” (Reported by Al-Bukhari) In the narration of Ahmad and An-Nasa’i, Busrah heard the Prophet saying, “Ablution is to be made by a person who touches his sexual organ.” This is general and encompasses touching one’s own sexual organs or touching somebody else’s. Abu Hurairah reported the Prophet (peace and blessings be upon him) as saying, “Whoever touches his sexual organ without any covering (between them) must perform ablution.” (Reported by Ahmad, Ibn Hibban and Al-Hakim, who classified it as sahih (authentic), as did Ibn `Abdul-Barr).
Ibn As-Sakin said, “That Hadith is from the best of what has been related on this topic.” Ash-Shaf’i related: “Any man who touches his penis must perform ablution. Any woman who touches her vagina must perform ablution.” Commenting on its authenticity, Ibn Al-Qayyim quotes Al-Hazimi as saying, “That chain is sahih.”
However, the Hanifi scholars are of the opinion, based on the following Hadith, that touching the sexual organ does not nullify the ablution: “A man asked the Prophet (peace and blessings be upon him) if touching the penis necessitates performing ablution. The Prophet (peace and blessings be upon him) said: “No, it is just a part of you.” (Reported by “the five” (five scholars of Hadith). Ibn Hibban classified it as sahih, and Ibn al-Madini said, “It is better than the Hadith of Busrah.”)”
Having stated the above, it becomes clear that the issue is subject to controversy among Muslims scholars and thus you can choose from the aforementioned opinions what you deem proper in your case.