As regards your question, Dr. Zainab Al-Alwani, instructor of Fiqh and Islamic studies, graduate school of Islamic and Social Sciences, stated,
A Muslim ought to repent sincerely to Allah and ask for His forgiveness.
He can choose to proceed with marriage, at least with marriage contract, in order to avoid any further violations.