It should be clear that riba (Arabic for: interest or usury) has been forbidden according to the text of the Qur’an. Almighty Allah says clearly, [Allah permits trading and forbids usury] (Al-Baqarah 2:275). He also threatens those dealing with riba with a severe punishment without clarifying the matter to them as he says, [O you who believe, fear Allah, and give up what remains of your demand for usury if you are indeed believers. If you do it not, take notice of war from Allah and His Messenger. But if you turn back, you will have your capital sums; deal not unjustly, and you will not be dealt with unjustly] (Al-Baqarah 2:278-279).

Dr. MonzerKahf, a prominent economist and counsellor, stated: It is of course forbidden for a Muslim to enter into a contract that makes him or her earn interest; any such contract does not produce any transfer of ownership of the interest.
A Muslim is required to be keen to avoid being a party to any interest-based contract, and it is certainly haram (Arabic for: forbidden according to Islamic teachings) to enter such contracts. This is why Islamic banks are established (i.e. as a substiute), and this is the whole story of the need for more Islamic finance.

But life, today and in the past, is complicated and there are circumstances in which interest is earned inadvertently, so other haram money (take an example of a prostitute who wants to repent and has money paid by men who had sex with her. Would any rational person suggest that she should return it to them?! It is a haram money, yet she must not return it to its owner; if she does, she would be giving the joy these men have got and their money, too; this is a foolish action, and a Muslim must not put himself or herself in such a foolish position).
What do we do with this interest money that comes to us inadvertently or from previous transactions before we decide to repent or from situations in which it is rationally wrong to allow a person or an institution to get away with both the benefit, they got from you and the price they are willing to give for it? All such circumstances do not make interest money a part of your property, and it is wrong to give it to the payer, what do we do with it?

Interest money is not like poison, you missed the analogy. In fact, interest money is like any other money, itself clean and useful. What is wrong with interest money is the way in which you acquired it. Realizing that it is wrong to give it back to the payer and that you cannot take it yourself, what would be the rational solution?
Nothing but to give it to the poor and needy, for whom even eating the flesh of a dead animal may — in extreme cases — be permissible. Of course, for the poor this money is not haram. We treat it as if it were money that has no owner, but it is wrong and haram to waste it because by wasting it you are in fact wasting a thing that has some utility and this is in fact haram.

There is of course reward in giving interest money to the poor and needy. It is the reward of keeping your own money clean from such evil income so that you enjoy your own property halal, all of it. This action is of course insha’Allahrewardable by Almighty Allah. But remember this is not a reward of giving sadaqah (Arabic for: charity) or zakah because these two must be given from your property, and interest money in your hand is not owned by you.

Finally, reducing the ability (within the law wherever one may be in the world that is in this regard the same as the Shari`ah) of a wrong doer is itself of course an act of virtue. Not returning the interest to its institutional payers whose business is to deal with interest and earn money from other people on its basis, is itself an act of goodness and improvement of life on earth. In this regard, it is also rewardable, is it not?