Coming from a man , go ahead and end it , since both of you had done things now , you guys will never look at things the same as before , you will always have doubt in the back of your mind , so better to end it now , clear your mind and do right for your kids and you