An affair

I have just ended an affair with a married friend. We are both Christians. I am also married. Neither of our spouses know about it. We didn't have sex, but have flirted and had inappropriate contact several times over the course of the last year. What is worse is that his wife is also my friend. Which I don't deserve to call her a friend, since all of this has happened. Both he and I feel very guilty and have stopped our relationship. I have asked for God's forgiveness and given up to Him my "desire" for this other man. This other man has been asked to be a deacon in his church, and he told me recently that he will have to admit to his wife our infidelity. I am very afraid of this for obvious reasons. The biggest one being I don't see the point in hurting her or their two small kids. Also if he tells her, I would have to tell my husband. This scares me as he has a temper, and I don't know what he would do. Please pray for us, and if you have any advice for me, I would greatly appreciate it.