Spiritually speaking, how can I possibly leave my husband for a man who won't even leave his wife?

Spiritually speaking, I really don't think there's anything you can do about him. If you're religious you know that what he wants you to do is wrong. And it looks like you've figured that out so, don't pry any longer. If you have doubt, don't continue the relationship.