You are correct that I don't think there is much doubt about whether the activity is "cheating" on a spouse if you are doing it without their knowledge and consent (since, in most cases, you don't bring it up because you innately understand they wouldn't consent to that and would consider it cheating).

However, I am not making a value judgement about whether the activity is wrong or not as circumstances absolutely play into that. There are many cases that this "outside" activity is truly helpful to the relationship between a man and his spouse. By removing that need/pressure, especially when the spouse is totally against those activities, you allow both parties to be happier.

But, that doesn't mean it still isn't "cheating" by pretty much everyone's definition outside of yourself.