Brady haunted me as a football fan for 20+ years. It felt like it would never end. Definitely the player I rooted against the most and it isn't close.
That said, the man was a psychotic, Type A ultra competitor whose entire life was football, preparing for football, eating for football, and getting ready for football, so any little thing that wasn't right he couldn't handle. On some level, it probably drive him nuts that not everyone was like him.
Any time, energy or thought spent on something non football related was a waste.
For the vast majority of guys, that would probably be exhausting. But he's Brady, so you can't say much.