I could not agree more. I now go to only 2 or 3 games a year (college and pro combined) and just make sure to pay what it takes to have awesome seats/experience when I go.
I mean when I'm at home I've got the food ready, the cold beer, the giant tv w/ NFL ticket, a second TV for the redzone to track fantasy, my computer is up for NFL chat/boards and fantasy, and there are great seats everywhere.
Going to games doesn't compete with that unless you have amazing seats.