Pro team's and stadiums generate a ton of tax revenue, so the owners aren't the only ones to benefit. Everything from parking spots, tickets, merchandise and concessions are taxed, with no team that tax stream dries up. Every player and coach pays taxes where the game takes place, more tax revenue gone. If the governments didn't benefit from helping with stadium costs, it wouldn't be an issue, but they do so it's a discussion.
Having a pro team is ultimately good for the community and if you look at the overall state budget spreading out the cost of the stadium over 5 years is a piss in a pool