I always wear my Bills jersey to away stadiums, and I always see people wearing the opposing team's jerseys at the Ralph.
If you are respectful of the people around you, you won't have any problems. If you act like an idiot, you will...anywhere. I always make friends with the folks around me, talk about football, baseball, whatever. There might be a jerk or two around, but most are fine, and I wouldn't worry.