I think there is something to the adage that "you can't go home again." I moved away to California for college, and while I still love Buffalo and consider myself a Buffalonian for life, it just isn't quite the same when I go back to visit. And you're right, I also get the "hey, it's TheBrownBear from Cali". It's kind of weird how you're labeled an outsider after you leave. It's definitely a very insular community compared to some of the bigger cities I've lived in.
But I'll always vouch for the city and it's people. It is absolutely a great place to grow up and raise kids.