I totally agree. I used to date a woman who had spent some time in stand-up. She taught me about 'delivery' being the key to a great routine, and we'd sit there and watch stand up all the time. Comedy Central (used to be called something else back in the early 90s....) was on all the time and we'd analyze routines, talk about what was good/bad, what could have been funnier, etc. It was really cool. But anyway, the point is that too many comics need devices such as profanity and controversy to get their names out there. I don't think a routine needs to be devoid of these things, but like you said, the point is HUMOR. Someone like a Richard Pryor, for example. Sure, he used profanity in his routines, but he didn't rely on it in the slightest. The guy was just a great comedian. Seinfeld, though not my favorite entertainer in the world, is a genius. I saw a documentary on him where he was heading back to his stand up roots after his show ended, and other comics being interviewed were just in awe of the guy. They would say how some of them had been formulating routines for YEARS, and still not be anywhere as good as Seinfeld when he's only got a few months to write some new material. Bill Cosby....maybe the KING of delivery. I don't laugh my arse off at his stuff as much as some other folks, but his expressions and intonation are brilliant.
Comedy is an art form, and IMO humor is a form of intelligence - particularly witty humor. I think some of what passes for comedy these days is pretty embarrassing, but thankfully there are still plenty of people in the business who can bring it "old school".