It's a really interesting debate. There's absolutely no denying his defensive genius. The amount of gameplans the guy put together over the years to stop a specific player or high powered offense, when you look at that the guy is just incredible and deserves all the accolades he gets.
But yeah, the winning % as a HC without Brady is also undeniably subpar. I also object to people saying he's a genius for picking Brady. They lucked into Brady pure and simple. It would have been the same for us if Peterman somehow turned into a HOF quarterback.
If the Pats crash and burn this year and he retires it'll be interesting to hear the debates that will go on for years ("is he really the GOAT?").