My primary issue right now is after watching two seasons of the little things (individual plays, decisions, drives, adjustments), and the big things (wins and losses), I don't think Rex is great at coaching in the 2016 NFL reality. I'm not interested in having someone who isn't great at coaching lead our team. Call it a carousel, but as soon as you identify that a player or a coach isn't good at what they do, you need to try to make a change and find someone for that role who is good at what they do.
Tyrod lost my faith by his inability to progress on his weak areas of finding the open receiver, throwing accurately, and using the middle of the field. He doesn't get five years to show us if he can or can't, and the sample is big enough that we can see he can't. Rex lost my faith in many ways over the last two seasons. Both get a year 1 pass and can raise red flags in their first year, but can't be fired. Year 2 is the burden on them to show important progress, and year three show real results. I don't see progress on the red flags raised last year, so you move on.
The Patriots did the same with seven head coaches in 21 seasons, and the only two who lasted more than three years won Coach of The Year in their first two years. The reason they have stability is that they found a great coach (and a great QB), not that they were patient in favor of stability.
Sticking with mediocrity is not a recipe for success as frustrating as blowing things up is. If you think Tyrod or Rex can really be great, then make a better case, and I could be wrong, but each have failed at the job they are supposed to do as clearly visible in the little things and the big things.