When it comes to the critics, being in the middle of the pack in those rankings won't really matter. The only three people he'll get compared to are Luck, Wilson and RGIII. All made the playoffs in their first year. Two of them repeated (while RGIII is dealing with drama/slump). EJ has gotten injured twice, and lead the Bills to another losing record while playing on a team with a "dominant defense" (according to certain experts). In the end, it doesn't look good for him.
Personally, I see bits and pieces of a good QB. But then I also see a lot of problems. I'm all for giving him time, but I also don't want to sit back and "settle" on him. So if the Bills have the opportunity, either in the draft, or FA, to get a QB that is either better, or equal to him, they should take that opportunity and let there be a QB competition next year. EJ hasn't earned the "unquestioned starter" title, yet.