They soured on him because he ran the team into the ground. Passing on Allen didn't help, but several other morons passed on Allen as well, so Elway isn't uniquely dumb for that reason...
Look at his QBs after Manning passed away in the 2015 season. In successive seasons, his top passers were Siemian (twice), Keenum, Flacco, Lock and Teddy B. He also passed on Russell Wilson, taking Brock Osweiler instead. He drafted Paxton Lynch (who was just run out of the XFL) because he didn't know what else to do. He drafted Chad Kelly!!! This is a HOF QB who can't understand how to draft a QB! His 2012, 2013, 2014, 2015, 2017 (Garett Bolles!!!) drafts were whiffs but for a few long term starters. Not satisfied with his Paxton Lynch debacle, he doubled down with Drew Lock.
He's a bonehead. His best moves were Miller and rolling the dice on Manning's broken neck.