What ownership has hired him since then as their HC? That's what we are talking about, not QB coach. He wasn't Stafford's QB anyway. And certainly Manning continued on his HOF career for years after Caldwell was no longer his "QB coach". Before that he had a 4400+ season, after that, he had 4500 and 4700.
Let me check their stats with Jimbo as HC:
Manning was 85-27 under Dungy. 48-26 with Caldwell. "After he left", Manning was no longer with the Colts, but he did set a single season record in Denver, also won a SB. "After he left", Colts with Pagano had 3 11-5 seasons in a row with a new QB.
Stafford had 3 of his biggest seasons passing all in a row---which ended the year Caldwell arrived. "After he left", Stafford was back to his baseline in 2020 (he was injured 2019).