The first 2 guys you mentioned are no longer with the team, and Klemm was pretty much a bust.
I agree that NE has a great O line, but look at where they were drafted.
LT: Matt Light, 2nd round, 17th pick
LG: Logan Mankins, 1st round, last pick
C: Dan Koppen, 5th round, 29th pick
RG: Stephen Neal, Undrafted Free Agent
RT: Brandon Gorin, 7th round 1st pick by SD (the starter Kaczur was injured but he was an Undrafted Free Agent).
So, 2/3rds of their line was either taken on the 2nd day or not at all.