Giving the QB time to throw only rushing 4 and not getting pressure is doing exactly what you're claiming blitzing does.
If the secondary is your weakness, you do not let the QB sit back there and wait for receivers to come open. You have to force the bad throws.
Most QBs in the NFL will find the receivers if given time to throw and if you're not getting pressure by just rushing the front 4, that's exactly what you'll be doing.
I'm certainly not advocating blitzing every down, but this defense has become way too predictable and has grown far too content with rushing 4, not getting pressure, and having our secondary exposed by meh QBs like Brissett.