It does matter if rules aren't being followed. If an offensive lineman is clearly holding they should call it, or they should change the rule.
As always, though, these rules are being inconsistently enforced from game to game and even from play to play in the same game. That's the real problem.