the only time i don't wear mine is when we're out on the hunting property throwing corn, or trimming shooting lanes, going around 5mph or less.
That and at the boat ramp. The only place I think a seat belt is a liability. when i'm backing the boat down the ramp, no one is in the truck, both windows are rolled down and no seatbelt. I've heard of guys getting their doors pinned against the dock and being unable to get out of their cars when something went wrong.
Other than those two examples, there is no reason not to wear the thing.