i sympathize with you man...it's hard when you're on a bike because you're not as agile. it's much much easier when you're running, because you can turn around and rooster your fist and scream at the top of your lungs to the dog that if he takes one more step toward you you're going to knock his !@#$ing head off and then reach down his neck, grab him by the balls and pull him completely inside out right there on the street. the dogs that i have done this to generally "sense" the danger and back off. honest to god, i once held off 6 junk yard dogs running up 32nd street in philadelphia with this technique (coupled with a kick to the chops of the largest and most aggressive pooch, who apparently did not speak english). and because i lived, i have complete confidence in this technique and believe it to be effective on 99.9% of all unleashed dogs. i will admit however, that there is one rottweiler on my usual route that, if it came down to it, would stand a chance of surviving should it come down to a me-vs.-him battle to the death. on the other hand, my tried and true intimidation technique would be my only chance, so i'd try it anyway.
in your case, it's hard to turn around abruptly and scream and kick and threaten to behead the dog when you are zipping down an 18inch wide shoulder in heavy traffic at 20mph. so i would also recommend the pepper spray. good luck.