Initially I thought he didn't know the mathematical odds of getting the 2nd pick. That's easy enough to explain.
Now that I look back, it's obvious he thinks it won't work out simply because we're Buffalo. Trying to explain to someone that there isn't a supernatural force, curse, or hex keeping the team down is a bit too much heavy lifting for me.