A nonexistence result on difference sets, partial difference sets and divisible difference sets