The rocky road to panspermia