The HARPS search for southern extra-solar planets - XXXI. The M-dwarf sample