Critical collapse of collisionless matter in spherical symmetry