Worldline approach to semi-classical conformal blocks