From observation to simulation