On the idea of Learning Trajectories: Promises and Pitfalls