Latency-Aware Short-Term Video Action Anticipation and its Application in Trajectory Prediction