Physics Guided Neural Networks for Time-Aware Fairness: An Application in Crop Yield Prediction