Modular forms and generalized anomaly cancellation formulas