Beyond heat baths II: framework for generalized thermodynamic resource theories