Algebraic structure of lepton and quark flavor invariants and CP violation