Barth syndrome is an X-linked disorder caused by loss-of-function mutations in Tafazzin (TAZ), an acyltransferase that catalyzes remodeling of cardiolipin, a signature phospholipid of the inner mitochondrial membrane. Patients develop cardiac and skeletal muscle weakness, growth delay, and neutropenia, although phenotypic expression varies considerably between patients. Taz knockout mice recapitulate many of the hallmark features of the disease. We used mouse genetics to test the hypothesis that genetic modifiers alter the phenotypic manifestations of Taz inactivation. We crossed TazKO/X females in the C57BL6/J inbred strain to males from 8 inbred strains and evaluated the phenotypes of first generation (F1) TazKO/Y progeny, compared to TazWT/Y littermates. We observed that genetic background strongly impacted phenotypic expression. C57BL6/J and CAST/EiJ[F1] TazKO/Y mice developed severe cardiomyopathy, whereas A/J[F1] TazKO/Y mice had normal heart function. C57BL6/J and WSB/EiJ[F1] TazKO/Y mice had severely reduced treadmill endurance, whereas endurance was normal in A/J[F1] and CAST/EiJ[F1] TazKO/Y mice. In all genetic backgrounds, cardiolipin showed similar abnormalities in knockout mice, and transcriptomic and metabolomic investigations identified signatures of mitochondrial uncoupling and activation of the integrated stress response. TazKO/Y cardiac mitochondria were small, clustered, and had reduced cristae density in knockouts in severely affected genetic backgrounds but were relatively preserved in the permissive A/J[F1] strain. Gene expression and mitophagy measurements were consistent with reduced mitophagy in knockout mice in genetic backgrounds intolerant of Taz mutation. Our data demonstrate that genetic modifiers powerfully modulate phenotypic expression of Taz loss-of-function and act downstream of cardiolipin, possibly by altering mitochondrial quality control.