Structure constants of heavy operators in ABJM and ABJ theory