The algebra of harmonic functions for a matrix-valued transfer operator