On the second fundamental theorem of invariant theory for the orthosymplectic supergroup

Abstract Let OSp ( V ) be the orthosymplectic supergroup on an orthosymplectic vector superspace V of superdimension ( m | 2 n ) . Lehrer and Zhang showed that there is a surjective algebra homomorphism F r r : B r ( m − 2 n ) → End OSp ( V ) ( V ⊗ r ) , where B r ( m − 2 n ) is the Brauer algebra of degree r with parameter m − 2 n . The second fundamental theorem of invariant theory in this setting seeks to describe the kernel Ker F r r of F r r as a 2-sided ideal of B r ( m − 2 n ) . In this paper, we show that Ker F r r ≠ 0 if and only if r ≥ r c : = ( m + 1 ) ( n + 1 ) , and give a basis and a dimension formula for Ker F r r . We show that Ker F r r as a 2-sided ideal of B r ( m − 2 n ) is generated by Ker F r c r c for any r ≥ r c , and we provide an explicit set of generators for Ker F r c r c . These generators coincide in the classical case with those obtained in recent papers of Lehrer and Zhang on the second fundamental theorem of invariant theory for the orthogonal and symplectic groups. As an application we obtain the necessary and sufficient conditions for the endomorphism algebra End osp ( V ) ( V ⊗ r ) over the orthosymplectic Lie superalgebra osp ( V ) to be isomorphic to B r ( m − 2 n ) .