Reflection subgroups of Coxeter systems

Let ( W, R) be a Coxetcr system, and T= IJ,,,. W wRw ’ denote the corresponding set of reflections. For u’ E W, define N(W) = {TV Tlf(tw)<l(w)} h w ere I denotes the length function of ( W, R). The principal result (3.3) of this paper is that if W’ is a reflection subgroup of W (i.e.. W’= ( W’n T)) then {[E TI N(z)n W’= { 1)) is a set of Coxeter generators for W’. In particular, W’ is a Coxeter group. In [6], V. Deodhar gives a geometric proof of this last fact using properties of the root system of W. It is easily seen that the Coxeter gcncrators for W’ defined by Deodhar coincide with those described above. The idea of the proof here is that N may be regarded as a cocycle with N(r)= {r} (reR), and Coxeter systems are characterized (among groups generated by involutions R) by the existence of such a cocycle. One then checks that w H N(W) n w’ (W E W’) gives a suitable cocycle for W’. The same argument applies to a wider class of “reflection systems” (G, X) characterized by the cxistcncc of a certain cocycle and so we give the proof for these. The Coxeter systems are precisely the reflection systems (G, X) in which X consists of involutions. The arrangement of this paper is as follows. In Section 2, we describe the main facts about reflection systems, closely paralleling [ 1, Chap. IV, Sect. 11. In Section 3: we prove that reflection subgroups of a reflection system are themselves reflection systems in a canonical way (3.3) and obtain some additional facts about their canonical generators (3.4) (3.11). In Section 4, we specialize to the case of a Coxeter system, and give a geometric criterion (4.4) for a set of reflections to be the set of canonical generators of a reflection subgroup, in terms of the inner products of the