Stability theorems for cancellative hypergraphs

A cancellative hypergraph has no three edges A, B, C with AΔB ⊂ C. We give a new short proof of an old result of Bollobas, which states that the maximum size of a cancellative triple system is achieved by the balanced complete tripartite 3-graph. One of the two forbidden subhypergraphs in a cancellative 3-graph is F 5 = {abc, abd, cde}. For n ≥ 33 we show that the maximum number of triples on n vertices containing no copy of F 5 is also achieved by the balanced complete tripartite 3-graph. This strengthens a theorem of Frankl and Furedi, who proved it for n ≥ 3000. For both extremal results, we show that a 3-graph with almost as many edges as the extremal example is approximately tripartite. These stability theorems are analogous to the Simonovits stability theorem for graphs.