The flickering nuclear activity of Fornax A