Galaxy And Mass Assembly (GAMA): The unimodal nature of the dwarf galaxy population