Many-species ecological fluctuations as a jump process from the brink of extinction

Many-species ecological communities can exhibit persistent fluctuations driven by species interactions. These dynamics feature many interesting properties, such as the emergence of long timescales and large fluctuations, that have remained poorly understood. We look at such dynamics, when species are supported by migration at a small rate. We find that the dynamics are characterized by a single long correlation timescale. We prove that the time and abundances can be rescaled to yield a well-defined limiting process when the migration rate is small but positive. The existence of this rescaled dynamics predicts scaling forms for both abundance distributions and timescales, which are verified exactly in scaling collapse of simulation results. In the rescaled process, a clear separation naturally emerges at any given time between rare and abundant species, allowing for a clear-cut definition of the number of coexisting species. Species move back and forth between the rare and abundant subsets. The dynamics of a species entering the abundant subset starts with rapid growth from rare, appearing as an instantaneous jump in rescaled time, followed by meandering abundances with an overall negative bias. The emergence of the long timescale is explained by another rescaling theory for earlier times. Finally, we prove that the number of abundant species is tuned to remain below and without saturating a well-known stability bound, maintaining the system away from marginality. This is traced back to the perturbing effect of the jump processes of incoming species on the abundant ones.