Switchbacks in the Near-Sun Magnetic Field: Long Memory and Impact on the Turbulence Cascade