Long-term optical/infrared variability in the quiescent X-ray transient V404 Cyg