Phase-referenced visibility averaging in optical long-baseline interferometry