Analysis of pair correlation functions for macroscopic particles in dusty plasmas: Numerical simulation and experiment