The geometric challenge of testing gravity with wide binaries