Fine-grained complexity of coloring unit disks and balls