Geometric approach to Lieb-Schultz-Mattis theorem without translation symmetry under inversion or rotation symmetry