New developments in the theory of X-ray absorption and core photoemission