Reconstruction of the quasinormal spectrum from pole skipping