In this publication, the contributors to in-field overlay metrology uncertainty have been parsed and quantified on a back end process and compared with results from a previous front end study1. Particular focus is placed on the unmodeled systematics, i.e. the components which contribute to residuals in a linear model after removal of random errors. These are the contributors which are often the most challenging to quantify and are suspected to be significant in the model residuals. The results show that in both back and front end processes, the unmodeled systematics are the dominant residual contributor, accounting for 60 to 70% of the variance, even when subsequent exposures are on the same scanner. A higher order overlay model analysis demonstrates that this element of the residuals can be further dissected into correctible and non-correctible high order systematics. A preliminary sampling analysis demonstrates a major opportunity to improve the accuracy of lot dispositioning parameters by transitioning to denser sample plans compared with standard practices. Field stability is defined as a metric to quantify the field to field variability of the intrafield correctibles.