Modeling complex measurement error in microbiome experiments