Methods for generating differentially-private (DP) synthetic data have received recent attention as large government agencies such as the U.S. Census have decided to release DP synthetic data for public usage. In the synthetic data generation process, it is common to post-process the privatized results so that the final synthetic data agrees with what the data curator considers public information. Our contributions are three fold: 1) we show empirically that using post-processing to incorporate public information in contingency tables can lead to sub-optimal inference, 2) we propose an alternative Bayesian sampling framework that directly incorporates both noise due to DP and public information constraints, leading to improved inference, and 3) we demonstrate the proposed methodology on a study of the relationship between mortality rate and race in small areas given priviatized data from the CDC and U.S. Census.