Evidence for past interaction with an asymmetric circumstellar shell in the young SNR Cassiopeia A