Allele frequency differences of causal variants have a major impact on low cross-ancestry portability of PRS