High precision in CKM unitarity tests in $b$ and $c$ decays