Can We Have Both Fish and Bear's Paw?: Improving Performance, Reliability, and both of them for Relation Extraction under Label Shift