Bridging Perception, Memory, and Inference through Semantic Relations