The rapid development of 6G can help to bring autonomous driving closed to the reality. Drivers and passengers will have more time for work and leisure spending in the vehicles, further generating a lot of data requirements. However, edge resources from small base stations are insufficient to match the wide variety of services of the future vehicular networks. Besides, due to the high-speed nature of the vehicles, users have to switch the connections among different base stations, whereas such way will cause external latency during the data request. Therefore, it is vital to enable the local cache of vehicle users to realize the reliable autonomous driving. In this paper, we consider caching the contents in the local cache, small base station, and edge server. In practice, the request preference of some single users may be different from a whole region. To maximize the efficiency of content cache, we design a strategy that uses reinforcement learning algorithm to optimize cache schemes on different devices. The experimental results demonstrate that our strategy can enhance the cache hit ratio by 10%-20% compared with the well-known counterparts.