With the fast development of Internet of Vehicles (IoV), various types of computation-intensive vehicular applications pose significant challenges to resource-constrained vehicles. The emerging Vehicular Edge Computing (VEC) and Edge Intelligence (EI) can alleviate this situation by offloading the computation tasks of vehicles to the roadside edge servers. However, with many vehicles contending for the communication and computation resources at the same time, how to quickly and efficiently make an optimal computation offloading decision for individual vehicles represents a fundamental research issue. In this paper, we propose a self-learning based distributed computation offloading scheme for IoV. Note that without any centralized controller, a fully distributed algorithm is necessary. The proposed scheme is devised based on a game-theoretic model. Specifically, through establishing an offloading framework with communication and computation for IoV, the computation offloading problem is first formulated as a distributed offloading decision-making game, in which each vehicle as a player makes its best response decision to minimize its joint cost (including latency and offloading cost). The existence of Nash Equilibrium can be proved. We then propose a self-learning based distributed computation offloading (DISCO) algorithm to reach the Nash Equilibrium, where a mutually satisfactory solution among vehicles is obtained and no vehicle is willing to change its decision. Using extensive simulations, we verify that DISCO can outperform the counterparts and achieve at least an order-of-magnitude improvement on time overhead and 88% performance gain on message overhead, only at up to 12% performance loss on joint cost over the centralized scheme.