This paper focus on efficient learning of new objects by robots, at both instance level and class level. Despite the great achievements in deep learning, new objects are not well handled due to the big data dependency and the closed set assumption, which are not satisfied in many applications. For example, in service robots, the desired objects vary among customers and environments, and it is hard to collect big data for all objects. In addition, new objects occur from time to time that are not contained in the training data. This paper aims to learn new objects on-the-fly after deployment of the robot, without the dependency on pre-defined big data. A practical system is proposed to learn a new instance in 1.5 minutes and a new class in 15 to 25 minutes, by integrating state-of-the-art works including online learning, incremental learning, salient detection, object detection, tracking, re-identification, etc. A novel incremental object detection framework is further proposed with better performance than the state-of-the-arts. Extensive evaluations are conducted to examine the contribution of each module. The proposed system is also deployed on a real robot for end to end performance test.