Observed color in images is easily affected by lighting conditions such as the sunlight location, weather, or time of day. Especially for image recognition processing on an outdoor navigated autonomous mobile robot, we should consider the effect of lighting conditions to achieve consistent robust color detection regardless of lighting condition change. In this paper, we employ new reference color patches to perform robust color detection to identify surrounding images by using the omnidirectional camera. As a demonstration of color detecting capability for the proposed approach, we apply specified human finding tasks defined by Tsukuba Challenge 2019 rules. Using the proposed method, we can demonstrate that the mobile robot was stably detected color, determining specified humans regardless of surrounding light change due to mobile robots.