Road scene analysis is a fundamental task for both autonomous vehicles and ADAS systems. Nowadays, one can find autonomous vehicles that are able to properly detect objects in the scene in good weather conditions; however, some improvements still need to be done when the visibility is altered. People claim that using some non-conventional sensors such as, infra-red or Lidar, combined with classical vision, enhances road scene analysis in optimal weather conditions. In this work, we present the improvements achieved using polarimetric imaging in the complex situation of some adverse weather conditions. This rich modality is known for its ability to describe an object not only by its intensity information, even under poor illumination or strong reflection. The experimental results have shown that, using a new multimodal dataset, polarimetric imaging was able to provide generic features for both good weather conditions and adverse weather conditions, especially fog. By combining polarimetric images with an adapted learning model, the different detection tasks under fog were improved by about 15% to 44%.