Lesson learned on how to develop and deploy light-weight models in the era of humongous Language Models