Jet splitting function in the vacuum and QCD medium