Toward unifying the description of meson and baryon properties