Characterization inference based on joint-optimization of multi-layer semantics and deep fusion matching network