Transcription factors (TFs) binding sites prediction and analysis are vital for comprehending cis-regulatory mechanisms. Recently, several deep learning-based methods have shown outstanding performance on TFs binding sites (TFBSs) recognition by leveraging solely base-pair arrangement of regulatory sequences. Except for the aforementioned genomic features, the epigenomics signature represented by the histone modification is also a critical factor related to TFs-DNA binding. We present a multi-omics based hybrid neural network, dubbed as BHSite, for TFBSs prediction by adaptively integrating base-pair arrangements and histone modification signatures. Experiments over 196 ChIP-seq datasets demonstrate that BHSite significantly outperforms several state-of-the-art methods in TFBSs prediction. Besides, studies of the relative importance of histone modification signatures prove that diverse signatures complement each other. Furthermore, visualization analysis of Squeeze-and-Excitation Network reveals the contribution of multi-omics latent features concerning different cell types to TFBS prediction. Thus, BHSite improves both performance and interpretability by combining the multi-omic features into deep learning architecture.