A hybrid deep generative neural model for financial report generation

Abstract Generating long macro reports from a piece of breaking news is quite a challenging task. Essentially, this task is a long text generation problem from short text. Apparently, the difficulty of this task lies in the logic inference of human beings. To address this issue, this paper proposes a novel hybrid deep generative neural model which first learns the outline of the input news and then generates macro financial reports from the learnt outline. In the outline generation component, we generate the outline text using the framework of Pointer-Generator network with attention mechanism. In the target report generation component, we generate the macro financial reports by the revised VAE model. To train our end-to-end model, we have collected the experimental dataset containing over one hundred thousand pairs of news-report data. Extensive experiments are then evaluated on this dataset. The proposed model achieves the SOTA performance against both the baseline models and the state-of-the-art models with respect to evaluation criteria BLEU, ROUGE and human scores. Although the readability of the generated reports by our approach is better than that of the rest models, it remains an open problem which needs further efforts in the future.

[1]  Xiaofeng Zhang,et al.  Generating Long Financial Report using Conditional Variational Autoencoders with Knowledge Distillation , 2020, AAAI.

[2]  Jingyu Wang,et al.  Adversarial and Domain-Aware BERT for Cross-Domain Sentiment Analysis , 2020, ACL.

[3]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[4]  Hang Li,et al.  “ Tony ” DNN Embedding for “ Tony ” Selective Read for “ Tony ” ( a ) Attention-based Encoder-Decoder ( RNNSearch ) ( c ) State Update s 4 SourceVocabulary Softmax Prob , 2016 .

[5]  Xu Tan,et al.  MASS: Masked Sequence to Sequence Pre-training for Language Generation , 2019, ICML.

[6]  Kai Liu,et al.  Wasserstein autoencoders for collaborative filtering , 2018, Neural Computing and Applications.

[7]  Arya D. McCarthy,et al.  Addressing Posterior Collapse with Mutual Information for Improved Variational Neural Machine Translation , 2020, ACL.

[8]  Mirella Lapata,et al.  Learning to Generate Product Reviews from Attributes , 2017, EACL.

[9]  Shuming Shi,et al.  QuaSE: Sequence Editing under Quantifiable Guidance , 2018, EMNLP.

[10]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[11]  Matt J. Kusner,et al.  GANS for Sequences of Discrete Elements with the Gumbel-softmax Distribution , 2016, ArXiv.

[12]  Andrew M. Dai,et al.  MaskGAN: Better Text Generation via Filling in the ______ , 2018, ICLR.

[13]  Jiaxin Pei,et al.  Pre-train and Plug-in: Flexible Conditional Text Generation with Variational Auto-Encoders , 2020, ACL.

[14]  Lantao Yu,et al.  SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient , 2016, AAAI.

[15]  Zhe Gan,et al.  Improving Adversarial Text Generation by Modeling the Distant Future , 2020, ACL.

[16]  Roberto Basili,et al.  GAN-BERT: Generative Adversarial Learning for Robust Text Classification with a Bunch of Labeled Examples , 2020, ACL.

[17]  Xiaocheng Feng,et al.  Topic-to-Essay Generation with Neural Networks , 2018, IJCAI.

[18]  Kevin Lin,et al.  Adversarial Ranking for Language Generation , 2017, NIPS.

[19]  Chin-Yew Lin,et al.  ROUGE: A Package for Automatic Evaluation of Summaries , 2004, ACL 2004.

[20]  Ming Zhou,et al.  Combining Multiple Resources to Improve SMT-based Paraphrasing Model , 2008, ACL.

[21]  Ming Zhou,et al.  Hierarchical Recurrent Neural Network for Document Modeling , 2015, EMNLP.

[22]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[23]  Alan Ritter,et al.  Adversarial Learning for Neural Dialogue Generation , 2017, EMNLP.

[24]  Bowen Zhou,et al.  Multiresolution Recurrent Neural Networks: An Application to Dialogue Response Generation , 2016, AAAI.

[25]  Yong Yu,et al.  Long Text Generation via Adversarial Training with Leaked Information , 2017, AAAI.

[26]  Markus Freitag,et al.  Beam Search Strategies for Neural Machine Translation , 2017, NMT@ACL.

[27]  Xiaofeng Zhang,et al.  A novel hybrid deep recommendation system to differentiate user's preference and item's attractiveness , 2020, Inf. Sci..

[28]  Chris Quirk,et al.  Monolingual Machine Translation for Paraphrase Generation , 2004, EMNLP.

[29]  Phil Blunsom,et al.  Neural Variational Inference for Text Processing , 2015, ICML.

[30]  Xiaofei Yang,et al.  Heterogeneous-Temporal Graph Convolutional Networks: Make the Community Detection Much Better , 2019 .

[31]  Wei-Ying Ma,et al.  Topic Aware Neural Response Generation , 2016, AAAI.

[32]  Salim Roukos,et al.  Bleu: a Method for Automatic Evaluation of Machine Translation , 2002, ACL.

[33]  Phil Blunsom,et al.  Language as a Latent Variable: Discrete Generative Models for Sentence Compression , 2016, EMNLP.

[34]  Hao Liu,et al.  Pattern-revising Enhanced Simple Question Answering over Knowledge Bases , 2018, COLING.

[35]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[36]  Lei Li,et al.  Enhancing Topic-to-Essay Generation with External Commonsense Knowledge , 2019, ACL.

[37]  Xiaojun Wan,et al.  T-CVAE: Transformer-Based Conditioned Variational Autoencoder for Story Completion , 2019, IJCAI.

[38]  Song Xu,et al.  Self-Attention Guided Copy Mechanism for Abstractive Summarization , 2020, ACL.

[39]  Gemma Boleda,et al.  Probing for Referential Information in Language Models , 2020, ACL.

[40]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[41]  Wei Zhao,et al.  SUPERT: Towards New Frontiers in Unsupervised Evaluation Metrics for Multi-Document Summarization , 2020, ACL.

[42]  Dongyan Zhao,et al.  Draft and Edit: Automatic Storytelling Through Multi-Pass Hierarchical Conditional Variational Autoencoder , 2020, AAAI.

[43]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[44]  Erhardt Barth,et al.  A Hybrid Convolutional Variational Autoencoder for Text Generation , 2017, EMNLP.

[45]  Yiming Yang,et al.  XLNet: Generalized Autoregressive Pretraining for Language Understanding , 2019, NeurIPS.

[46]  Samy Bengio,et al.  Generating Sentences from a Continuous Space , 2015, CoNLL.

[47]  Lav R. Varshney,et al.  CTRL: A Conditional Transformer Language Model for Controllable Generation , 2019, ArXiv.

[48]  Christopher D. Manning,et al.  Get To The Point: Summarization with Pointer-Generator Networks , 2017, ACL.