DeepSumm: A Novel Deep Learning-Based Multi-Lingual Multi-Documents Summarization System
Subject Areas : Natural Language ProcessingShima Mehrabi 1 , Seyed Abolghassem Mirroshandel 2 * , Hamidreza Ahmadifar 3
1 - University of Guilan
2 - University of Guilan
3 - University of Guilan
Keywords: Artificial Neural Networks, , Deep Learning, , Text Stigmatization, , Multi-Documents, , Natural Language Processing, ,
Abstract :
With the increasing amount of accessible textual information via the internet, it seems necessary to have a summarization system that can generate a summary of information for user demands. Since a long time ago, summarization has been considered by natural language processing researchers. Today, with improvement in processing power and the development of computational tools, efforts to improve the performance of the summarization system is continued, especially with utilizing more powerful learning algorithms such as deep learning method. In this paper, a novel multi-lingual multi-document summarization system is proposed that works based on deep learning techniques, and it is amongst the first Persian summarization system by use of deep learning. The proposed system ranks the sentences based on some predefined features and by using a deep artificial neural network. A comprehensive study about the effect of different features was also done to achieve the best possible features combination. The performance of the proposed system is evaluated on the standard baseline datasets in Persian and English. The result of evaluations demonstrates the effectiveness and success of the proposed summarization system in both languages. It can be said that the proposed method has achieve the state of the art performance in Persian and English.
[1] D. Das and A. Martins, “A Survey on Automatic Text Summarization,” Literature Survey for the Language and Statistics II Course at Carnegie Mellon University, 2007, pp.1-31.
[2] D. Timothy, T. Allison, S. Blair-goldensohn, J. Blitzer, A. Elebi, S. Dimitrov, E. Drabek, A. Hakim, W. Lam, D. Liu et al., “Mead A Platform For Multidocument Multilingual Text Summarization,” in International Conference on Language Resources and Evaluation, 2004, pp. 699-702.
[3] T. A. S. Pardo, L. H. M. Rino, and M. d. G. V. Nunes, “Gistsumm: A Summarization Tool Based On A New Extractive Method,” in International Workshop on Computational Processing of the Portuguese Language. Springer, 2003, pp. 210–218.
[4] M. Hassel and N. Mazdak, “Farsisum - A Persian Text Summarizer,” in Proceedings of the Workshop on Computational Approaches to Arabic Script-based Languages, 2004, pp. 82–84.
[5] Z. Karimi and M. Shamsfard, “Summarization of Persian Text,” in Proceedings of the 12th Computer Society of Iran, 2007, pp. 1286-1294. [6] M. A. Honarpisheh, G. Ghassem-Sani, and G. Mirroshandel, “A Multidocument Multi-Lingual Automatic Summarization System,” in Proceedings of the Third International Joint Conference on Natural Language Processing: Volume-II, 2008, pp. 733-738.
[7] F. Kiyoumarsi and F. Rahimi Esfahani, “Optimizing Persian Text Summarization Based on Fuzzy Logic Approach,” Proceedings of the International Conference on Intelligent Building and Management, 2011, pp. 264-269.
[8] Y. Bengio, “Learning Deep Architectures for AI,” Foundations and Trends in Machine Learning, 2009, vol. 2, no. 1, pp. 1–127.
[9] G. Hinton, L. Deng, D. Yu, G. Dahl, A. Mohamed, N. Jaitly, A. Senior, V. Vanhoucke, P. Nguyen, B. Kingsbury et al., “Deep Neural Networks for Acoustic Modeling in Recognition,” IEEE Signal processing magazine, 2012, vol. 29, no. 6, pp. 82-97.
[10] R. Collobert and J. Weston, “A Unified Architecture for Natural Language Processing: Deep Neural Networks with Multitask Learning,” in Proceedings of the 25th international conference on Machine learning. ACM, 2008, pp. 160–167.
[11] R. Collobert, J. Weston, L. Bottou, M. Karlen, M. Kayukcuoglu, and P. Kuksa, “Natural Language Processing (almost) from Scratch,” Journal of Machine Learning Research, 2011, vol. 12, no. Aug, pp. 2493-2537.
[12] E. Arisoy, T. N. Sainath, B. Kingsbury, and B. Ramabhadran, “Deep Neural Network Language Models,” in Proceedings of the NAACL-HLT 2012 Workshop: Will We Ever Really Replace the N-gram Model? On the Future of Language Modeling for HLT. Association for Computational Linguistics, 2012, pp. 20–28.
[13] Y. Liu, S. Zhong, and W. Li, “Query-Oriented Multi-Document Summarization via Unsupervised Deep Learning,” in Proceedings of the 26th Conference on Artificial Intelligence, 2012, pp. 1699-1705.
[14] M. Yousefi-Azar and L. Hamey, “Text Summarization Using Unsupervised Deep Learning,” Expert System with Application, 2017, vol. 68, pp. 93-105.
[15] A. Jain, D. Bhatia, and M. K. Thakur, “Extractive Text Summarization using Word Vector Embedding,” in Proceedings of International Conference on Machine learning and Data Science, 2017, pp. 51-55.
[16] N. S. Shirwandkar and S. Kulkarni, “Extractive Text Summarization Using Deep Learning,” in Proceedings of 4th International Conference on Computing Communication Control and Automation, 2018, pp. 1-5.
[17] H. Geoffrey, O. Simon, and T. Yee-Whye, “A Fast Learning Algorithm for Deep Belief Nets,” Neural Computation, 2008, vol. 18, pp. 1527-1554.
[18] A. Fischer and C. Igel, “An Introduction to Restricted Boltzmann Machines,” in Proceedings of the 17th Iberoamerican Congress on Pattern Recognition, 2012, pp. 14-36.
[19] P. Vincent, H. Larochelle, I. Lajoie, Y. Bengio, and P. Manzagol, “Stacked Denoising Autoencoders: Learning Useful Representation in a Deep Network with a Local Denoising Criterion,” Journal of Machine Learning Research, 2010, vol. 11, pp. 3371-3408.
[20] A. Pourmasoumi, M. Kahani, A. Toosi, A. Estiri, and H. Ghaemi , “Ijaz: A Single Document Summarization System for Persian News Text,” Signal and Data Processing, 2014, vol. 21, no. 1, pp. 33-48.
[21] M. Prabhakar and N. Chandra, “Automatic Text Summarization Based on Pragmatic Analysis,” International Journal of Scientific and Research Publications, 2012, vol. 2, no. 5, pp. 1-4.
[22] R. Mihalecea and P. Tarau, “TextRank: Bringing Order into Texts,” in Proceedings of the Empirical Methods in Natural Language Processing, 2004, pp. 404-411.
[23] M. Shamsfard, “Challenges and Open Problems in Persian Text Processing,” in Proceedings of the 5th Language and Technology Conference, 2011, pp.65-69.
[24] B. Behmadi Moghaddas, M. Kahani, S.A. Toosi, A. Pourmasoumi, and A. Estiri, “Pasokh: A Standard Corpus for the Evaluation of Persian Text Summarizers,” in Proceedings of the International Conference on Computer and Knowledge Engineering, 2013, pp. 471-475.
[25] D. M. Ward Powers, “Evaluation: From Precision, Recall, and F-Measure to Roc, Informedness, markedness & Correlation,” Journal of Machine Learning Technologies, 2011, vol. 2, no. 1, pp. 37-63.
[26] C. Lin, “Rouge: A Package for Automatic Evaluation of Summaries,” in Proceedings of the ACL Workshop on Text Summarization Branches out, 2004, pp. 74-81.
[27] X. Wan and J. Xiao, “Graph Based Multi-Modality Learning for Topic Focused Multi Document Summarization,” in Proceedings of the 21st International joint conference on Artificial intelligence, 2009, pp. 1586-1591.
[28] T. Joachims, “Optimizing Search Engines Using Click Through Data,” in Proceedings of the 8th International Conference on Knowledge Discovery and Data Mining, 2002, pp. 133-142.
[29] Y. Ouyang, W. J. Li, S. J. Li, and Q. Lu, “Applying Regression Models to Query Focused Multi Document Summarization,” Information Processing and Management, 2011, vol. 47, no. 2, pp. 227-237.
[30] H. T. Dang, “Overview of DUC 2005,” in Proceedings of the Document Understanding Conference, 2005, pp. 1-12.
[31] D. Mendoza, C. Cobos, E. Len, M. Lozano, F. Rodrguez, E. Herrera-Viedma, “A new memetic algorithm for multi-document summarization based on CHC algorithm and greedy search,” Human-Inspired Computing and Its Applications, Springer International Publishing, 2014, vol. 8856, pp. 125-138.
[32] R.M. Alguliev, R.M. Aliguliyev, N.R. Isazade, “An unsupervised approach to generating generic summaries of documents,” Applied Soft Computing, 2015, vol. 34, pp. 236-250.
[33] P. Ren, Z. Chen, Z. Ren, F. Wei, L. Nie, J. Ma, M. de Rijke,” Sentence relations for extractive summarization with deep neural networks,” ACM Transactions on Information Systems, 2018, vol. 36, pp. 1-32.