Ensemble learning of daboosting based on deep weighting for classification of hand-written numbers in Persian
Subject Areas : Image Processing
amir asil
1
,
hamed Alipour
2
,
Shahram mojtahedzadeh
3
,
hasan Asil
4
*
1 - Department of Electrical Engineering, Faculty of Electrical and Computer Engineering, Islamic Azad University, Azarshahr Branch
2 - Department of Electrical Engineering, Faculty of Electrical and Computer Engineering, Islamic Azad University, Tabriz Branch
3 - Department of Electrical Engineering, Faculty of Electrical and Computer Engineering, Islamic Azad University, Azarshahr Branch
4 - Department of Electrical Engineering, Faculty of Electrical and Computer Engineering, Islamic Azad University, Azarshahr Branch
Keywords: Deep learning, adaboosting, handwritten data, convolution, classification,
Abstract :
Today, the hand-written data volume is huge, which prohibits these data from being manually converted into electronic files. During the past years, different types of solutions were developed to convert machine learning-based handwritten data. Each method classifies or clusters the data according to the data type and application. In the present paper, a new approach is presented based on compound methods and deep learning for the classification of Persian handwritten data, where a deeper investigation is made of the data in basic learning by combining the Ada boosting and convolution. The present study aims at providing a new technique for classification of the images of handwritten Persian numbers. The structure of this technique is founded on Ada Boosting, which in turn, is based on weak learning. This technique improves learning by iteration of the weak learning processes and updating weights. In the meantime, the proposed method tried to employ stronger learners and present a stronger algorithm by combining these strong learners. The method was assessed on the standard Hoda dataset containing 60000 training data. The results show that the proposed method has a lower error rate than the previous methods by more than 1%. In the future, by developing basic learner, new mechanisms can be provided to improve the results by new types of learning. – Today, the hand-written data volume is huge, which prohibits these data from being manually converted into electronic files. During the past years, different types of solutions were developed to convert machine learning-based handwritten data. Each method classifies or clusters the data according to the data type and application. In the present paper, a new approach is presented based on compound methods and deep learning for the classification of Persian handwritten data, where a deeper investigation is made of the data in basic learning by combining the Ada boosting and convolution. The method was assessed on the standard Hoda dataset containing 60000 training data. The results showed that the error rate of the method has decreased by more than 1% compared to the previous methods.
[1] Maziyar Kazemi, Muhammad Yousefnezhad, Saber Nourian, "Persian Handwritten Letter Recognition Using Ensemble SVM Classifiers Based on Feature Extraction", National Conference on Intelligent Systems and Information and Communications Technology, At Tabriz, Iran, Volume: 12
[2] Addakiri K., Bahaj M. (2012) "On-line Handwritten Arabic Character Recognition using Artificial Neural Network", International Journal of Computer Applications (IJCA), Volume 55.
[3] Alizadeh H., Yous efnezhad M., Minaei-Bidgoli B. (2015)"Wisdom of Crowds Cluster Ensemble", Intelligent Data Analysis, IOS Press, Vol. 19(3).
[4] sadafsafavi,mehrdad jalali,Recommendation model of places of interest according to people's behavior pattern based on friends list based on deep learning,Journal of Information Systems and Telecommunication (JIST),2021-11-17,http://ijece.org/en/Article/29095
[5] Jamshid bagherzadeh, hasan asil, proposing a New Method of Image Classification Based on the AdaBoost Deep Belief Network Hybrid Method, TELKOMNIKA, Vol 17, No 5, 2019
[6] Mohammad Ebrahim Khademi,Mohammad Fakhredanesh,Farsi Conceptual Text Summarizer: A New Model in Continuous Vector Space,Journal of Information Systems and Telecommunication (JIST),2019-11-04,http://jist.ir/en/Article/15222
[7] Cheng Ju and Aur´elien Bibaut and Mark J. van der Laan, The Relative Performance of Ensemble Methods with Deep Convolutional Neural Networks for Image Classification, cornell University,2017: arXiv:1704.01664v1 [stat.ML] 5 Apr 2017
[8] amin Ollah Mah Abadi, Abdolmajid Jazemian,” Fuzzy diagnosis of Persian handwritten numbers”، THE CSI JOURNAL ON COMPUTER SCIENCE AND ENGINEERING, no 4, 2006
[9] Najmeh Ghanbari, mohamad Razavi, hasan Nabavi,” A Smart Properties Selection Method Based on Binary Gravitational Search Algorithm in Persian Handwriting Number Recognition System”, Electrical and Computer Engineering of Iran, 2011
[10] esmaiel miri,mohamad razavi, javad razavi, “The effect of clustering on the recognition of Persian manuscript cultivars with fuzzy classifier” ,2016
[11] ehsan jabir, reza jabir, reza ebrahimpour,” A combination of two-class clauses for recognizing Persian manuscript cultivars”, 16th Iranian Electrical Engineering Conference, 2007
[12] Corinna Cortes, Mehryar Mohri, Umar Syed, Deep Boosting, e 31 st International Conference on Machine Learning, Beijing, China, 2014
[13] Jafar tanha, Ensemble approaches to semi-supervised learning, UvA-DARE (Digital Academic Repository),2013,http://hdl.handle.net/11245/1.393046
[14] Ji Zhu, Hui Zou, Saharon Rosset and Trevor Hastie, Multi-class AdaBoost, Statistics and Its Interface Volume 2, 2009 349–360
[15] Freund, Y. and Schapire, R. A decision theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences, 1997 119–139 MR1473055
[16] Friedman, J. Greedy function approximation: a gradient boosting machine. Annals of Statistics, 2001 1189–1232. MR1873328
[17] Friedman, J., Hastie, T., and Tibshirani, R. Additive logistic regression: a statistical view of boosting. Annals of Statistics 2000, 337–407. MR1790002
[18] Jamshid bagherzadeh, hasan asil, proposing a New Method of Image Classification Based on the AdaBoost Deep Belief Network Hybrid Method, TELKOMNIKA, Vol 17, No 5, 2019
[19] N-ary Decomposition for Multi-class Classification. Machine Learning Journal (MLJ), 2019
[20] Learning with Annotation of Various Degrees, IEEE Transactions on Neural Network and Learning Systems (TNNLS), 2019.
[21] Farrokhi, Alireza and Razavi, Seyed Nasser, Recognition of handwritten digits using deep learning, International Conference on Nonlinear Systems and Optimization of Electrical and Computer Engineering, 2014, https://civilica.com/doc/383305
[22] Mehdi Noroozi and Paolo Favaro. “Unsupervised learning of visual representations by solving jigsaw puzzles”. In European Conference on Computer vision.Pages 69–84. Springer. 2016
[23] Karen Simonyi and Andrew Zisserman. Very deep convolutional networks for large scale image recognition. ArXiv preprint arXiv: 1409.1556, 2014.
[24] Pourqasem, Hossein and Hel Forosh, Mohammad Sadegh and Daneshvar, Sablan, Semantic classification of text images based on text value model, 2017, https://civilica.com/doc/1372237
[25] Aslimi Zamanjani, Javad and Shakur, Mohammad Hossein and Rahmani, Mohsen, Classification of noisy texture images using deep neural network and complete local binary pattern, 2021, https://civilica.com/doc/1362855
[26] Shahabinejad, Athara and Iftikhari, Mehdi, Image classification using deep convolutional neural networks based on distributed attention and Bayes inference, 5th National Technology Conference in Electrical and Computer Engineering, 2021, https:// civilica.com/doc/1281540
[27] Babaian, Vahidah and Madiri, Shaghaigh and Behlgardi, Seyedah Kausar, Classification of images using deep neural networks, The 5th National Conference on the Application of New Technologies in Engineering Sciences, Torbat Heydarieh, 2018, https://civilica.com/doc /1202833
[28] Transfer Hashing: From Shallow to Deep, IEEE Transactions on Neural Network and Learning Systems (TNNLS), 2018
[29] Harri Valpola. From neural PCA to deep unsupervised learning. In Adv. in Independent Component Analysis and Learning Machines, pages 143–171. Elsevier, 2015.arXiv:1411.7783.
[30] Learning Common and Feature-Specific Patterns: A Novel Multiple-Sparse-Representation-Based Tracker. IEEE Trans. Image Processing 27(4): 2022-2037 (2018)
[31] Aaron vandenOord, NalKalchbrenner, Pixel Recurrent Neural Networks, international Conference on Machine Learning, New York, NY, USA, 2016 ,2016,arXiv:1601.06759v3
[32] Jamishid Bagherzadeh, Hasan Asil, A review of various semi-supervised learning models with a deep learning and memory approach, Iran Journal of Computer Science, 2018
[33] Alex Krizhevsky, IlyaSutskever, and Geoffrey E Hinton.Imagenet classification with deep convolutional neural networks. IN Advances in neural information processing systems, pages 1097–1105, 2012.
[34] Christian Szegedy, Wei Liu, YangqingJia, Pierre Sermanet, Scott Reed, DragomirAnguelov, DumitruErhan,
[35] Andrej Karpathy, George Toderici, SankethShetty, Thomas Leung, Rahul Sukthankar, and Li Fei-Fei. Large-scale video classification with convolutional neural networks. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014.
[36] Christian Szegedy, Wei Liu, YangqingJia, Pierre Sermanet, Scott Reed, DragomirAnguelov, DumitruErhan, Vincent Vanhoucke, and Andrew Rabinovich. Going deeper with convolutions. ArXiv preprint arXiv: 1409.4842, 2014.
[37] M.M. Javidi Fatemeh SharifizadehFatemeh Sharifizadeh,” A Modified Decision Templates Method for Persian Handwritten Digit Recognition A Modified Decision Templates Method for Persian Handwritten Digit Recognition”, Journal of American Science, 2012