Handwritten Digits Recognition Using an Ensemble Technique Based on the Firefly Algorithm
Subject Areas : Data MiningAzar Mahmoodzadeh 1 , Hamed Agahi 2 * , Marzieh Salehi 3
1 - Islamic Azad University Shiraz
2 - Islamic Azad University Shiraz
3 - Islamic Azad University,Shiraz Branch, Iran
Keywords: Optical character recognition, , feature selection, , multi-objective genetic algorithm, , classifiers ensemble, , evolutionary firefly algorithm,
Abstract :
This paper develops a multi-step procedure for classifying Farsi handwritten digits using a combination of classifiers. Generally, the technique relies on extracting a set of characteristics from handwritten samples, training multiple classifiers to learn to discriminate between digits, and finally combining the classifiers to enhance the overall system performance. First, a pre-processing course is performed to prepare the images for the main steps. Then three structural and statistical characteristics are extracted which include several features, among which a multi-objective genetic algorithm selects those more effective ones in order to reduce the computational complexity of the classification step. For the base classification, a decision tree (DT), an artificial neural networks (ANN) and a k-nearest neighbor (KNN) models are employed. Finally, the outcomes of the classifiers are fed into a classifier ensemble system to make the final decision. This hybrid system assigns different weights for each class selected by each classifier. These voting weights are adjusted by a metaheuristic firefly algorithm which optimizes the accuracy of the overall system. The performance of the implemented approach on the standard HODA dataset is compared with the base classifiers and some state-of-the-art methods. Evaluation of the proposed technique demonstrates that the proposed hybrid system attains high performance indices including accuracy of 98.88% with only eleven features.
[1] Liu CL, Nakashima K, Sako H, Fujisawa H. Handwritten digit recognition: benchmarking of state-of-the-art techniques. Pattern recognition. 2003 Oct 1;36(10):2271-85.
[2] Al-Jawfi R. Handwriting Arabic character recognition LeNet using neural network. Int. Arab J. Inf. Technol.. 2009 Jul 1;6(3):304-9.
[3] Ayyaz, M.N., Javed, I. and Mahmood, W., 2016. Handwritten character recognition using multiclass SVM classification with hybrid feature extraction. Pakistan Journal of Engineering and Applied Sciences.
[4] Hassan, A.K.A., 2018. Arabic (Indian) Handwritten Digits Recognition Using Multi feature and KNN Classifier. Journal of University of Babylon, 26(4), pp.10-17.
[5] Savas B, Eldén L. Handwritten digit classification using higher order singular value decomposition. Pattern recognition. 2007 Mar 1;40(3):993-1003
[6] Chen, Y., Xu, Z., Cai, S., Lang, Y. and Kuo, C.C.J., 2017. A Saak Transform Approach to Efficient, Scalable and Robust Handwritten Digits Recognition. arXiv preprint arXiv:1710.10714.
[7] Lu, W.S., 2017, August. Handwritten digits recognition using PCA of histogram of oriented gradient. In Communications, Computers and Signal Processing (PACRIM), 2017 IEEE Pacific Rim Conference on (pp. 1-5). IEEE.
[8] Boukharouba, A. and Bennia, A., 2017. Novel feature extraction technique for the recognition of handwritten digits. Applied Computing and Informatics, 13(1), pp.19-26.
[9] Qiao, J., Wang, G., Li, W. and Chen, M., 2018. An adaptive deep Q-learning strategy for handwritten digit recognition. Neural Networks.
[10] Alaei, A., U. Pal, and P. Nagabhushan. Using modified contour features and SVM based classifier for the recognition of Persian/Arabic handwritten numerals. in Advances in Pattern Recognition, 2009. ICAPR'09. Seventh International Conference on. 2009. IEEE.
[11] Nahvi, M., Kiaee, K., Ebrahimpour, R., improvement the feature extraction method of Gradient based on the discrete cosine transform for recognizing Farsi handwritten digits. 18th Iranian Conference on Electrical Engineering, 2010, Isfahan Iran, (in Farsi).
[12] Ebrahimpour, R., Esmkhani, A., Faridi, S.: Farsi handwritten digit recognition based on mixture of RBF experts. IEICE Electron. Express 7(14), 1014–1019 (2010)
[13] Abdi,M.J., Salimi, H.: Farsi handwriting recognition with mixture of RBF experts based on particle swarm optimization. Int. J. Inf. Sci. Comput. Math. 2, 129–136 (2010)
[14] Rashnodi O., Sajedi H., Abadeh, M., Elci A., Munot M., Joshi M., Sharma N., Gupta N., Sharma R., and Mihajlov B., “Persian Handwritten Digit Recognition Using Support Vector Machines,” the International Journal of Computer Applications, vol. 29, no. 12, pp. 1-6, 2011.
[15] Soltanzadeh, H. and M. Rahmati, Recognition of Persian handwritten digits using image profiles of multiple orientations. Pattern Recognition Letters, 2004. 25(14): p. 1569-1576.
[16] Ziaratban, M., K. Faez, and F. Faradji. Language-based feature extraction using template-matching in Farsi/Arabic handwritten numeral recognition. in Document Analysis and Recognition, 2007. ICDAR 2007. Ninth International Conference on. 2007. IEEE.
[17] Razavi, M., and Kabir, E., On-line Recognition of Farsi separate letters using the neural network. Third conference on machine vision and image processing, 2004, Tehran, Iran (in Farsi).
[18] El Kessab, B., et al., Extraction method of handwritten digit recognition tested on the mnist database. International Journal of Advanced Science & Technology, 2013. 50.
[19] Wshah, S., Z. Shi, and V. Govindaraju. Segmentation of Arabic handwriting based on both contour and skeleton segmentation. in Document Analysis and Recognition, 2009. ICDAR'09. 10th International Conference on. 2009. IEEE.
[20] Mah-Abadi, A., Kazemian, A., and, Torkamani, MA., Fuzzy recognition of Farsi handwritten numbers. Journal of Computer Science and Engineering, Spring 2006. P. 19-25, (In Farsi).
[21] Ghanbari.Y., Golzari, S., and Askari, S., Recognition of Farsi handwritten numbers using the Gabor filter, Principle component analysis and K nearest neighbor algorithm. 7th International Conference on Information and Knowledge Technology, Summer 2015. Orumieh. Iran, (In Farsi).
[22] Parseh, M.J. and Meftahi, M., 2017. A new combined feature extraction method for Persian handwritten digit recognition. International Journal of Image and Graphics, 17(02), p.1750012.
[23] Sadri J, Suen CY, Bui TD. Application of support vector machines for recognition of handwritten Arabic/Persian digits. InProceedings of Second Iranian Conference on Machine Vision and Image Processing 2003 Feb (Vol. 1, pp. 300-307).
[24] Mowlaei, A. and K. Faez. Recognition of isolated handwritten Persian/Arabic characters and numerals using support vector machines. in Neural Networks for Signal Processing, 2003. NNSP'03. 2003 IEEE 13th Workshop on. 2003. IEEE.
[25] Impedovo, D. and G. Pirlo, Zoning methods for handwritten character recognition: A survey. Pattern Recognition, 2014. 47(3): p. 969-981.
[26] Salimi, H. and D. Giveki, Farsi/Arabic handwritten digit recognition based on ensemble of SVD classifiers and reliable multi-phase PSO combination rule. International Journal on Document Analysis and Recognition (IJDAR), 2013. 16(4): p. 371-386.
[27] AlKhateeb, J.H. and M. Alseid. DBN-Based learning for Arabic handwritten digit recognition using DCT features. in Computer Science and Information Technology (CSIT), 2014 6th International Conference on. 2014. IEEE.
[28] Biglari, M., F. Mirzaei, and J.G. Neycharan, Persian/Arabic handwritten digit recognition using local binary pattern. International Journal of Digital Information and Wireless Communications (IJDIWC), 2014. 4(4): p. 486-492.
[29] Khorashadizadeh, S. and A. Latif, Arabic/farsi handwritten digit recognition using histogram of oriented gradient and chain code histogram. Int. Arab J. Inf. Technol., 2016. 13(4): p. 367-374.
[30] Fouladi, K., B.N. Araabi, and E. Kabir, A fast and accurate contour-based method for writer-dependent offline handwritten Farsi/Arabic subwords recognition. International Journal on Document Analysis and Recognition (IJDAR), 2014. 17(2): p. 181-203.
[31] Sajedi, H., Handwriting recognition of digits, signs, and numerical strings in Persian. Computers & Electrical Engineering, 2016. 49: p. 52-65.
[32] Safdari, R. and M.-S. Moin. A hierarchical feature learning for isolated Farsi handwritten digit recognition using sparse autoencoder. in Artificial Intelligence and Robotics (IRANOPEN), 2016. 2016. IEEE.
[33] Duda, R., P. Hart, and D. Stork, Pattern classification. 2nd edn Wiley. New York, 2001: p. 632.
[34] Khosravi, H. and E. Kabir, Introducing a very large dataset of handwritten Farsi digits and a study on their varieties. Pattern recognition letters, 2007. 28(10): p. 1133-1141.
[35] Askari, S., KHarashadizadeh, M., and Sadri, J., A new method for Recognizing Farsi handwritten numbers based on Pre-Classification. First Conference on Pattern Recognition and Image Analysis, 2012, Birjand, Iran (in Farsi).
[36] Deodhare, D., N.R. Suri, and R. Amit, Preprocessing and Image Enhancement Algorithms for a Form-based Intelligent Character Recognition System. IJCSA, 2005. 2(2): p. 131-144.
[37] Zeyaratban M, Faez K, Mozzafari S, Azvaji M. Presenting a New Structural Method Based on Partitioning Thinned Image for Recognition of Handwritten Farsi-Arabic Numerals. InProceedings of the Third Conference on Machine Vision, Image Processing and Applications, Farsi, Iran 2005 Feb (Vol. 1, pp. 76-82).
[38] Gunavathi, C. and K. Premalatha, Performance analysis of genetic algorithm with KNN and SVM for Feature Selection in Tumor Classification. International Journal of Computer, Electrical, Automation, Control and Information Engineering, 2014. 8(8): p. 1490-1497.
[39] Padma, A., Sukanesh, R., 2011 A wavelet based automatic segmentation of brain tumor in CT images using optimal statistical texture features, Int. J. Image Process., 5, (5), pp. 552–563
[40] Deb K, Pratap A, Agarwal S, Meyarivan T (2002) A fast and elitist multi objective genetic algorithm: NSGA-II. IEEE Trans Evolut Comput 6(2):181–197
[41] Breslow, L.A. and D.W. Aha, Simplifying decision trees: A survey. The Knowledge Engineering Review, 1997. 12(1): p. 1-40.
[42] Esmaeili, M., Data Mining Concepts And Techniques. Niaz Danesh Publication, Second edition, July 2012. P:233-260 (in Farsi).
[43] Lior R. Data mining with decision trees: theory and applications. World scientific; 2014 Sep 3.
[44] Singh S, Gupta P. Comparative study ID3, cart and C4. 5 decision tree algorithm: a survey. Int J Adv Inf Sci Technol Internet. 2014 Jul;27:97-103.
[45] Aggarwal, C.C. and S.Y. Philip, A general survey of privacy-preserving data mining models and algorithms, in Privacy-preserving data mining. 2008, Springer. p. 11-52.
[46] Van Gerven M, Bohte S, editors. Artificial neural networks as models of neural information processing. Frontiers Media SA; 2018 Feb 1.
[47] Shavlik, J.W., R.J. Mooney, and G.G. Towell, Symbolic and neural learning algorithms: An experimental comparison. Machine learning, 1991. 6(2): p. 111-143.
[48] Zhang X, Tian Y, Jin Y. A knee point-driven evolutionary algorithm for many-objective optimization. IEEE Transactions on Evolutionary Computation. 2015 Dec;19(6):761-76.
[49] Yang XS. Firefly algorithms for multimodal optimization. In International symposium on stochastic algorithms 2009 Oct 26 (pp. 169-178). Springer, Berlin, Heidelberg.
[50] Bozorg-Haddad O, Solgi M, Loà HA. Meta-heuristic and Evolutionary Algorithms for Engineering Optimization. John Wiley & Sons; 2017 Oct 9.
[51] Ekbal, A. and S. Saha, Combining feature selection and classifier ensemble using a multiobjective simulated annealing approach: application to named entity recognition. Soft Computing, 2013. 17(1): p. 1-16.