List of Articles Deep Learning

• Open Access Article
  • Abstract Page
  • Full-Text
  
  1 - Information Bottleneck and its Applications in Deep Learning
  Hassan Hafez Kolahi Shohreh Kasaei
  10.7508/jist.2018.03.001
  Information Theory (IT) has been used in Machine Learning (ML) from early days of this field. In the last decade, advances in Deep Neural Networks (DNNs) have led to surprising improvements in many applications of ML. The result has been a paradigm shift in the communit More

  Information Theory (IT) has been used in Machine Learning (ML) from early days of this field. In the last decade, advances in Deep Neural Networks (DNNs) have led to surprising improvements in many applications of ML. The result has been a paradigm shift in the community toward revisiting previous ideas and applications in this new framework. Ideas from IT are no exception. One of the ideas which is being revisited by many researchers in this new era, is Information Bottleneck (IB); a formulation of information extraction based on IT. The IB is promising in both analyzing and improving DNNs. The goal of this survey is to review the IB concept and demonstrate its applications in deep learning. The information theoretic nature of IB, makes it also a good candidate in showing the more general concept of how IT can be used in ML. Two important concepts are highlighted in this narrative on the subject, i) the concise and universal view that IT provides on seemingly unrelated methods of ML, demonstrated by explaining how IB relates to minimal sufficient statistics, stochastic gradient descent, and variational auto-encoders, and ii) the common technical mistakes and problems caused by applying ideas from IT, which is discussed by a careful study of some recent methods suffering from them. Manuscript profile
• Open Access Article
  • Abstract Page
  • Full-Text
  
  2 - DeepSumm: A Novel Deep Learning-Based Multi-Lingual Multi-Documents Summarization System
  Shima Mehrabi Seyed Abolghassem Mirroshandel Hamidreza  Ahmadifar
  10.7508/jist.2019.03.005
  With the increasing amount of accessible textual information via the internet, it seems necessary to have a summarization system that can generate a summary of information for user demands. Since a long time ago, summarization has been considered by natural language pro More

  With the increasing amount of accessible textual information via the internet, it seems necessary to have a summarization system that can generate a summary of information for user demands. Since a long time ago, summarization has been considered by natural language processing researchers. Today, with improvement in processing power and the development of computational tools, efforts to improve the performance of the summarization system is continued, especially with utilizing more powerful learning algorithms such as deep learning method. In this paper, a novel multi-lingual multi-document summarization system is proposed that works based on deep learning techniques, and it is amongst the first Persian summarization system by use of deep learning. The proposed system ranks the sentences based on some predefined features and by using a deep artificial neural network. A comprehensive study about the effect of different features was also done to achieve the best possible features combination. The performance of the proposed system is evaluated on the standard baseline datasets in Persian and English. The result of evaluations demonstrates the effectiveness and success of the proposed summarization system in both languages. It can be said that the proposed method has achieve the state of the art performance in Persian and English. Manuscript profile
• Open Access Article
  • Abstract Page
  • Full-Text
  
  3 - Utilizing Gated Recurrent Units to Retain Long Term Dependencies with Recurrent Neural Network in Text Classification
  Nidhi Chandra Laxmi  Ahuja Sunil Kumar Khatri Himanshu Monga
  10.52547/jist.9.34.89
  20.1001.1.23221437.2021.9.34.2.9
  The classification of text is one of the key areas of research for natural language processing. Most of the organizations get customer reviews and feedbacks for their products for which they want quick reviews to action on them. Manual reviews would take a lot of time a More

  The classification of text is one of the key areas of research for natural language processing. Most of the organizations get customer reviews and feedbacks for their products for which they want quick reviews to action on them. Manual reviews would take a lot of time and effort and may impact their product sales, so to make it quick these organizations have asked their IT to leverage machine learning algorithms to process such text on a real-time basis. Gated recurrent units (GRUs) algorithms which is an extension of the Recurrent Neural Network and referred to as gating mechanism in the network helps provides such mechanism. Recurrent Neural Networks (RNN) has demonstrated to be the main alternative to deal with sequence classification and have demonstrated satisfactory to keep up the information from past outcomes and influence those outcomes for performance adjustment. The GRU model helps in rectifying gradient problems which can help benefit multiple use cases by making this model learn long-term dependencies in text data structures. A few of the use cases that follow are – sentiment analysis for NLP. GRU with RNN is being used as it would need to retain long-term dependencies. This paper presents a text classification technique using a sequential word embedding processed using gated recurrent unit sigmoid function in a Recurrent neural network. This paper focuses on classifying text using the Gated Recurrent Units method that makes use of the framework for embedding fixed size, matrix text. It helps specifically inform the network of long-term dependencies. We leveraged the GRU model on the movie review dataset with a classification accuracy of 87%. Manuscript profile
• Open Access Article
  • Abstract Page
  • Full-Text
  
  4 - Rough Sets Theory with Deep Learning for Tracking in Natural Interaction with Deaf
  Mohammad Ebrahimi Hossein Ebrahimpour-Komeleh
  10.52547/jist.16060.10.39.211
  20.1001.1.23221437.2022.10.39.7.1
  Sign languages commonly serve as an alternative or complementary mode of human communication Tracking is one of the most fundamental problems in computer vision, and use in a long list of applications such as sign languages recognition. Despite great advances in recent More

  Sign languages commonly serve as an alternative or complementary mode of human communication Tracking is one of the most fundamental problems in computer vision, and use in a long list of applications such as sign languages recognition. Despite great advances in recent years, tracking remains challenging due to many factors including occlusion, scale variation, etc. The mistake detecting of head or left hand instead of right hand in overlapping are, modes like this, and due to the uncertainty of the hand area over the deaf news video frames; we proposed two methods: first, tracking using particle filter and second tracking using the idea of the rough set theory in granular information with deep neural network. We proposed the method for Combination the Rough Set with Deep Neural Network and used for in Hand/Head Tracking in Video Signal DeafNews. We develop a tracking system for Deaf News. We used rough set theory to increase the accuracy of skin segmentation in video signal. Using deep neural network, we extracted inherent relationships available in the frame pixels and generalized the achieved features to tracking. The system proposed is tested on the 33 of Deaf News with 100 different words and 1927 video files for words then recall, MOTA and MOTP values are obtained. Manuscript profile
• Open Access Article
  • Abstract Page
  • Full-Text
  
  5 - Deep Transformer-based Representation for Text Chunking
  Parsa Kavehzadeh Mohammad Mahdi  Abdollah Pour Saeedeh Momtazi
  10.61186/jist.19894.11.43.176
  20.1001.1.23221437.2023.11.43.2.1
  Text chunking is one of the basic tasks in natural language processing. Most proposed models in recent years were employed on chunking and other sequence labeling tasks simultaneously and they were mostly based on Recurrent Neural Networks (RNN) and Conditional Random F More

  Text chunking is one of the basic tasks in natural language processing. Most proposed models in recent years were employed on chunking and other sequence labeling tasks simultaneously and they were mostly based on Recurrent Neural Networks (RNN) and Conditional Random Field (CRF). In this article, we use state-of-the-art transformer-based models in combination with CRF, Long Short-Term Memory (LSTM)-CRF as well as a simple dense layer to study the impact of different pre-trained models on the overall performance in text chunking. To this aim, we evaluate BERT, RoBERTa, Funnel Transformer, XLM, XLM-RoBERTa, BART, and GPT2 as candidates of contextualized models. Our experiments exhibit that all transformer-based models except GPT2 achieved close and high scores on text chunking. Due to the unique unidirectional architecture of GPT2, it shows a relatively poor performance on text chunking in comparison to other bidirectional transformer-based architectures. Our experiments also revealed that adding a LSTM layer to transformer-based models does not significantly improve the results since LSTM does not add additional features to assist the model to achieve more information from the input compared to the deep contextualized models. Manuscript profile
• Open Access Article
  • Abstract Page
  • Full-Text
  
  6 - Deep Learning-based Educational User Profile and User Rating Recommendation System for E-Learning
  Pradnya Vaibhav  Kulkarni Sunil Rai Rajneeshkaur Sachdeo Rohini Kale
  10.61186/jist.27448.11.43.185
  20.1001.1.23221437.2023.11.43.3.2
  In the current era of online learning, the recommendation system for the eLearning process is quite important. Since the COVID-19 pandemic, eLearning has undergone a complete transformation. Existing eLearning Recommendation Systems worked on collaborative filtering or More

  In the current era of online learning, the recommendation system for the eLearning process is quite important. Since the COVID-19 pandemic, eLearning has undergone a complete transformation. Existing eLearning Recommendation Systems worked on collaborative filtering or content-based filtering based on historical data, students’ previous grade, results, or user profiles. The eLearning system selected courses based on these parameters in a generalized manner rather than on a personalized basis. Personalized recommendations, information relevancy, choosing the proper course, and recommendation accuracy are some of the issues in eLearning recommendation systems. In this paper, existing conventional eLearning and course recommendation systems are studied in detail and compared with the proposed approach. We have used, the dataset of User Profile and User Rating for a recommendation of the course. K Nearest Neighbor, Support Vector Machine, Decision Tree, Random Forest, Nave Bayes, Linear Regression, Linear Discriminant Analysis, and Neural Network were among the Machine Learning techniques explored and deployed. The accuracy achieved for all these algorithms ranges from 0.81 to 0.97. The proposed algorithm uses a hybrid approach by combining collaborative filtering and deep learning. We have improved accuracy to 0.98 which indicate that the proposed model can provide personalized and accurate eLearning recommendation for the individual user. Manuscript profile
• Open Access Article
  • Abstract Page
  • Full-Text
  
  7 - An Autoencoder based Emotional Stress State Detection Approach by using Electroencephalography Signals
  Jia Uddin
  10.52547/jist.32267.11.41.24
  20.1001.1.23221437.2023.11.41.6.1
  Identifying hazards from human error is critical for industrial safety since dangerous and reckless industrial worker actions, as well as a lack of measures, are directly accountable for human-caused problems. Lack of sleep, poor nutrition, physical deformities, and wea More

  Identifying hazards from human error is critical for industrial safety since dangerous and reckless industrial worker actions, as well as a lack of measures, are directly accountable for human-caused problems. Lack of sleep, poor nutrition, physical deformities, and weariness are some of the key factors that contribute to these risky and reckless behaviors that might put a person in a perilous scenario. This scenario causes discomfort, worry, despair, cardiovascular disease, a rapid heart rate, and a slew of other undesirable outcomes. As a result, it would be advantageous to recognize people's mental states in the future in order to provide better care for them. Researchers have been studying electroencephalogram (EEG) signals to determine a person's stress level at work in recent years. A full feature analysis from domains is necessary to develop a successful machine learning model using electroencephalogram (EEG) inputs. By analyzing EEG data, a time-frequency based hybrid bag of features is designed in this research to determine human stress dependent on their sex. This collection of characteristics includes features from two types of assessments: time-domain statistical analysis and frequency-domain wavelet-based feature assessment. The suggested two layered autoencoder based neural networks (AENN) are then used to identify the stress level using a hybrid bag of features. The experiment uses the DEAP dataset, which is freely available. The proposed method has a male accuracy of 77.09% and a female accuracy of 80.93%. Manuscript profile
• Open Access Article
  • Abstract Page
  • Full-Text
  
  8 - Convolutional Neural Networks for Medical Image Segmentation and Classification: A Review
  Jenifer S Carmel Mary Belinda M J
  10.61186/jist.37936.11.44.347
  Medical imaging refers to the process of obtaining images of internal organs for therapeutic purposes such as discovering or studying diseases. The primary objective of medical image analysis is to improve the efficacy of clinical research and treatment options. Deep le More

  Medical imaging refers to the process of obtaining images of internal organs for therapeutic purposes such as discovering or studying diseases. The primary objective of medical image analysis is to improve the efficacy of clinical research and treatment options. Deep learning has revamped medical image analysis, yielding excellent results in image processing tasks such as registration, segmentation, feature extraction, and classification. The prime motivations for this are the availability of computational resources and the resurgence of deep Convolutional Neural Networks. Deep learning techniques are good at observing hidden patterns in images and supporting clinicians in achieving diagnostic perfection. It has proven to be the most effective method for organ segmentation, cancer detection, disease categorization, and computer-assisted diagnosis. Many deep learning approaches have been published to analyze medical images for various diagnostic purposes. In this paper, we review the works exploiting current state-of-the-art deep learning approaches in medical image processing. We begin the survey by providing a synopsis of research works in medical imaging based on convolutional neural networks. Second, we discuss popular pre-trained models and General Adversarial Networks that aid in improving convolutional networks’ performance. Finally, to ease direct evaluation, we compile the performance metrics of deep learning models focusing on covid-19 detection and child bone age prediction. Manuscript profile