Sentiment Analysis of NU Online Applications Using Artificial Neural Network
Keywords:
Neural Network, NU online, Sentiment
Abstract
The NU Online app on the Playstore serves the needs of Muslims, especially those in Islamic boarding schools, by providing information and services. Its success is gauged not just by the number of downloads or popularity but by the quality of user interactions and how well it meets user needs. Sentiment analysis of user reviews provides deeper insights into these aspects. This research focused on finding words influencing sentiment from NU online and producing the best performance of artificial neural networks. This study collected user reviews from the NU Online app between February 9, 2021, and May 31, 2024, totalling 12613 reviews. After preprocessing, 8546 reviews remained. Using the Indonesian Sentiment Lexicon (INSET), 66% of the reviews showed positive sentiment, 21% were neutral, and 13% were negative. The words "aplikasi" (application) and "nya" (its) appeared in the top three across all sentiment classes, while "fitur" (feature) was common in both positive and negative sentiments. For neutral sentiments, "nan" was frequently mentioned. The data were split into training and testing sets in an 80:20 ratio, preserving the proportions of each sentiment class. Sentiment analysis was performed using a neural network, with input neurons ranging from the top 10 words from each sentiment class to all words. Accuracy improved as more words were used, peaking at 0.95 for the top 1690 words, compared to 0.71 for the top 10 words. The findings highlight the importance of using a comprehensive set of words to train the ANN. Including more words significantly enhances the model's performance, indicating that a richer vocabulary captures sentiment nuances better.
References
Abbas, M., Ali, K., Memon, S., Jamali, A., Memon, S., & Ahmed, A. (2019). Multinomial Naive Bayes Classification Model for Sentiment Analysis. International Journal of Computer Science and Network Security, 19(3), 62–67. https://doi.org/10.13140/RG.2.2.30021.40169
Abdulla, N. A., Ahmed, N. A., Shehab, M. A., & Al-Ayyoub, M. (2013). Arabic sentiment analysis: Lexicon-based and corpus-based. 2013 IEEE Jordan Conference on Applied Electrical Engineering and Computing Technologies (AEECT), 1–6. https://doi.org/10.1109/AEECT.2013.6716448
Ahmad, M., Aftab, S., Salman, M., & Hameed, N. (2018). Sentiment Analysis using SVM: A Systematic Literature Review. International Journal of Advanced Computer Science and Applications, 9(2), 182–188. https://doi.org/10.14569/IJACSA.2018.090226
Ahmad, M., Aftab, S., Salman, M., Hameed, N., Ali, I., & Nawaz, Z. (2018). SVM Optimization for Sentiment Analysis. International Journal of Advanced Computer Science and Applications, 9(4), 393–398. https://doi.org/10.14569/IJACSA.2018.090455
Amalia, C., & Sibaroni, Y. (2020). Analisis Sentimen Data Tweet Menggunakan Model Jaringan Saraf Tiruan Dengan Pembobotan Delta Tf-idf. eProceedings of Engineering, 7(2), Article 2.
Azhar, Y. (2017). Metode Lexicon-Learning Based Untuk Identifikasi Tweet Opini Berbahasa Indonesia. Jurnal Nasional Pendidikan Teknik Informatika : JANAPATI, 6(3), Article 3. https://doi.org/10.23887/janapati.v6i3.11739
Bajari, A., Wahyudin, U., Koswara, I., & Erlandia, D. (2021). The influence of android-based ‘ISLAM SEHAT’ application for traditional Islamic boarding schools’ (TIBS) students in Indonesia. Kasetsart Journal of Social Sciences, 42, 862–870. https://doi.org/10.34044/j.kjss.2021.42.4.21
Betos, J. Z., David, M. L., Doctor, G. J., Lagan, J., Catacutan-Bangit, A., Fabito, B., & Manuel, R. (2021). NU READY: A Web and Mobile Application Framework for School Emergency Response. TENCON 2021 - 2021 IEEE Region 10 Conference (TENCON), 676–680. https://doi.org/10.1109/TENCON54134.2021.9707439
Borele, P., & Borikar, D. A. (2016). An Approach to Sentiment Analysis using Artificial Neural Network with Comparative Analysis of Different Techniques. IOSR Journal of Computer Engineering (IOSR-JCE), 18(2), 64–69.
Bothma, R. (2023). Factors that influence user retention in mobile apps [Masters]. University of Cape Town.
Budianita, E., & Cynthia, E. P. (2022). Pendekatan berbasis Machine Learning dan Leksikal Pada Analisis Sentimen. Seminar Nasional Teknologi Informasi Komunikasi dan Industri, 0, Article 0.
Cindo, M., Rini, D. P., & Ermatita, E. (2020). Sentiment Analysis On Twitter By Using Maximum Entropy And Support Vector Machine Method. SINERGI, 24(2), 87–94. https://doi.org/10.22441/sinergi.2020.2.002
CNN Indonesia. (2023, March 6). MPR: Mengkhawatirkan, 72 Persen Muslim Indonesia Buta Aksara Al-Quran. nasional. https://www.cnnindonesia.com/nasional/20230306064622-20-921284/mpr-mengkhawatirkan-72-persen-muslim-indonesia-buta-aksara-al-quran
Duric, A., & Song, F. (2012). Feature selection for sentiment analysis based on content and syntax models. Decision Support Systems, 53(4), 704–711. https://doi.org/10.1016/j.dss.2012.05.023
Gunawan, A. L. S. A.-Z., Jondri, J., & Lhaksamana, K. M. (2021). Analisis Sentimen Pada Media Sosial Twitter Terhadap Penanganan Bencana Banjir Di Jawa Barat Dengan Metode Jaringan Saraf Tiruan. eProceedings of Engineering, 8(2), Article 2.
Han, K.-X., Chien, W., Chiu, C.-C., & Cheng, Y.-T. (2020). Application of Support Vector Machine (SVM) in the Sentiment Analysis of Twitter DataSet. Applied Sciences, 10(3), Article 3. https://doi.org/10.3390/app10031125
Haykin, S. (1999). Neural Networks: A Comprehensive Foundation. Prentice Hall.
Hidayatullah, A. F., & Ma’arif, M. R. (2017). Pre-processing Tasks in Indonesian Twitter Messages. Journal of Physics: Conference Series, 801(1), 012072. https://doi.org/10.1088/1742-6596/801/1/012072
Huda, F. (2020). Teachers’ Barriers Of Using Information And Communication Technology (Ict) In Teaching English: A Case Study Of English Teachers At Islamic Boarding School Mu’allimin Muhammadiyah Bangkinang [Thesis, UIN Suska Riau]. https://repository.uin-suska.ac.id/38586/
Ihsan, F. M., Permana, P. Y. E., Arifin, N. R., & Muhtadi, R. (2020). Islamic Boarding Schools Toward the Industrial Revolution 4.0; Opportunities and Challenges. FIKROTUNA: Jurnal Pendidikan Dan Manajemen Islam, 9(01), Article 01. https://doi.org/10.32806/jf.v11i01.3940
Indrioko, E. (2023). Utilization of Social Media as a Digital Publication Strategy at Nurul Ulum Islamic Boarding School. Al-Hayat: Journal of Islamic Education, 7(1), Article 1. https://doi.org/10.35723/ajie.v7i1.338
Iqbal, A. M. (2020). Challenging moderate Islam in Indonesia: NU Garis Lurus and its construction of the “authentic” NU online. In Rising Islamic Conservatism in Indonesia (1st ed.). Routledge.
Irmi, R. F., Farabi, M. A., & Darlis, A. (2023). Technology Education in the Quran. Solo Universal Journal of Islamic Education and Multiculturalism, 1(01), Article 01. https://doi.org/10.61455/sujiem.v1i01.4
Isnain, A. R., Supriyanto, J., & Kharisma, M. P. (2021). Implementation of K-Nearest Neighbor (K-NN) Algorithm For Public Sentiment Analysis of Online Learning. IJCCS (Indonesian Journal of Computing and Cybernetics Systems), 15(2), Article 2. https://doi.org/10.22146/ijccs.65176
Koto, F., & Rahmaningtyas, G. Y. (2017). Inset lexicon: Evaluation of a word list for Indonesian sentiment analysis in microblogs. 2017 International Conference on Asian Language Processing (IALP), 391–394. https://doi.org/10.1109/IALP.2017.8300625
Kusumadewi, S. (2004). Membangun jaringan syaraf tiruan (menggunakan matlab & excel link). Graha ilmu.
Liu, B. (2022). Sentiment Analysis and Opinion Mining. Springer Nature.
Masters, T. (1993). Practical Neural Network Recipies in C++. Elsevier.
Muhith, A., Dwiyono, Y., Munawati, S., Mustofa, A., & Haryanto, S. (2023). Challenges of Islamic Boarding School Organizational Culture in The Millennial Generation and The Digital Era 4.0. Nazhruna: Jurnal Pendidikan Islam, 6(3), Article 3. https://doi.org/10.31538/nzh.v6i3.4231
NU Online Official. (2024). NU Online: Quran Sholat Tahlil - Aplikasi di Google Play. https://play.google.com/store/apps/details?id=id.or.nu.app&hl=id
Pamuji, A. (2021). Performance of the K-Nearest Neighbors Method on Analysis of Social Media Sentiment. Jurnal Informatika Dan Sistem Informasi, 7(1), Article 1.
Razzaq, A., Asim, M., Ali, Z., Qadri, S., Mumtaz, I., Khan, D. M., & Niaz, Q. (2019). Text sentiment analysis using frequency-based vigorous features. China Communications, 16(12), 145–153. China Communications. https://doi.org/10.23919/JCC.2019.12.011
Siang, J. J. (2009). Jaringan Syaraf Tiruan Dan Pemrograman Menggunakan Matlab. ANDI. https://www.belbuk.com/jaringan-syaraf-tiruan-dan-pemrograman-menggunakan-matlab-p-8134.html
Sutarman, D., Baehaqi, A., Sanusi, A., Wasliman, I., & Gintings, A. (2020). Management Learning of Assyamilah Almaktabah Base on Information and Communication Technology ICT in Pondok Pesantren. IJO - International Journal of Educational Research (ISSN: 2805-413X), 3(02), Article 02.
Thahir, M. (2014). The Role and Function of Islamic Boarding School: An Indonesian Context. TAWARIKH, 5(2), Article 2. https://doi.org/10.2121/tawarikh.v5i2.574
Wankhade, M., Rao, A. C. S., & Kulkarni, C. (2022). A survey on sentiment analysis methods, applications, and challenges. Artificial Intelligence Review, 55(7), 5731–5780. https://doi.org/10.1007/s10462-022-10144-1
Wendemuth, A., Opper, M., & Kinzel, W. (1999). The effect of correlations in neural networks. Journal of Physics A: Mathematical and General, 26(13), 3165. https://doi.org/10.1088/0305-4470/26/13/021
Wikarsa, L., Angdresey, A., & Kapantow, J. (2022). Implementasi Metode Naïve Bayes Dan Lexicon-Based Approach Untuk Mengklasifikasi Sentimen Netizen Pada Tweet Berbahasa Indonesia. Jurnal Ilmiah Realtech, 18(1), Article 1. https://doi.org/10.52159/realtech.v18i1.5
Wongkar, M., & Angdresey, A. (2019). Sentiment Analysis Using Naive Bayes Algorithm Of The Data Crawler: Twitter. 2019 Fourth International Conference on Informatics and Computing (ICIC), 1–5. https://doi.org/10.1109/ICIC47613.2019.8985884
Xie, X., Ge, S., Hu, F., Xie, M., & Jiang, N. (2019). An improved algorithm for sentiment analysis based on maximum entropy. Soft Computing, 23(2), 599–611. https://doi.org/10.1007/s00500-017-2904-0
Zulkarnain, & Tamwifi, I. (2023). Islamic Boarding Schools in the Middle of Digitalization: A Comprehensive Study of the Impact of Information Technology on Teaching and Learning Methods. Remittances Review, 8(4), Article 4.
Abdulla, N. A., Ahmed, N. A., Shehab, M. A., & Al-Ayyoub, M. (2013). Arabic sentiment analysis: Lexicon-based and corpus-based. 2013 IEEE Jordan Conference on Applied Electrical Engineering and Computing Technologies (AEECT), 1–6. https://doi.org/10.1109/AEECT.2013.6716448
Ahmad, M., Aftab, S., Salman, M., & Hameed, N. (2018). Sentiment Analysis using SVM: A Systematic Literature Review. International Journal of Advanced Computer Science and Applications, 9(2), 182–188. https://doi.org/10.14569/IJACSA.2018.090226
Ahmad, M., Aftab, S., Salman, M., Hameed, N., Ali, I., & Nawaz, Z. (2018). SVM Optimization for Sentiment Analysis. International Journal of Advanced Computer Science and Applications, 9(4), 393–398. https://doi.org/10.14569/IJACSA.2018.090455
Amalia, C., & Sibaroni, Y. (2020). Analisis Sentimen Data Tweet Menggunakan Model Jaringan Saraf Tiruan Dengan Pembobotan Delta Tf-idf. eProceedings of Engineering, 7(2), Article 2.
Azhar, Y. (2017). Metode Lexicon-Learning Based Untuk Identifikasi Tweet Opini Berbahasa Indonesia. Jurnal Nasional Pendidikan Teknik Informatika : JANAPATI, 6(3), Article 3. https://doi.org/10.23887/janapati.v6i3.11739
Bajari, A., Wahyudin, U., Koswara, I., & Erlandia, D. (2021). The influence of android-based ‘ISLAM SEHAT’ application for traditional Islamic boarding schools’ (TIBS) students in Indonesia. Kasetsart Journal of Social Sciences, 42, 862–870. https://doi.org/10.34044/j.kjss.2021.42.4.21
Betos, J. Z., David, M. L., Doctor, G. J., Lagan, J., Catacutan-Bangit, A., Fabito, B., & Manuel, R. (2021). NU READY: A Web and Mobile Application Framework for School Emergency Response. TENCON 2021 - 2021 IEEE Region 10 Conference (TENCON), 676–680. https://doi.org/10.1109/TENCON54134.2021.9707439
Borele, P., & Borikar, D. A. (2016). An Approach to Sentiment Analysis using Artificial Neural Network with Comparative Analysis of Different Techniques. IOSR Journal of Computer Engineering (IOSR-JCE), 18(2), 64–69.
Bothma, R. (2023). Factors that influence user retention in mobile apps [Masters]. University of Cape Town.
Budianita, E., & Cynthia, E. P. (2022). Pendekatan berbasis Machine Learning dan Leksikal Pada Analisis Sentimen. Seminar Nasional Teknologi Informasi Komunikasi dan Industri, 0, Article 0.
Cindo, M., Rini, D. P., & Ermatita, E. (2020). Sentiment Analysis On Twitter By Using Maximum Entropy And Support Vector Machine Method. SINERGI, 24(2), 87–94. https://doi.org/10.22441/sinergi.2020.2.002
CNN Indonesia. (2023, March 6). MPR: Mengkhawatirkan, 72 Persen Muslim Indonesia Buta Aksara Al-Quran. nasional. https://www.cnnindonesia.com/nasional/20230306064622-20-921284/mpr-mengkhawatirkan-72-persen-muslim-indonesia-buta-aksara-al-quran
Duric, A., & Song, F. (2012). Feature selection for sentiment analysis based on content and syntax models. Decision Support Systems, 53(4), 704–711. https://doi.org/10.1016/j.dss.2012.05.023
Gunawan, A. L. S. A.-Z., Jondri, J., & Lhaksamana, K. M. (2021). Analisis Sentimen Pada Media Sosial Twitter Terhadap Penanganan Bencana Banjir Di Jawa Barat Dengan Metode Jaringan Saraf Tiruan. eProceedings of Engineering, 8(2), Article 2.
Han, K.-X., Chien, W., Chiu, C.-C., & Cheng, Y.-T. (2020). Application of Support Vector Machine (SVM) in the Sentiment Analysis of Twitter DataSet. Applied Sciences, 10(3), Article 3. https://doi.org/10.3390/app10031125
Haykin, S. (1999). Neural Networks: A Comprehensive Foundation. Prentice Hall.
Hidayatullah, A. F., & Ma’arif, M. R. (2017). Pre-processing Tasks in Indonesian Twitter Messages. Journal of Physics: Conference Series, 801(1), 012072. https://doi.org/10.1088/1742-6596/801/1/012072
Huda, F. (2020). Teachers’ Barriers Of Using Information And Communication Technology (Ict) In Teaching English: A Case Study Of English Teachers At Islamic Boarding School Mu’allimin Muhammadiyah Bangkinang [Thesis, UIN Suska Riau]. https://repository.uin-suska.ac.id/38586/
Ihsan, F. M., Permana, P. Y. E., Arifin, N. R., & Muhtadi, R. (2020). Islamic Boarding Schools Toward the Industrial Revolution 4.0; Opportunities and Challenges. FIKROTUNA: Jurnal Pendidikan Dan Manajemen Islam, 9(01), Article 01. https://doi.org/10.32806/jf.v11i01.3940
Indrioko, E. (2023). Utilization of Social Media as a Digital Publication Strategy at Nurul Ulum Islamic Boarding School. Al-Hayat: Journal of Islamic Education, 7(1), Article 1. https://doi.org/10.35723/ajie.v7i1.338
Iqbal, A. M. (2020). Challenging moderate Islam in Indonesia: NU Garis Lurus and its construction of the “authentic” NU online. In Rising Islamic Conservatism in Indonesia (1st ed.). Routledge.
Irmi, R. F., Farabi, M. A., & Darlis, A. (2023). Technology Education in the Quran. Solo Universal Journal of Islamic Education and Multiculturalism, 1(01), Article 01. https://doi.org/10.61455/sujiem.v1i01.4
Isnain, A. R., Supriyanto, J., & Kharisma, M. P. (2021). Implementation of K-Nearest Neighbor (K-NN) Algorithm For Public Sentiment Analysis of Online Learning. IJCCS (Indonesian Journal of Computing and Cybernetics Systems), 15(2), Article 2. https://doi.org/10.22146/ijccs.65176
Koto, F., & Rahmaningtyas, G. Y. (2017). Inset lexicon: Evaluation of a word list for Indonesian sentiment analysis in microblogs. 2017 International Conference on Asian Language Processing (IALP), 391–394. https://doi.org/10.1109/IALP.2017.8300625
Kusumadewi, S. (2004). Membangun jaringan syaraf tiruan (menggunakan matlab & excel link). Graha ilmu.
Liu, B. (2022). Sentiment Analysis and Opinion Mining. Springer Nature.
Masters, T. (1993). Practical Neural Network Recipies in C++. Elsevier.
Muhith, A., Dwiyono, Y., Munawati, S., Mustofa, A., & Haryanto, S. (2023). Challenges of Islamic Boarding School Organizational Culture in The Millennial Generation and The Digital Era 4.0. Nazhruna: Jurnal Pendidikan Islam, 6(3), Article 3. https://doi.org/10.31538/nzh.v6i3.4231
NU Online Official. (2024). NU Online: Quran Sholat Tahlil - Aplikasi di Google Play. https://play.google.com/store/apps/details?id=id.or.nu.app&hl=id
Pamuji, A. (2021). Performance of the K-Nearest Neighbors Method on Analysis of Social Media Sentiment. Jurnal Informatika Dan Sistem Informasi, 7(1), Article 1.
Razzaq, A., Asim, M., Ali, Z., Qadri, S., Mumtaz, I., Khan, D. M., & Niaz, Q. (2019). Text sentiment analysis using frequency-based vigorous features. China Communications, 16(12), 145–153. China Communications. https://doi.org/10.23919/JCC.2019.12.011
Siang, J. J. (2009). Jaringan Syaraf Tiruan Dan Pemrograman Menggunakan Matlab. ANDI. https://www.belbuk.com/jaringan-syaraf-tiruan-dan-pemrograman-menggunakan-matlab-p-8134.html
Sutarman, D., Baehaqi, A., Sanusi, A., Wasliman, I., & Gintings, A. (2020). Management Learning of Assyamilah Almaktabah Base on Information and Communication Technology ICT in Pondok Pesantren. IJO - International Journal of Educational Research (ISSN: 2805-413X), 3(02), Article 02.
Thahir, M. (2014). The Role and Function of Islamic Boarding School: An Indonesian Context. TAWARIKH, 5(2), Article 2. https://doi.org/10.2121/tawarikh.v5i2.574
Wankhade, M., Rao, A. C. S., & Kulkarni, C. (2022). A survey on sentiment analysis methods, applications, and challenges. Artificial Intelligence Review, 55(7), 5731–5780. https://doi.org/10.1007/s10462-022-10144-1
Wendemuth, A., Opper, M., & Kinzel, W. (1999). The effect of correlations in neural networks. Journal of Physics A: Mathematical and General, 26(13), 3165. https://doi.org/10.1088/0305-4470/26/13/021
Wikarsa, L., Angdresey, A., & Kapantow, J. (2022). Implementasi Metode Naïve Bayes Dan Lexicon-Based Approach Untuk Mengklasifikasi Sentimen Netizen Pada Tweet Berbahasa Indonesia. Jurnal Ilmiah Realtech, 18(1), Article 1. https://doi.org/10.52159/realtech.v18i1.5
Wongkar, M., & Angdresey, A. (2019). Sentiment Analysis Using Naive Bayes Algorithm Of The Data Crawler: Twitter. 2019 Fourth International Conference on Informatics and Computing (ICIC), 1–5. https://doi.org/10.1109/ICIC47613.2019.8985884
Xie, X., Ge, S., Hu, F., Xie, M., & Jiang, N. (2019). An improved algorithm for sentiment analysis based on maximum entropy. Soft Computing, 23(2), 599–611. https://doi.org/10.1007/s00500-017-2904-0
Zulkarnain, & Tamwifi, I. (2023). Islamic Boarding Schools in the Middle of Digitalization: A Comprehensive Study of the Impact of Information Technology on Teaching and Learning Methods. Remittances Review, 8(4), Article 4.
Published
2024-06-30
How to Cite
Lusia, D., Anuraga, G., & Rahman, F. (2024). Sentiment Analysis of NU Online Applications Using Artificial Neural Network. Southeast Asian Journal of Islamic Education, 6(2), 197-208. https://doi.org/10.21093/sajie.v6i2.8822
Section
Articles
