Vol. 12 No. 8 (2026)
Open Access
Peer Reviewed

Remote Access Trojan Malware Detection Analysis on Android Operating System using Reverse Engineering Method

Authors

Dista Nahda , Vera Suryani , Erwid M Jadied

DOI:

10.29303/jppipa.v12i8.6334

Published:

2026-08-31

Downloads

Abstract

This study detected malware on the Android operating system using a system called (syscall). Remote Access Trojan Malware Analysis on the Android operating system using Reverse Engineering Methods helps discover the technological principles of a device, object, or system by analyzing its structure, functions, and operation. This method employed dynamic analysis, namely running every malware on the Android operating system to get information on the system call (syscall) that is running. The results of the system call (syscall) information were selected using the Machine Learning selection feature using the Random Forest method. The purpose of this study is to determine the characteristics based on the system call (syscall) and the detection system's accuracy for the characteristics of Remote Access Trojan malware on the Android operating system. The average accuracy results in this study with four scenarios using the Random Forest method is 96%, and the f1-score value is 94%. This accuracy value is quite good and can be implemented in detecting remote access Trojan malware based on system call (syscall) on the Android Operating System

Keywords:

Android Operating System Malicious Software Random Forest Remote Access Trojan Reverse Engineering

References

Adekotujo, A., Odumabo, A., Adedokun, A., & Aiyeniko, O. (2020). A Comparative Study of Operating Systems: Case of Windows, UNIX, Linux, Mac, Android and iOS. International Journal of Computer Applications, 176(39). https://doi.org/10.5120/ijca2020920494

Alimardani, H., & Nazeh, M. (2018). A taxonomy on recent mobile malware: Features, analysis methods, and detection techniques. ACM International Conference Proceeding Series, 44–49. https://doi.org/10.1145/3230467.3230478

Anupama, M. L., Vinod, P., Visaggio, C. A., Arya, M. A., Philomina, J., Raphael, R., … Mathiyalagan, P. (2022). Detection and robustness evaluation of android malware classifiers. Journal of Computer Virology and Hacking Techniques, 18(3), 147–170. https://doi.org/10.1007/s11416-021-00390-2

Aprilliansyah, D., Riadi, I., & Sunardi. (2022). Analysis of Remote Access Trojan Attack using Android Debug Bridge. IJID (International Journal on Informatics for Development), 10(2), 102–111. https://doi.org/10.14421/ijid.2021.2839

Asher, S. W., Jan, S., Tsaramirsis, G., Khan, F. Q., Khalil, A., & Obaidullah, M. (2021). Reverse engineering of mobile banking applications. Computer Systems Science and Engineering, 38(3). https://doi.org/10.32604/CSSE.2021.016787

Aslan, O., & Samet, R. (2020). A Comprehensive Review on Malware Detection Approaches. IEEE Access, Vol. 8. https://doi.org/10.1109/ACCESS.2019.2963724

Banerjee, P. (2019). Random Forest Classifier Tutorial | Kaggle. Retrieved from https://www.kaggle.com/prashant111/random-forest-classifier-tutorial

Bhatia, T., & Kaushal, R. (2017). Malware detection in android based on dynamic analysis. 2017 International Conference on Cyber Security And Protection Of Digital Services, Cyber Security 2017. https://doi.org/10.1109/CyberSecPODS.2017.8074847

Castillo-Zúñiga, I., Luna-Rosas, F. J., Rodríguez-Martínez, L. C., Muñoz-Arteaga, J., López-Veyna, J. I., & Rodríguez-Díaz, M. A. (2020). Internet data analysis methodology for cyberterrorism vocabulary detection, combining techniques of big data analytics, NLP and semantic web. International Journal on Semantic Web and Information Systems, 16(1), 69–86. https://doi.org/10.4018/IJSWIS.2020010104

Christodorescu, M., & Jha, S. (2003). Static analysis of executables to detect malicious patterns. Proceedings of the 12th USENIX Security Symposium, 169–186.

Dennis, M. A. (2019). cybercrime | Definition, Statistics, & Examples | Britannica.

Donalds, C., Barclay, C., & Osei-Bryson, K.-M. (2022). Towards a Cybercrime Classification Ontology. In Cybercrime and Cybersecurity in the Global South. https://doi.org/10.1201/9781003028710-15

Eom, T., Kim, H., An, S. M., Park, J. S., & Kim, D. S. (2018). Android malware detection using feature selections and random forest. Proceedings - 2018 4th International Conference on Software Security and Assurance, ICSSA 2018, 55–61. https://doi.org/10.1109/ICSSA45270.2018.00023

Gibert, D., Mateu, C., & Planes, J. (2020). The rise of machine learning for detection and classification of malware: Research developments, trends and challenges. Journal of Network and Computer Applications, Vol. 153. https://doi.org/10.1016/j.jnca.2019.102526

Hemalatha, J., Roseline, S. A., Geetha, S., Kadry, S., & Damaševičius, R. (2021). An efficient densenet‐based deep learning model for Malware detection. Entropy, 23(3). https://doi.org/10.3390/e23030344

Isohara, T., Takemori, K., & Kubota, A. (2011). Kernel-based behavior analysis for android malware detection. Proceedings - 2011 7th International Conference on Computational Intelligence and Security, CIS 2011, 1011–1015. https://doi.org/10.1109/CIS.2011.226

Joseph, A. E. (2017). Cybercrime definition. Computer Crime Research Center, (June 2017).

Kunang, Y. N. K. Y. N., & ... (2022). Analisis Forensik Malware Pada Platform Android. Analisis Forensik …. Retrieved from http://eprints.binadarma.ac.id/10591/%0Ahttp://eprints.binadarma.ac.id/10591/1/yesi novaria kunang_analisis forensik malware android_ubd.2.pdf

Li, H., Zhou, S., Yuan, W., Li, J., & Leung, H. (2020). Adversarial-Example Attacks Toward Android Malware Detection System. IEEE Systems Journal, 14(1). https://doi.org/10.1109/JSYST.2019.2906120

Liu, X., Du, X., Zhang, X., Zhu, Q., Wang, H., & Guizani, M. (2019). Adversarial samples on android malware detection systems for IoT systems. Sensors (Switzerland), 19(4). https://doi.org/10.3390/s19040974

M. Aranitasi, A. Daci, and A. G. (2022). Mobile malware detection techniques using system calls. International Journal of Engineering Research and Applications Www.Ijera.Com, 12, 54–57.

Mat, S. R. T., Razak, M. F. A., Kahar, M. N. M., Arif, J. M., & Firdaus, A. (2022). A Bayesian probability model for Android malware detection. ICT Express, 8(3). https://doi.org/10.1016/j.icte.2021.09.003

Pan, Y., Ge, X., Fang, C., & Fan, Y. (2020). A Systematic Literature Review of Android Malware Detection Using Static Analysis. IEEE Access, 8. https://doi.org/10.1109/ACCESS.2020.3002842

Qamar, A., Karim, A., & Chang, V. (2019). Mobile malware attacks: Review, taxonomy & future directions. Future Generation Computer Systems, 97, 887–909. https://doi.org/10.1016/j.future.2019.03.007

Qiu, J., Zhang, J., Luo, W., Pan, L., Nepal, S., & Xiang, Y. (2021). A Survey of Android Malware Detection with Deep Neural Models. ACM Computing Surveys, Vol. 53. https://doi.org/10.1145/3417978

Rashmitha, B., Alwina, J., Angelin, B., & Ramesh, E. R. (n.d.). Malware analysis and detection using reverse Engineering. International Journal of Computer Science and Information Technology Research, 10(4). Retrieved from www.researchpublish.com,

Roseline, S. A., Geetha, S., Kadry, S., & Nam, Y. (2020). Intelligent Vision-Based Malware Detection and Classification Using Deep Random Forest Paradigm. IEEE Access, 8. https://doi.org/10.1109/ACCESS.2020.3036491

Scikit-learn. (2022). sklearn ensemble Random Forest Classifier. Scikit-Learn. Retrieved from 11/13/22, 9:13 AMsklearn.ensemble.RandomForestClassifier — scikit-learn 1.1.3 documentationhttps://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html#sklearn.ensemble.RandomForestClassifier

Setia, T. P., Aldya, A. P., & Widiyasono, N. (2019). Reverse Engineering untuk Analisis Malware Remote Access Trojan. Jurnal Edukasi Dan Penelitian Informatika (JEPIN), 5(1). https://doi.org/10.26418/jp.v5i1.28214

Shakya, S., & Dave, M. (2022). Analysis, Detection, and Classification of Android Malware using System Calls. Retrieved from https://arxiv.org/abs/2208.06130v1%0Ahttps://arxiv.org/ftp/arxiv/papers/2208/2208.06130.pdf

Sharma, M. (2019). A Study on RAT (Remote Access Trojan). Academic Journal of Forensic Sciences, 02(02).

Shhadat, I., Bataineh, B., Hayajneh, A., & Al-Sharif, Z. A. (2020). The Use of Machine Learning Techniques to Advance the Detection and Classification of Unknown Malware. Procedia Computer Science, 170, 917–922. https://doi.org/10.1016/j.procs.2020.03.110

Strace - Trace System Calls and Signals. (2021). Retrieved January 25, 2023, from http://linux.die.net/man/1/strace

Techopedia. (2018). What is Cybercrime? - Definition from Techopedia.

Urooj, B., Shah, M. A., Maple, C., Abbasi, M. K., & Riasat, S. (2022). Malware Detection: A Framework for Reverse Engineered Android Applications Through Machine Learning Algorithms. IEEE Access, 10, 89031–89050. https://doi.org/10.1109/ACCESS.2022.3149053

Uttarwar, P. S., Tidke, R. P., Dandwate, D. S., & Tupe, U. J. (2021). A Literature Review on Android-A Mobile Operating system. International Research Journal of Engineering and Technology, (September).

Zhang, Y. (2003). Research into the engineering application of reverse engineering technology. Journal of Materials Processing Technology, 139(1-3 SPEC). https://doi.org/10.1016/S0924-0136(03)00513-2

Zhou, Y., & Jiang, X. (2012). Dissecting Android malware: Characterization and evolution. Proceedings - IEEE Symposium on Security and Privacy, 95–109. https://doi.org/10.1109/SP.2012.16

Zhou, Z., Chen, D., & Xie, S. (Shengquan). (2007). Springer Series in Advanced Manufacturing. Thermoplastics and Thermoplastic Composites, 863–866. Retrieved from http://www.springer.com/series/7113%0Ahttp://linkinghub.elsevier.com/retrieve/pii/B9781856174787500118

Author Biographies

Dista Nahda, Telkom University

Author Origin : Indonesia

Vera Suryani, Telkom University

Author Origin : Indonesia

Telkom University, Indonesia

Erwid M Jadied, Telkom University

Author Origin : Indonesia

Telkom University, Indonesia

Downloads

Download data is not yet available.

How to Cite

Nahda, D., Suryani, V., & Jadied, E. M. (2026). Remote Access Trojan Malware Detection Analysis on Android Operating System using Reverse Engineering Method. Jurnal Penelitian Pendidikan IPA, 12(8), 243–251. https://doi.org/10.29303/jppipa.v12i8.6334