YOLO Algorithm-based Visitor Detection System for Small Retail Stores using Single Board Computer
DOI:
https://doi.org/10.37385/jaets.v4i2.1872Keywords:
Monitoring System, Small Retail Store, Raspberry Pi, YOLOAbstract
In Indonesia, assistance for small enterprises has grown in recent years. However, a monitoring system is required to support these enterprises and ensure their expansion and survival. Using a single-board computer and the YOLO algorithm, we construct a visitor tracking system in this study to meet this demand. To capture objects and categorize them as human or non-human, we employ the YOLOv4-tiny model, which has a mAP of 89.21%. Human visitors are welcomed with the use of a speaker. A telegraph bot that notifies the owner of the retail establishment of the visitor's presence also makes the presumption as to whether the visitor is a potential customer or an intruder. Our research demonstrates that the created monitoring system effectively recognizes and categorizes visits, enabling retail store owners to make defensible choices regarding visitor interaction and security precautions. Small business owners can save personnel costs while still maintaining high levels of client engagement and security. The theoretical application of this research is the creation of a visitor monitoring system that is affordable and may be used in small enterprises, particularly in Indonesia. The practical ramifications of our research include the possibility for small retail business owners to boost profits by lowering labor expenses while raising customer satisfaction and security. The importance of our study lies in its role in creating a monitoring system that will support small enterprises and increase their sustainability.
Downloads
References
Aravamuthan, G., Rajasekhar, P., Verma, R. K., Shrikhande, S. V, Kar, S., & Babu, S. (2020). Physical intrusion detection system using stereo video analytics. Proceedings of 3rd International Conference on Computer Vision and Image Processing: CVIP 2018, Volume 2, 173–182. https://doi.org/10.1007/978-981-32-9291-8_15
Arun, M., Baraneetharan, E., Kanchana, A., & Prabu, S. (2020). Detection and monitoring of the asymptotic COVID-19 patients using IoT devices and sensors. International Journal of Pervasive Computing and Communications, 18(4), 407–418. https://doi.org/10.1108/IJPCC-08-2020-0107
Babu, S., Pragathi, B. S., Chinthala, U., & Maheshwaram, S. (2020). Subject Tracking with Camera Movement Using Single Board Computer. 2020 IEEE-HYDCON, 1–6. https://doi.org/10.1109/HYDCON48903.2020.9242811
Berg, S., Kutra, D., Kroeger, T., Straehle, C. N., Kausler, B. X., Haubold, C., Schiegg, M., Ales, J., Beier, T., & Rudy, M. (2019). Ilastik: interactive machine learning for (bio) image analysis. Nature Methods, 16(12), 1226–1232. https://doi.org/10.1038/s41592-019-0582-9
Bhambani, K., Jain, T., & Sultanpure, K. A. (2020). Real-time face mask and social distancing violation detection system using yolo. 2020 IEEE Bangalore Humanitarian Technology Conference (B-HTC), 1–6. DOI: 10.1109/B-HTC50970.2020.9297902
Caggiano, A., Zhang, J., Alfieri, V., Caiazzo, F., Gao, R., & Teti, R. (2019). Machine learning-based image processing for on-line defect recognition in additive manufacturing. CIRP Annals, 68(1), 451–454. https://doi.org/10.1016/j.cirp.2019.03.021
Cermeño, E., Pérez, A., & Sigüenza, J. A. (2018). Intelligent video surveillance beyond robust background modeling. Expert Systems with Applications, 91, 138–149. https://doi.org/10.1016/j.eswa.2017.08.052
Feng, J.-C., Hong, F.-T., & Zheng, W.-S. (2021). Mist: Multiple instance self-training framework for video anomaly detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 14009–14018. DOI: 10.1109/CVPR46437.2021.01379
Generosi, A., Ceccacci, S., & Mengoni, M. (2018). A deep learning-based system to track and analyze customer behavior in retail store. 2018 IEEE 8th International Conference on Consumer Electronics-Berlin (ICCE-Berlin), 1–6. https://doi.org/10.1109/ICCE-Berlin.2018.8576169
Gollapudi, S., & Gollapudi, S. (2019). OpenCV with Python. Learn Computer Vision Using OpenCV: With Deep Learning CNNs and RNNs, 31–50. https://doi.org/10.1007/978-1-4842-4261-2_3
Guha, S., Chakrabarti, A., Biswas, S., & Banerjee, S. (2020). Implementation of Face Recognition Algorithm on a Mobile Single Board Computer for IoT Applications. 2020 IEEE 17th India Council International Conference (INDICON), 1–5. https://doi.org/10.1109/INDICON49873.2020.9342290
Hashemzehi, R., Mahdavi, S. J. S., Kheirabadi, M., & Kamel, S. R. (2020). Detection of brain tumors from MRI images base on deep learning using hybrid model CNN and NADE. Biocybernetics and Biomedical Engineering, 40(3), 1225–1232. https://doi.org/10.1016/j.bbe.2020.06.001
Hermawan, M. S., & Nugraha, U. (2022). The development of Small-Medium Enterprises (SMEs) and the role of digital ecosystems during the COVID-19 pandemic: A case of Indonesia. In Handbook of Research on Current Trends in Asian Economics, Business, and Administration (pp. 123–147). IGI Global. DOI: 10.4018/978-1-7998-8486-6.ch007
Jadon, A., Omama, M., Varshney, A., Ansari, M. S., & Sharma, R. (2019). FireNet: a specialized lightweight fire & smoke detection model for real-time IoT applications. ArXiv Preprint ArXiv:1905.11922. https://doi.org/10.48550/arXiv.1905.11922
Jafriz, I. Z., & Mansor, S. (2022). Smart Retail Monitoring System using Intel OpenVINO Toolkit. International Journal of Technology, 13(6), 1241. https://doi.org/10.14716/ijtech.v13i6.5872
Jaihar, J., Lingayat, N., Vijaybhai, P. S., Venkatesh, G., & Upla, K. P. (2020). Smart home automation using machine learning algorithms. 2020 International Conference for Emerging Technology (INCET), 1–4. DOI: https://doi.org/10.1109/INCET49848.2020.9154007
Jiang, Y., Pang, D., & Li, C. (2021). A deep learning approach for fast detection and classification of concrete damage. Automation in Construction, 128, 103785. https://doi.org/10.1016/j.autcon.2021.103785
Kim, J., Min, K., Jung, M., & Chi, S. (2020). Occupant behavior monitoring and emergency event detection in single-person households using deep learning-based sound recognition. Building and Environment, 181, 107092. https://doi.org/10.1016/j.buildenv.2020.107092
Kim, S.-Y., Kim, M., & Ho, Y.-S. (2013). Depth image filter for mixed and noisy pixel removal in RGB-D camera systems. IEEE Transactions on Consumer Electronics, 59(3), 681–689. DOI: https://doi.org/10.1109/TCE.2013.6626256
Korgaonkar, P., Becerra, E. P., Mangleburg, T., & Bilgihan, A. (2021). Retail employee theft: When retail security alone is not enough. Psychology & Marketing, 38(5), 721–734. https://doi.org/10.1002/mar.21460
Li, S., Liu, F., & Jiao, L. (2022). Self-training multi-sequence learning with transformer for weakly supervised video anomaly detection. Proceedings of the AAAI Conference on Artificial Intelligence, 36(2), 1395–1403. https://doi.org/10.1016/j.ssci.2015.01.013
Lohani, D., Crispim-Junior, C., Barthélemy, Q., Bertrand, S., Robinault, L., & Tougne, L. (2021). Spatio-temporal convolutional autoencoders for perimeter intrusion detection. Reproducible Research in Pattern Recognition: Third International Workshop, RRPR 2021, Virtual Event, January 11, 2021, Revised Selected Papers, 47–65. https://doi.org/10.1007/978-3-030-76423-4_4
Ma, X., Niu, Y., Gu, L., Wang, Y., Zhao, Y., Bailey, J., & Lu, F. (2021). Understanding adversarial attacks on deep learning based medical image analysis systems. Pattern Recognition, 110, 107332. https://doi.org/10.1016/j.patcog.2020.107332
Maksum, I. R., Rahayu, A. Y. S., & Kusumawardhani, D. (2020). A social enterprise approach to empowering micro, small and medium enterprises (SMEs) in Indonesia. Journal of Open Innovation: Technology, Market, and Complexity, 6(3), 50. https://doi.org/10.3390/joitmc6030050
Matern, D., Condurache, A. P., & Mertins, A. (2013). Automated Intrusion Detection for Video Surveillance Using Conditional Random Fields. MVA, 298–301.
Mathur, S., Subramanian, B., Jain, S., Choudhary, K., & Prabha, D. R. (2017). Human detector and counter using raspberry Pi microcontroller. 2017 Innovations in Power and Advanced Computing Technologies (i-PACT), 1–7. DOI: 10.1109/IPACT.2017.8244984
Milella, A., Marani, R., Petitti, A., Cicirelli, G., & D’Orazio, T. (2021). 3d vision-based shelf monitoring system for intelligent retail. Pattern Recognition. ICPR International Workshops and Challenges: Virtual Event, January 10–15, 2021, Proceedings, Part II, 447–459. https://doi.org/10.1007/978-3-030-68790-8_35
Moen, E., Bannon, D., Kudo, T., Graf, W., Covert, M., & Van Valen, D. (2019). Deep learning for cellular image analysis. Nature Methods, 16(12), 1233–1246. https://doi.org/10.1038/s41592-019-0403-1
Nayak, R., Pati, U. C., & Das, S. K. (2021). A comprehensive review on deep learning-based methods for video anomaly detection. Image and Vision Computing, 106, 104078. https://doi.org/10.1016/j.imavis.2020.104078
Priyanka, J. S., Kiran, M. S., & Nalla, P. (2022). A Secured IoT-Based Health Care Monitoring System Using Body Sensor Network. In Emergent Converging Technologies and Biomedical Systems: Select Proceedings of ETBS 2021 (pp. 483–490). Springer. https://doi.org/10.1007/978-981-16-8774-7_39
Protopapadakis, E., Voulodimos, A., Doulamis, A., Doulamis, N., & Stathaki, T. (2019). Automatic crack detection for tunnel inspection using deep learning and heuristic image post-processing. Applied Intelligence, 49, 2793–2806. https://doi.org/10.1007/s10489-018-01396-y
Raharja, S. J., Kostini, N., Muhyi, H. A., & Rivani. (2019). Utilisation analysis and increasing strategy: e-commerce use of SMEs in Bandung, Indonesia. International Journal of Trade and Global Markets, 12(3–4), 287–299. https://doi.org/10.1504/IJTGM.2019.101557
Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2020). You only look once: Unified, real-time object detection. arXiv 2015. ArXiv Preprint ArXiv:1506.02640. https://doi.org/10.1109/CVPR.2016.91
Scime, L., & Beuth, J. (2019). Using machine learning to identify in-situ melt pool signatures indicative of flaw formation in a laser powder bed fusion additive manufacturing process. Additive Manufacturing, 25, 151–165. . https://doi.org/10.1016/j.addma.2018.11.010
Shao, L., Han, J., Kohli, P., & Zhang, Z. (2014). Computer vision and machine learning with RGB-D sensors (Vol. 20). Springer. https://doi.org/10.1007/978-3-319-08651-4
Sharma, P., Berwal, Y. P. S., & Ghai, W. (2020). Performance analysis of deep learning CNN models for disease detection in plants using image segmentation. Information Processing in Agriculture, 7(4), 566–574. https://doi.org/10.1016/j.inpa.2019.11.001
Sim, H. S., Kim, H. I., & Ahn, J. J. (2019). Is deep learning for image recognition applicable to stock market prediction? Complexity, 2019. https://doi.org/10.1155/2019/4324878
Sultani, W., Chen, C., & Shah, M. (2018). Real-world anomaly detection in surveillance videos. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 6479–6488. DOI: https://doi.org/10.1109/CVPR.2018.00678
Ullah, M. B. (2020). CPU based YOLO: A real time object detection algorithm. 2020 IEEE Region 10 Symposium (TENSYMP), 552–555. DOI: 10.1109/TENSYMP50017.2020.9230778
Valikhani, A., Jaberi Jahromi, A., Pouyanfar, S., Mantawy, I. M., & Azizinamini, A. (2021). Machine learning and image processing approaches for estimating concrete surface roughness using basic cameras. Computer?Aided Civil and Infrastructure Engineering, 36(2), 213–226. https://doi.org/10.1111/mice.12605
Vijverberg, J. A., Janssen, R. T. M., de Zwart, R., & de With, P. H. N. (2014). Perimeter-intrusion event classification for on-line detection using multiple instance learning solving temporal ambiguities. 2014 IEEE International Conference on Image Processing (ICIP), 2408–2412. DOI: https://doi.org/10.1109/ICIP.2014.7025487
Villamizar, M., Martínez-González, A., Canévet, O., & Odobez, J.-M. (2018). Watchnet: Efficient and depth-based network for people detection in video surveillance systems. 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), 1–6. DOI: https://doi.org/10.1109/AVSS.2018.8639165
Wang, C., Tan, X. P., Tor, S. B., & Lim, C. S. (2020). Machine learning in additive manufacturing: State-of-the-art and perspectives. Additive Manufacturing, 36, 101538. https://doi.org/10.1016/j.addma.2020.101538
Zhang, Y.-L., Zhang, Z.-Q., Xiao, G., Wang, R.-D., & He, X. (2015). Perimeter intrusion detection based on intelligent video analysis. 2015 15th International Conference on Control, Automation and Systems (ICCAS), 1199–1204. DOI: https://doi.org/10.1109/ICCAS.2015.7364811
CITEDNESS IN SCOPUS
CITEDNESS IN WOS
