Virtual Drum System Development using Motion Detection
DOI:
https://doi.org/10.52731/liir.v005.263Keywords:
Deep Learning, Motion Recognition, Virtual Drumming, Real-time TrackingAbstract
This paper proposes an innovative approach to overcome the spatial, cost, and mobility constraints of traditional drum playing. By combining the latest deep learning technolo- gies, YOLOv8 and the Pose Landmark model, we have developed a virtual drum system that precisely tracks the user’s movements in real-time, allowing drumming anywhere. This technology significantly enhances musical creativity and accessibility by providing an ex- perience similar to actual drumming without requiring expensive drum equipment. Further- more, this system minimizes geographical and economic constraints in music education and practice, offering educators and learners a flexible and effective way to learn music. This study explores the technical details of the virtual drum system and its potential impact on music education and performance, making a significant contribution to the future of digital music performance tools.
References
Aerodrums, “Aerodrums: Air drums and virtual electronic drum kit,” 2013. [Online]. Available: https://aerodrums.com/
Aeroband, “Pocketdrum: Drum Anywhere, Anytime,” 2016. [Online]. Available: https://www.aeroband.net/
A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep con- volutional neural networks,” Commun. ACM, vol. 60, no. 6, pp. 84–90, 2017.
J. Redmon and A. Angelova, “Real-time grasp detection using convolutional neural networks,” in Proc. 2015 IEEE Int. Conf. Robot. Autom. (ICRA), 2015, pp. 1316–1322.
M. Tan, R. Pang, and Q. V. Le, “Efficientdet: Scalable and efficient object detec- tion,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2020, pp. 10781–10790.
C. T. Tolentino, A. Uy, and P. Naval, “Air drums: Playing drums using computer vi- sion,” in Proc. 2019 Int. Symp. Multimedia Commun. Technol. (ISMAC), 2019, pp. 1–6.
H. Yadid, A. Algranti, M. Levin, and A. Taitler, “A2D: Anywhere Anytime Drumming,” in Proc. 2023 IEEE Region 10 Symp. (TENSYMP), Canberra, Australia, 2023, pp. 1–6, doi: 10.1109/TENSYMP55890.2023.10223631.
D. Reis, J. Kupec, J. Hong, and A. Daoudi, “Real-Time Flying Object Detection with YOLOv8,” 2023. [Online]. Available: https://doi.org/10.48550/arXiv.2305.09972. arXiv:2305.09972 [cs.CV].
I. Grishchenko et al., “BlazePose GHUM Holistic: Real-time 3D Human Landmarks and Pose Estimation,” in Proc. CVPR Workshop Comput. Vis. Augmented Virtual Re- ality, New Orleans, LA, 2022, doi: 10.48550/arXiv.2206.11678. [Online]. Available: https://doi.org/10.48550/arXiv.2206.11678.
F. Ciaglia et al., “Roboflow 100: A Rich, Multi-Domain Object Detec- tion Benchmark,” arXiv:2211.13523 [cs.CV], Nov. 2022. [Online]. Available: https://doi.org/10.48550/arXiv.2211.13523.
A. Bhardwaj, M. Bhardwaj, and A. Gaur, “Virtual Reality: An Overview,” 2016.
Y. Chen et al., “An overview of augmented reality technology,” J. Phys. Conf. Ser., vol. 1237, no. 2, pp. 022082, 2019, doi: 10.1088/1742-6596/1237/2/022082.
A. C. Younkin and P. J. Corriveau, “Determining the Amount of Audio-Video Syn- chronization Errors Perceptible to the Average End-User,” IEEE Trans. Broadcast., vol. 54, no. 3, pp. 623–627, Sept. 2008, doi: 10.1109/TBC.2008.2002102.
