AI-Based Singing Performance Assessment and Intelligent Feedback for Music Education
Keywords:
artificial intelligence; music education; singing performance assessment; intelligent feedback; audio analysis; system architecture; fairness; governance; sustainabilityAbstract
Singing performance assessment has traditionally depended on the subjective judgment of expert instructors, leading to variability across evaluators, institutions, and pedagogical traditions. Recent advances in music information retrieval, audio representation learning, and intelligent educational systems have created new opportunities to support singing pedagogy through computational analysis and automated feedback. This paper presents a system-level examination of AI-based singing performance assessment and intelligent feedback for music education. It focuses on architectural foundations, data infrastructure, acoustic and musical feature modeling, assessment model design, feedback generation, fairness, robustness, governance, deployment, sustainability, and institutional adoption. The discussion emphasizes structural trade-offs among interpretability, predictive accuracy, latency, privacy, and pedagogical validity. A central concern is the need to combine signal-level analysis with meaningful educational scaffolding rather than treating automated scoring as a replacement for human instruction. The paper further examines how large annotated singing corpora, self-supervised audio representations, and modular assessment pipelines can support more consistent and scalable feedback, while also introducing risks related to bias, cultural diversity, vocal health, and data governance. By integrating perspectives from music information retrieval, educational technology, human-computer interaction, and sociotechnical systems research, the article develops a critical framework for designing, deploying, and regulating intelligent singing assessment tools. It argues that sustainable adoption requires not only technical accuracy but also institutional trust, teacher involvement, transparent evaluation protocols, and alignment with established music education values. Future directions include multimodal analysis, longitudinal learner modeling, privacy-preserving federated learning, and the development of open benchmarking practices that reflect diverse musical traditions.
References
1. Schedl, M., Gómez, E., & Urbano, J. (2014). Music information retrieval: Recent developments and applications. Foundations and Trends in Information Retrieval, 8(2–3), 127–261. https://doi.org/10.1561/1500000029
2. Lerch, A., Arthur, C., Pati, A., & Gururani, S. (2020). An interdisciplinary review of music performance analysis. Transactions of the International Society for Music Information Retrieval, 3(1), 221–245. https://doi.org/10.5334/tismir.53
3. Turchet, L., Fischione, C., Essl, G., Keller, D., & Barthet, M. (2018). Internet of musical things: Vision and challenges. IEEE Access, 6, 61994–62017. https://doi.org/10.1109/ACCESS.2018.2872625
4. Raffel, C., McFee, B., Humphrey, E. J., Salamon, J., Nieto, O., Liang, D., & Ellis, D. P. W. (2014). mir_eval: A transparent implementation of common MIR metrics. In Proceedings of the 15th International Society for Music Information Retrieval Conference (pp. 367–372).
5. Bittner, R. M., Salamon, J., Tierney, M., Mauch, M., Cannam, C., & Bello, J. P. (2014). MedleyDB: A multitrack dataset for annotation-intensive MIR research. In Proceedings of the 15th International Society for Music Information Retrieval Conference (pp. 155–160).
6. Zhang, Y., Pan, C., Guo, W., Li, R., Zhu, Z., Wang, J., ... & Zhao, Z. (2024). Gtsinger: A global multi-technique singing corpus with realistic music scores for all singing tasks. Advances in Neural Information Processing Systems, 37, 1117-1140.
7. Meseguer-Brocal, G., Cohen-Hadria, A., & Peeters, G. (2018). DALI: A large dataset of synchronized audio, lyrics and notes. In Proceedings of the 19th International Society for Music Information Retrieval Conference (pp. 653–660).
8. Bittner, R. M., Pasalo, K., Bosch, J. J., Meseguer-Brocal, G., & Rubinstein, D. (2022). Vocadito: A dataset of solo vocals with pitch, f0, and lyrics. In Proceedings of the 23rd International Society for Music Information Retrieval Conference.
9. Nakano, T., & Goto, M. (2012). VocaListener: A singing-to-singing synthesis system based on iterative parameter estimation. In Proceedings of the 9th Sound and Music Computing Conference (pp. 127–132).
10. Hsu, W.-N., Bolte, B., Tsai, Y.-H. H., Lakhotia, K., Salakhutdinov, R., & Mohamed, A. (2021). HuBERT: Self-supervised speech representation learning by masked prediction of hidden units. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 29, 3451–3460. https://doi.org/10.1109/TASLP.2021.3122291
11. Baevski, A., Zhou, Y., Mohamed, A., & Auli, M. (2020). wav2vec 2.0: A framework for self-supervised learning of speech representations. Advances in Neural Information Processing Systems, 33, 12449–12460.
12. Choi, K., Fazekas, G., Sandler, M., & Cho, K. (2017). Convolutional recurrent neural networks for music classification. In 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp. 2392–2396). https://doi.org/10.1109/ICASSP.2017.7952585
13. Pons, J., & Serra, X. (2019). musicnn: Pre-trained convolutional neural networks for music audio tagging. In Proceedings of the 20th International Society for Music Information Retrieval Conference (Late-Breaking Demo).
14. Amershi, S., Weld, D., Vorvoreanu, M., Fourney, A., Nushi, B., Collisson, P., ... & Horvitz, E. (2019). Guidelines for human-AI interaction. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (pp. 1–13). https://doi.org/10.1145/3290605.3300233
15. Luckin, R., Holmes, W., Griffiths, M., & Forcier, L. B. (2016). Intelligence unleashed: An argument for AI in education. Pearson.
16. Holmes, W., Bialik, M., & Fadel, C. (2019). Artificial intelligence in education: Promises and implications for teaching and learning. Center for Curriculum Redesign.
17. Williamson, B., Eynon, R., & Potter, J. (2020). Pandemic politics, pedagogies and practices: Digital technologies and distance education during the coronavirus emergency. Learning, Media and Technology, 45(2), 107–114. https://doi.org/10.1080/17439884.2020.1761641
18. Bauer, W. I. (2020). Music learning today: Digital pedagogy for creating, performing, and responding to music (2nd ed.). Oxford University Press.
19. Duan, Z., Essid, S., Liem, C. C. S., Richard, G., & Sharma, G. (2019). Audiovisual analysis of music performances: Overview and applications. IEEE Signal Processing Magazine, 36(1), 63–73. https://doi.org/10.1109/MSP.2018.2875511
20. Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2021). A survey on bias and fairness in machine learning. ACM Computing Surveys, 54(6), Article 115. https://doi.org/10.1145/3457607
21. Selbst, A. D., Boyd, D., Friedler, S. A., Venkatasubramanian, S., & Vertesi, J. (2019). Fairness and abstraction in sociotechnical systems. In Proceedings of the Conference on Fairness, Accountability, and Transparency (pp. 59–68). https://doi.org/10.1145/3287560.3287598
22. Holstein, K., Wortman Vaughan, J., Daumé III, H., Dudík, M., & Wallach, H. (2019). Improving fairness in machine learning systems: What do industry practitioners need? In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (pp. 1–16). https://doi.org/10.1145/3290605.3300830
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Journal of Engineering Systems and Digital Innovation
This work is licensed under a Creative Commons Attribution 4.0 International License.
