Usability Analysis of Voice-Based Interfaces in Mobile Applications
Keywords:
voice user interface, mobile usability, System Usability Scale,, NASA-TLXAbstract
Voice-based interaction has moved from a peripheral accessibility feature to a mainstream input modality in mobile applications, yet its usability is still evaluated largely with instruments designed for graphical interfaces. This study reports a controlled, within-subjects usability evaluation comparing voice-based and touch-based interaction across six representative mobile tasks performed by 120 participants under two acoustic conditions (quiet and ambient noise). Effectiveness, efficiency, error recovery, perceived usability (System Usability Scale) and perceived workload (NASA-TLX) were measured. Voice interaction achieved a mean task completion rate of 80.0% against 95.2% for touch, but was markedly faster in quiet conditions (24.6 s vs 31.8 s per task). A significant modality × noise interaction indicated that the efficiency advantage of voice collapsed under ambient noise, where completion time rose to 33.8 s and completion fell to 71.3%. Mean SUS for the voice condition was 68.4 compared with 79.6 for touch, and perceived workload was significantly higher for voice. Analysis of 1,152 repair episodes showed that recognition errors accounted for 41.1% of breakdowns, while discoverability and feedback ambiguity together accounted for a further 27.5%. The findings indicate that the usability ceiling of mobile voice interfaces is determined less by raw recognition accuracy than by command discoverability, confirmation design and environmental robustness, and that hybrid multimodal designs offer the most defensible interaction strategy.
References
1. Bangor, A., Kortum, P. T., & Miller, J. T. (2008). An empirical evaluation of the System Usability Scale. International Journal of Human-Computer Interaction, 24(6), 574–594.
2. Brooke, J. (1996). SUS: A "quick and dirty" usability scale. In P. W. Jordan, B. Thomas, B. A. Weerdmeester, & I. L. McClelland (Eds.), Usability Evaluation in Industry (pp. 189–194). Taylor & Francis.
3. Corbett, E., & Weber, A. (2016). What can I say? Addressing user experience challenges of a mobile voice user interface for accessibility. In Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '16) (pp. 72–82). ACM.
4. Cowan, B. R., Pantidi, N., Coyle, D., Morrissey, K., Clarke, P., Al-Shehri, S., Earley, D., & Bandeira, N. (2017). "What can I help you with?": Infrequent users' experiences of intelligent personal assistants. In Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '17) (pp. 1–12). ACM.
5. Furqan, A., Myers, C., & Zhu, J. (2017). Learnability through adaptive discovery tools in voice user interfaces. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA '17) (pp. 1617–1623). ACM.
6. Ghosh, D., Foong, P. S., Zhang, S., & Zhao, S. (2018). Assessing the utility of the System Usability Scale for evaluating voice-based user interfaces. In Proceedings of the Sixth International Symposium of Chinese CHI (ChineseCHI '18) (pp. 11–15). ACM.
7. Hart, S. G., & Staveland, L. E. (1988). Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In P. A. Hancock & N. Meshkati (Eds.), Human Mental Workload (pp. 139–183). North-Holland.
8. Hone, K. S., & Graham, R. (2000). Towards a tool for the subjective assessment of speech system interfaces (SASSI). Natural Language Engineering, 6(3–4), 287–303.
9. Kocaballi, A. B., Laranjo, L., & Coiera, E. (2019). Understanding and measuring user experience in conversational interfaces. Interacting with Computers, 31(2), 192–207.
10. Koenecke, A., Nam, A., Lake, E., Nudell, J., Quartey, M., Mengesha, Z., Toups, C., Rickford, J. R., Jurafsky, D., & Goel, S. (2020). Racial disparities in automated speech recognition. Proceedings of the National Academy of Sciences, 117(14), 7684–7689.
11. Lewis, J. R. (2018). The System Usability Scale: Past, present, and future. International Journal of Human-Computer Interaction, 34(7), 577–590.
12. Luger, E., & Sellen, A. (2016). "Like having a really bad PA": The gulf between user expectation and experience of conversational agents. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16) (pp. 5286–5297). ACM.
13. Murad, C., Munteanu, C., Cowan, B. R., & Clark, L. (2019). Revolution or evolution? Speech interaction and HCI design guidelines. IEEE Pervasive Computing, 18(2), 33–45.
14. Myers, C., Furqan, A., Nebolsky, J., Caro, K., & Zhu, J. (2018). Patterns for how users overcome obstacles in voice user interfaces. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18) (pp. 1–7). ACM.
15. Nielsen, J., & Molich, R. (1990). Heuristic evaluation of user interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '90) (pp. 249–256). ACM.
16. Pal, D., Arpnikanondt, C., Funilkul, S., & Chutimaskul, W. (2020). The adoption analysis of voice-based smart IoT products. IEEE Internet of Things Journal, 7(11), 10852–10867.
17. Porcheron, M., Fischer, J. E., Reeves, S., & Sharples, S. (2018). Voice interfaces in everyday life. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18) (pp. 1–12). ACM.
18. Zwakman, D. S., Pal, D., & Arpnikanondt, C. (2021). Usability evaluation of artificial intelligence-based voice assistants: The case of Amazon Alexa. SN Computer Science, 2(1), Article 28.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 The journal retains copyright of all published articles, ensuring that authors have control over their work while allowing wide dissenmination.
This work is licensed under a Creative Commons Attribution 4.0 International License.
Articles are published under the Creative Commons Attribution 4.0 International License (CC BY 4.0), allowing others to distribute, remix, adapt, and build upon the work for non-commercial purposes while crediting the original author.
