This project was a speculative UX case study to explore AI/ML opportunities in consumer mobile applications, focusing on real-world applicability and user-centered AI system design.
This project was developed as part of the Human Factors in AI course on Coursera, focusing on applying user-centered design principles to AI-driven applications.
Role: UX Designer
Type of Project: Personal Project
Duration: 2 Weeks
Purpose: Online Certification Assignment
Disclaimer: This feature has already been launched by Certinia. All data and name included in the prototype are entirely fictional and intended solely for demonstration purposes. For more information and latest updates, please read Certinia official release notes.
Problem statement：Singing beginner encounter difficulties in learning music theory and sing perfect pitch during their practice
Objective: Encourage music beginner to participant more music activities with more confident and allow them to learn music in a easy and efficient way through self learning
The target user of this application is for music beginner who have no or little music background, and they want to learn how to sing.
The above diagram demonstrate the user journey of learner learning to sing a new song. The user challenges and opportunities I identified in the user journey included:
Requires high manual effort in identifying sofa syllables and correct pitch for the new music. With using AI, it can help users to understand the melody of the new music effectively.
Also, beginner can easily make mistake and those mistakes will influence their confident in learning music, therefore a tools with AI to assess learners’ pitch, rhythm and vocal and give recommendation can speed up learner learning progress
Regarding the research, I believe AI singing mentor application can help music beginner learn new music more efficiently by reducing 20% of learning time.
Referencing previous task analysis, user journey for the application is designed. The AI occurs while the application generate sofa syllables from the images using image processor, also evaluating learner performance and giving suggestion for improvement.
To ensure model transparency, the application uses interpretable model. The performance score is determined on pitch, tempo and vocal. The determination of tempo and pitch score are based on non AI technology like turner and BPM, and determination of the vocal score is based on AI. To train the data model, singing audio from different level of singer are collected, and professional singing teacher are asked to score the performance. And the system will find the pattern and trend of the score by vocal characteristics.
Besides of the end to end user journey, feedback loop and fallback UX flow are designed to accompany different situation and AI technologies constraints:
Sometime the app may not give the right score and accurate sofa syllables, as there are chance that the chord and tempo are not included in the database. Regarding this scenarios, it will give an confidence score, so the users can determine whether they want to follow all the syllables presented. We also rely on user feedback to determine whether the generated sofa syllables have any error.
This app relies heavily on sound detection technology, making it sensitive to environmental noise, which can affect recognition accuracy. In addition to using a confidence score to assess result reliability, the system monitors for high ambient noise levels and the presence of secondary human voices, prompting a reminder to users to ensure optimal recording conditions.
Understand how easily users can complete singing training tasks and perception of AI generated score
Task-based testing using a 'Wizard of Oz' prototype, where users are asked to complete the end to end learning journey to evaluate usability. Also, we will simulate situation where the app cannot provide accurate result to evaluate how users perceive the fallout scenarios. The research mix with both quantitive and qualitative data, where success rate is measured and user feedbacks are summarised.
Users with vary singing skill and age group will be equally represented in the group of paricipants.
Regarding all consideration, here are the prototype of the application which demonstrate the user experience of learning new song. To maintain transparency, there is a disclaimer (by clicking the i icon) stated that sofa syllables are generated by AI. User can learn more about the data model and they can report any error through provided link.
Same with performance score, there is a disclaimer stated that the score evaluated by AI. User can learn more about the data model and they can report any error through provided link. Also, contributed factors to the performance score are clearly stated, and suggestion of improving the score is provided.
I also did a presentation for the project, please feel free to watch it:
Concern: This app may collect users individual‘s acoustic fingerprint which is directly identifiable and may cause concern with cyber security.
Privacy Law: As the application is an online service and analyse user data, it will be regulated by the privacy laws of each country that the application provides service for, including GDPR. Although this application won’t receive any federal funding, the application may be utilised in public educational section, it will follow educational privacy law, such as it won’t disclose any user data to third party and providing consent during onboarding.
Representation bias: Data collection may introduce representation bias due to the limited diversity of volunteers and singing tutors in terms of gender, ethnicity, musical background, and vocal style. Tutors may favor certain vocal characteristics or genres, and if the training data lacks variety, specific vocal types could be underrepresented in the model.
Method to mitigate bias: 1. Recruit diverse singing tutor and audio sample. 2. Selective of audio will be re evaluated by human singing tutors
Fairness: The intended purpose of the application is only used for teaching users how to sing, therefore it should not cause any fairness issues. There will be a disclaimer in the consent to state the designed purpose of the model and warn users it may caused unfairness if it is used for other purposes, such as admission for music school, auditioning and more.
Transparency: Explaining the attributes that contribute to the score and insight how user can improve their performance
Accountability: Provide a way for users to report any bias and unfair use.
As this is a self learning project, there is lack of time and resources to conduct proper research and usability testing in with prototype.