As AI and machine learning tools become more prevalent and accessible, product and engineering teams across all types of organizations are developing innovative AI-powered products and features. AI is particularly suited to pattern recognition, prediction and forecasting, and user experience personalization, all of which are common in organizations that process data.
Data is a precursor to the application of AI - a lot, a lot! Large datasets are usually needed to train an AI model, and any organization with large data sets will undoubtedly face challenges that AI can help solve. Alternatively, data collection can be the "first phase" of AI product development if the datasets don't already exist.
Whatever datasets you plan to use, there's a good chance that people have been involved in capturing that data or are in some way engaging with your functionality. 'artificial intelligence. Principles for UX Data design and visualization should be an early consideration when entering data and / or presenting data to users.
1. Take a quick look at the user experience
Understanding how users will interact with your AI product early in model development can help put some useful guardrails on your AI project and ensure the team is focused on a common end goal.
If we take the "Recommended for you" section of a movie streaming service, for example, outlining what the user will see in that feature before starting the data analysis will allow the team to focus only on the model outputs that will add value. So if your user research determined that the movie title, picture, cast, and length will be valuable information for the user to see in the recommendation, the engineering team would have important background to decide which datasets should train the model. Data on the actors and length of the film seems essential to ensure the accuracy of the recommendations.
User experience can be broken down into three parts:
- Before - What is the user trying to accomplish? How does the user arrive at this experience? Where are they going? What should they expect?
- During - What should they see to orient themselves? Is it clear what to do next? How are they guided through the errors?
- After - Has the user achieved their goal? Is there a clear "end" to the experience? What are the follow-up steps (if any)?
Knowing what a user should see before, during, and after interacting with your model will ensure that the engineering team trains the AI model on accurate data from the start, as well as providing output that is the more useful to users.
2. Be transparent about how you use data
Will your users know what happens to the data you collect from them and why you need it? Would your users need to read pages of your T & Cs to get a clue? Consider adding the rationale in the product itself. A simple "this data will allow us to recommend better content" could remove the friction points from the user experience and add a layer of transparency to the experience.
When users seek the support of an advisor from The Trevor Project, we make it clear that the information we request before connecting them to an advisor will be used to provide them with better support.
If your model is showing results to users, go further and explain how your model came to its conclusion. "Why this announcement?" The Google option gives you insight into what drives the search results you see. It also allows you to turn off ad personalization completely, giving the user control over how their personal information is used. Explaining how your model works or its level of precision can increase confidence in your user base and allow users to decide on their own terms to engage with the outcome. Low levels of precision can also be used as a prompt to collect additional information from users to improve your model.
3. Collect user information on the performance of your model
Inviting users to provide feedback on their experience enables the product team to make continuous improvements to the user experience over time. When thinking about collecting feedback, consider how the AI engineering team could benefit from user feedback as well. Sometimes humans can spot obvious mistakes AI wouldn't make, and your user base is made up entirely of humans!
An example of collecting user feedback in action is when Google identifies an email as unsafe, but allows the user to use their own logic to mark the email as "Safe." This continuous manual user correction allows the model to continuously learn what unsafe messages look like over time.
If your user base also has the contextual knowledge to explain why AI is incorrect, that context could be crucial in improving the model. If a user notices an anomaly in the results returned by the AI, consider how you could include a way for the user to easily report the anomaly. What question (s) could you ask a user to gather key information for the engineering team and to provide useful signals to improve the model? Engineering teams and UX designers can work together during model development to plan to collect feedback early on and set the model in place for continuous iterative improvement.
4. Evaluate accessibility when collecting user data
Accessibility issues lead to biased data collection, and AI that is trained on exclusionary datasets can create AI bias. For example, facial recognition algorithms that have been trained on a dataset of mostly white male faces will work poorly for anyone who is neither white nor male. For organizations like The Trevor Project that directly support LGBTQ youth, taking sexual orientation and gender identity into account is extremely important. Searching for externally inclusive datasets is just as important as making sure the data you bring in or intend to collect is inclusive.
When collecting user data, consider the platform your users will use to interact with your AI, and how you might make it more accessible. If your platform requires payment, does not meet accessibility guidelines, or has a particularly heavy user experience, you will receive fewer signals from those who cannot afford to subscribe, who have accessibility needs, or who are less tech savvy.
Every product manager and AI engineer has the ability to ensure that marginalized and under-represented groups in society can access the products they build. Understanding who you subconsciously exclude from your dataset is the first step in building more inclusive AI products.
5. Think about how you will measure equity at the start of model development.
Fairness goes with ensuring that your training data is inclusive. To measure fairness in a model, you need to understand how your model may be less fair in certain use cases. For models using people data, looking at how the model performs in different demographics can be a good start. However, if your dataset does not include demographic information, this type of fairness analysis might not be possible.
When designing your model, think about how the output might be skewed by your data, or how it might underestimate some people. Make sure the data sets you use for training and the data you collect from users are rich enough to measure fairness. Consider how you will monitor fairness as part of regular model maintenance. Set a fairness threshold and create a plan to adjust or recycle the model if it becomes less fair over time.
As a novice or seasoned technician developing AI-powered tools, it's never too early or too late to examine how your tools are viewed by your users and their impact. AI technology has the potential to reach millions of users at scale and can be applied in high-stakes use cases. Looking at user experience holistically - including the impact of the AI release on people - is not only a best practice, but can be an ethical necessity.
