Evaluating the website UX design through the best design principles and usability of the account creation and sign-in flow of a Nigerian eBay-inspired start-up company
My Role: Team Lead UX Researcher | My Team: 2 Lead UX Researchers
Duration: 14 weeks
BACKGROUND
ATrade is an e-commerce startup focused on the Nigerian market that allows customers to buy and sell products online. What sets ATrade apart from its competitors in Nigeria is its commitment to providing a safe online buying and selling environment.
A seamless account creation and sign-in flow is important for ATrade success. If this process is difficult, customers may abandon the platform, which would prevent customers from continuing to use the website, save products in carts for later purchases, track deals, save their payment method for future purchases, communicate with sellers, and monitoring order delivery.
PLANNING THE RESEARCH: WHAT WAS TESTED?
Main Research Question
How effectively and efficiently can users complete account creation and sign-in tasks, and how satisfied are they with the overall experience?
Research Goal
To evaluate whether users can create accounts and sign in efficiently, to improve engagement, retention, and perceived security.
Hypotheses
H1) Account Creation Process: 100% of users will be able to complete the account creation process by putting in their email address to register.
Task: Create an account
H2) Sign-In Flow: It will take the users more than 6 steps to log out, find the temporary password in their emails, and successfully log back in. Their perceived effort will increase during this process.
Task: Please log out and sign back in
H3) Password Creation: Difficulty locating the password link will frustrate users and may cause them to abandon the site. The complexity of searching between the website and email and checking spam folders will negatively affect their overall satisfaction with the platform.
Task: Locating the password in the email, or changing it from the account settings
WHAT METHODS WERE USED?
Quasi-experimental design: Summative Usability Test & User Insights
A summative usability test is used to evaluate a product through defined measures. It uses UX metrics of users' success to assess whether the product meets defined usability success metrics and can stand up to competition, .
*Calculated needed sample size: 25 participants. My study was conducted with 36% of the required sample size, so more participants are needed to confirm statistical significance and population representativeness.
Full Budget/Screening Criteria: See here for full screening requirements for a larger sample size study
THE EXPERIENCE & TESTED PROTOTYPE: HOW WAS IT TESTED?
The users conducted their given tasks on a Figma prototype, as shown below.
Flow 1: Account Creation
Email Login: Locating Password & Sign-In Flow
Redesign: Mock-up of Potential Redesign for Account Creation
Clicking the wrong email
Pressing report not spam first
Finally finding the email in Spam
Clicking the Search bar first
Google survey
EXECUTING THE RESEARCH: TESTING THE EXPERIENCE WITH PARTICIPANTS
Understanding what usability metrics we wanted to collect to evaluate the user's experience and satisfaction with the website. The next step was to come up with a script and come up with questions for a semi-structured interview after the usability test.
The Script
Introduction:
Hello, thank you for taking the time to participate in this experience of going through the ATrade website with us today. Any feedback and opinions from this experience will be greatly appreciated by the researcher and developers. For this session, before we begin, do we have your verbal consent to record this session for research and data analysis purposes later?
[wait for consent before starting to record]
Okay, this session is now being recorded. Can we have on record that you have given us your permission to record this session with you, and permission to use the data collected during this session for analysis purposes later? This information is not going to be seen by anyone outside of our UX Research team, and your information, like names and any other personal information, will not be used for any other purposes. Any feedback and opinions from this experience, said by you, will be greatly appreciated by the researchers and developers. Do we have your consent and permission?
[wait for answer]
In addition, we want to let you know, this session is expected to be 20 minutes long, and you will be getting a $5 USD gift card as an appreciation for your time and feedbacks, failure to complete this session or having to drop out of this session at any time will result in you not receiving on this compensation.
Can you confirm you understood how long this session will take and your participation, and compensation?
If time goes over 20 minutes, you will not be compensated for those times, it will be volunteer to you to continue with the session; will you be okay with this arrangement? In addition, everything that you learned or saw here today must stay anonymous; this would be considered a verbal NDA, as the website is still in the beta version and has not been publicly launched yet. Do we have your agreement?
Pre-task questions:
Before we start the tasks, let’s get to know a little about you.
- How old are you?
- What is your occupation?
- Have you used any online shopping platform before?
- Have you bought anything?
- Created an account online?
- Which sites have you used, or did you recently use, to purchase the last item you bought?
- How often do you purchase a product/item using an online platform?
Thank you for the information. Now we are moving on to the tasks.
Before we start, this is also a think-aloud test. Please, as you go through the task, we would appreciate you verbalizing your thinking on every action you are doing as you carry out each task.
You have just arrived on this website. Please take 1 minute to explore this homepage and website.
[wait]
Task 1: You are interested in creating an account. Can you create an account?
Go to survey link: https://docs.google.com/forms/d/e/1FAIpQLSd5SFwpLJZrT-AJESASYrMXQ6ZIcneLdsF1XnyR_xFyUStoog/viewform
Answer questions 1 and 2 on the survey
1) Do you disagree or agree that the process of creating an account the first time is simple?
1- Strongly Disagree
2- Disagree
3- Neither Agree or Disagree
4- Agree
5- Strongly Agree
2. Do you disagree or agree that the process of registering an account felt secure?
1- Strongly Disagree
2- Disagree
3- Neither Agree or Disagree
4- Agree
5- Strongly Agree
Thank you. Now, you are done browsing. Please log out.
Task 2: Please log back in. (Record steps starting from here to how long it takes for them to complete the tasks)
Please know that at this point, you have not created a password yet. What would you normally do, or where would you go to check for your password or to create a new one?
[wait for response and change to email login flow]
[wait for them to complete the task]
Task 3: Can you go change your password from the account settings?
Thank you. That is the end of our usability test. I have some questions that I would like to ask you to learn more about what you think about this experience. Please give your honest opinion and feedback in the rest of this survey.
Survey Questions
3. How easy or difficult was it to complete the account creation process?
1- Very easy
2- Easy
3- Neither easy or difficult
4- Difficult
5- Very difficult
4. How easy or difficult was the process for the sign-in flow for the first time (e.g., creating a new password or retrieving the password)?
1- Very easy
2- Easy
3- Neither easy or difficult
4- Difficult
5- Very difficult
5. Would you continue to use the website, or use this website again, having experienced this process?
Yes
No
No opinion
[wait for them to finish the survey]
Thank you for completing the survey. There are some additional questions I want to ask to learn more about your experience.
Qualitative Questions
Q1: How do you feel about creating an account? For example, just putting in your e-mail address?
Q2: What do you think about having to go into your email to retrieve or create a password?
Q3: What do you think about the sign-in process?
Q4: What do you think should be added to this register or sign-in process that would make the process feel more secure?
Debrief:
That is the end of our usability test with ATrade. We appreciate your time and feedback on helping us test the account creation and sign-in process on our e-commerce website today. Because this website is currently still under development, please do not share any information or what you did here today with anybody. But your feedback is very valuable to us as we are still in the beta version, and your opinions will help us improve the overall experience before we launch the website publicly. Thank you.
Example from the slide deck to the stakeholder to identify the success rate
ANALYZING THE RESEARCH: WHAT DID WE DO WITH THE DATA COLLECTED?
I analyzed the usability test results metrics using benchmarking with industry standards, binomial mid-p probability proportion, and proportion estimation statistics for binary and rating scale metrics. Thematic analysis was used to analyze the user interviews to find common themes mentioned.
Tools Used: Miro Board, Notion, Condens, ChatGPT, Quantifying the User Experience, Bayard Institute, MeasuringU
Analyzation Measures Data Collected
Data collected from our survey to understand user's perception of usability of account creation and sign in flow, and satisfaction of website
RESEARCH INSIGHTS: WHAT DID WE FIND?
Based on the analysis, I found the following insights...
A TALE OF TWO PERSPECTIVES: THEY FAILED BUT FOUND IT EASY?
Digging further into the user qualitative, I found that the answer to that question, quite simple: users are not aware of their errors and frustrations in the moment.
RECOMMENDATIONS: SO WHAT'S NEXT?
The Account Creation and Sign-In flow is usable from the user's perspective, but based on our research findings, I recommend the following changes to make their experience even better.
STUDY'S LIMITATIONS: A LITTLE REMINDER...
Our test results were collected with may not fully represent ATrade's targeted user base (Nigerians), so the results yielded may not be most representative of the targeted users, in addition to not being tested with the calculated sample size (n=25). However, further testing with a larger sample size and with a more representative sample and A/B comparison between the redesign and original design could provide more definitive insights into the efficiency of the account creation and sign-in flow.
Have lingering questions? Contact me for more information :)
lao.serena@gmail.com