Challenge
Deliverables
Role
learn how to analyse usability testing data to redesign Gucci website.
website
UX researcher & UX designer
Redesign the Gucci website product pages based on usability testing, including testing 80 participants, and analysing qualitative and quantitative data.
Introduction
Gucci started its online purchase website in 2002 and had 9.2 million visits in February 2021 (Sabanoglu 2021). As a case study, it is worthwhile to do the usability testing for further study.
User Task Identification
Top Taks
Designers satisfy users’ needs by learning about what they do on websites (VirothaisakunIn 2015). For identifying priority needs, participants chose the most important five tasks among sixty tasks on fashion websites. According to the results, it reveals that eight tasks account for 51 percent of all tasks. The top tasks are image quality, search, and delivery. In addition, big tasks contain sizing information, checkout, filtering, shopping cart, and customer reviews.
The following user testing tasks are used in the evaluation.
-
Search for a jacket for night out at the weekend and filter for your size.
-
Check sizing information and select the correct size, and add to basket.
-
Find out the fastest delivery and the price of it.
Including below website page:
Analysis
Performance
Introduction:
Nielsen (2012) suggested that effective websites may be more acceptable to users, which means high performance is significant to usability. Performance metrics can collect this objective data and make usability measurable (Nielsen 1994). Therefore, it is vital to analyse the performance data of Gucci to evaluate its usability. Rick Owens's website will be used for comparison.
Setting and Sample
16 groups of students evaluated www.gucci.com. According to an article (Nielsen 2012), this sample size is up to the standard of minimum sample size.
Data Collection
When participants completed user tasks in Gucci and Rick Owens, researchers recorded the following data:
-
Time
-
the number of errors during the tasks
-
the possibility of success
-
whether follow the optimum pathway
Data analysis
Based on a small sample size (n1=16, n2=16), non-parametric statistics were applied. Wilcoxon Signed Rank was used because of the same participants evaluating two websites. The researcher chose a one-tailed to calculate. This is because one-tailed can show whether one group is better than another one and two-tailed can show the significance (Birkett 2019)
Results
Task Success:
Wilcoxon Signed Rank Test (one-tailed) was used to investigate if Gucci has a higher possibility of completing tasks than Rick Owens.
The result revealed that there was a statistically significant difference in the task success between Gucci (Md=100) and Rick Owens (Md=83.3), z = -2.2014, p < .05, with a small effect size (r=-0.39) (Cohen 1988 pp 284-7).
The possibility of completing the task in Gucci is 95.825 percent, while in Rick Owens is 83.3 percent, which means that the Gucci website is much easier to use.
Time:
Wilcoxon Signed Rank (one-tallied) was used to study whether Gucci has a shorter task time than Rick Owens.
The result revealed that there was no statistically significant difference in the task time between Gucci (Md=162) and Rick Owens (Md=176), z = -1.3631, p=0.08692
Gucci and Rick Owens have similar performance in task time.
Error:
Wilcoxon Signed Rank (one tallied) was used to find out if Gucci has more errors than Rick Owens.
The result revealed that there was no significant difference in the number of errors between Gucci (Md=1) and Rick Owens (Md=2), z = -1.5993 p<.05
Gucci and Rick Owens are similar in the errors aspect.
Efficiency:
Lostness is a calculation when analysing effort data.
Wilcoxon Signed Rank Test (one-tailed) was used to find out if Gucci has a lower lostness than Rick Owens.
The test showed no significant difference in lostness between Gucci (Md =0.22) and Rick Owens (Md=0.23), z= -0.5112, p = .30503.
Gucci and Rick Owens have similar efficiency.
Conclusion:
The results indicate that Gucci and Rick Owens have similar performance in time, errors, and efficiency. However, it is simpler to use the Gucci website than Rick Owens due to the small effect size in task completion. It seems that Gucci has a high performance and may not be changed greatly in the redesign. Although the sample size met the requirement, it is more accurate when using a larger sample size.
Satisfaction
Introduction:
Emotion is an essential part of user experience, which cannot be analysed in quantitative measures (Bargas-Avila and Hornbæk 2011). The satisfaction metric is used to collect users’ emotional data.
System Usability Scale (SUS) is a common satisfaction metric, which was created for collecting feedback more quickly and simply (Brooke 1996). SUS usually sets up questions and participants select a point between 1 and 5 (from disagreement to agreement) on a scale.
Think Aloud asks participants to express their thinking when doing tests (Jaspers et al 2014). It is a direct way to learn about users’ feelings.
Setting and Sample:
Although 40 participants are the basic recommended sample size, 78 participants took part in the SUS (Budiu and Moran 2021). 3 participants took part in Think Aloud because of limited conditions.
Data Collection:
Participants completed tasks on 4 websites (website1=Gucci, website2= Rick Owens, website3=Supreme website4=Zara) and an online survey with 10 questions (Brooke 1996). They expressed agreement or disagreement by choosing 1 to 5.
Participants talked about their thinking when doing tasks. The microphone was used to record conversation.
Data Analysis:
Use the SUS calculator to get the SUS score of 4 websites and compare them. Kolmogorov-Smirnov test is used to identify if the data is normal. Based on the result, parametric statistics were applied. Dependent T-test was used because of the same participants. The researcher chose a one-tailed to calculate. This is because one-tailed can show whether one group is better than another one (Birkett 2019).
Thematic analysis may be divided into 3 parts: Do, Think, and Feel.
Results:
SUS Score:
The average SUS score on the Gucci website is 74.3 (acceptable), Rick Owens is 46.8 (unacceptable), Supreme is 24.5(unacceptable), and Zara is 71.4 (acceptable).
The dependent t-test (one-tailed) is to investigate whether the Gucci website has better satisfaction than Rick Owens. There was a statistically significant difference in SUS scores between Gucci (M= 74.3, SD= 13.5) and Rick Owens (M= 46.8, SD=18), t (77) =-13.256743, p= <.00001. The mean difference was -27.44. The ETA squared statistic (. 695) indicated a large effect size. Gucci SUS score is the highest among 4 websites and it is higher than 70. It suggests that the Gucci website has a better satisfaction than Rick Owens.
Thematic Analysis:
Combined with the Do, Think, and Feel code table, it indicates that:
-
Size choice on the product page is confusing, need to use the size guide
-
Product descriptions are various but important information is not listed first
-
The return policy is a little bit complicated to fully understand
-
Customer reviews are needed
-
checkout may not need to sign up
-
Various Images with different views are good
-
The estimated shipping information is helpful
Gucci has high satisfaction data on the product images, product details, and shipping information. However, the size guide on the product page and the return policy can be clearer. Important information like material should be placed at the top. Besides, customer reviews are needed, and users may purchase items without login.
The following tables are DO, Think, and Feel code tables:
Conclusion
The results suggest that Gucci has acceptable satisfaction, and it is better than Rick Owens. However, some problems like size and return policy should be improved.Based on the results, it seems that Gucci may not be changed greatly. However, the sample size in the Think Aloud method is too small, which may cause less comprehensive and reliable results.
Biometrics
Introduction:
Biometrics is to analyse body reactions to make users’ thinking understandable. Eye-tracking is one of the biometrics methods, which is to record eye movement Ehmke and Wilson (2007) suggested that designers may know users’ attention, cognition, and behaviour patterns through eye tracking.
Data Collection and Analysis:
Researchers use Eye-Quant to collect eye movement data.
Heatmaps can show users’ attention visually. When users look more and longer in a certain area, it may become red. Green or blue areas mean less fixation. Users may neglect areas without colour.
Result:
The text on the banner on the homepage is red, which indicates that it gains main attention. Lots of areas have no colours, like top right corner. Search is placed on the top right, which possibly means that users cannot find how to search.
On the search page, the product images gain much attention. The filter area shows less attention.
On the product page, users may focus on model images first, then the shopping bag button. Color and price information show less attention.
When scrolling down, users still focuses on the model images. Other information like product description may not be noticed.
Conclusion:
According to eye tracking results, product images gain much attention. Some functions like search and filter may be neglected. Less attention to some information like product price and colour.
In the redesign, designers should use colour contrast to stress text or enlarge the text size.
Heuristics
Introduction
Heuristics is an effective way for evaluation (Nielsen 1994). It requires several researchers with basic usability knowledge to work together. The result may be professional and less time-consuming. Researchers may find solutions from it. Nielsen (2020) updated 10 heuristics principles for designers to evaluate better.
Method
6 people in one group shared opinions of the Gucci website and listed them in both positive and negative aspects, based on 10 heuristic principles (Nielsen 2020). They listed problems with different levels of importance and discussed solutions of these problems.
Result
Conclusion
According to the heuristic evaluation,
Positive:
-
Use colour to show status
-
Suitable icon design
-
Elements placed properly
-
Provide similar results when typing in the search
-
Major information is placed in the middle of the product page
Negative:
-
Search: no history. When no result shows up, just a simple reminder.
-
Filter: only select the category, and then the size options show up.
-
Lacks customer reviews
-
Shopping bag: cannot edit
Recommendation:
-
Search: add history. Provide similar results when no result shows up
-
Filter: add size options in the original filter
-
Add customer reviews
-
Shopping bag: can edit
Accessibility
Introduction
Accessibility can be explained as making design accessible to all people (Sauer 2020). When designers need to evaluate the accessibility of a website, the Web Content Accessibility Guidelines (WCAG 2.1) are commonly used. WCAG 2.1 (W3C 2018) mainly focuses on four types of disabilities, including hearing, vision, mobility, and understanding. According to these disabilities, content can be divided into four parts, which are perceivable, operable, understandable, and robust design.
Setting and sampling
6 students participated.
Data Collection
Researchers evaluated the Gucci website based on WACG 2.1 Level A (W3C 2018) and shared findings on miro. Wave was used to test WACG 2.1 Level AA.
Results
Level A Accessibility Issues:
Diversity assessment
Categorisation
Level AA Accessibility Issues
Main accessibility issues from WCAG 2.1 AA - according to Wave
-
130 empty links
-
303 instances of text and background colour being too low contrast to pass Level AA or AAA.
-
1200 design issues, including text that is too small to read
Conclusion
The Gucci website was evaluated for accessibility based on WCAG 2.1. The result shows that the Website fails WCAG 2.1’s basic guidelines.
The main perceivable issue is size choice in the filter only shows up when the category is selected on the search page. On the product pages, product images are too large, and they mix up with the navigation bar and other parts. The video on the product page is played automatically. Important product information is placed at the back of the description. The text size is too small, and the colour is not obvious.
The main operational issues are empty links and users cannot slide pictures easily when using the keyboard.
The main understandable issue is that the measurement way in the size guide is not clear.
Conclusion
The evaluation result indicates that the Gucci website need to be redesigned in some parts. Designer should mainly use original website elements to design.
Designers should:
-
size choice should be placed stable in the filter
-
smaller product images
-
provide video playing option on the product page
-
larger and darker text
-
place the product description in bigger place and put the important information first
Psychology
Introduction
There are five psychological principles of design, which are Colour theory, Gestalt principles, Information Scent, Dual coding theory and Mental models. Colour theory enables designers to use colour to convey information based on colour temperature and colour scheme (Interaction design foundation no date). Gestalt principles are the basic rule to sense shapes, patterns and elements. These principles include proximity, similarity, figure-ground, continuity, closure and connections (interaction design foundation no date). Information scent is a principle that users have the assumption of the location of different elements (Nielsen 2004). Dual coding theory enables users to gain long-term memories when using words and figures together (Jiang and Benbasat 2007). Mental models are theories that users prefer elements that are connected to the real world because they can easily be familiar with them. (Nielsen 2010).
Result
Colour Theory:
-
When moose clicks on search, the navigation bar changes to black, which is a good way to address the status.
-
The colour of the reminder in search pages is not the supplement colour with the background. On the login page, red is used to remind and error.
-
When the moose moves to a certain item, the colour becomes brighter.
-
Low colour contrast between images and text
-
Use colour contrast to emphasize
Gestalt principles:
-
Based on common regions, users may think products with the same boundary contain the same content. Each option in the filter has the same boundary as others.
-
Items in variation
-
Shipping and return, gift, and payment
-
Based on proximity, when items are close, users may think they are more related than those are far away. Items in variation is too far from the description.
-
Based on similarity, elements with the same shapes are usually considered as a similar function. There are the same plus icons in each content. Although the filter button and sort button are far away, they can still be considered as the same function because of the same shapes.
-
Based on figure/ground, different shapes of objects in different layers can be easily noticed by users. After adding to the shopping cart, shopping bag information shows up. When choosing colour, variation jumps out.
Information scent:
-
Do not find size in the filter when typing words on the search.
-
Lack of washing information and list details randomly
Dual coding theory:
-
Some icons have text descriptions, while others do not. Search, Wishlist, the share function does not have text, so users may get confused about it.
-
Only text, no icon. Sign-in and help do not have icons, which makes them less obvious.
Mental models:
-
Icons are related to reality
-
Use common items in the navigation bar
Conclusion
The evaluation result reveals that Gucci has some problems in psychology, which are mainly about colour theory, gestalt principles, information scent and dual coding.
Redesign:
According to colour theory, some texts and background colours do not follow the colour contract. Some elements should be changed in colour to convey information. According to Gestalt Principles, product information should be put in a more salient place. Also, important information may be placed at the top. According to the information scent, the size option in the filter should be placed on the search page. In product information, information should be listed in a logical way. According to dual coding theory, icons and texts should be combined for better understanding.
Summary of Usability and Accessibility Issues
Usability Issues
Lack of element
-
Search: no history. On no-result page, no similar items are recommended
-
Filter on the search page, If no category is selected, the size option missing.
-
Customer reviews
-
Shopping bag: In Pop-up status, cannot be edited.
-
Quick buy: add an item to the shopping bag first.
-
Login: cannot checkout without login
Order Of Elements:
-
Product page: the product description is displayed on a small corner
-
Product description: less vital information is put in the first. The text size is small
Other:
-
Product Images: product images are too large.
-
Customer service: too small to find
-
Icons: without text
-
size guide: measurement is too simple
Accessibility
Colour
-
The colour of the navigation bar is changed to black only when the mouse moves to it. It is easy to neglect.
-
The colour of icons and text is inconspicuous.
Text
-
Some text sizes are too small to read, like customer help, sign up, product description
-
Important content text size should larger
Non-text content
-
The video on the product page plays automatically and repeatedly. Users cannot control speed, pause and other functions
-
The boundary of images is too large. Images on the product page are easily mixed up with other parts.
Redesign of Gucci Website
Details
Homepage
-
Navigation bar: background colour changes into black. Based on colour theory in psychology, strong colour contract can emphasize the importance.
-
Icon: larger icon with texts. Based on perceivable design in accessibility evaluation, text alternatives for non-text content. Also, dual coding theory in psychology suggests that icons should be with text together.
-
Text: Based on WAVE result of accessibility issues from WCAG 2.1 AA, text size should be larger.
Search
-
According to presenting system status in heuristics, add search history.
-
On the search page, the size on the filter is missing. According to the perceivable design of accessibility evaluation, the logical structure of different elements may affect usability.
Product page
-
Based on WACG 2.1 Level AA and colour theory, text size is larger, and colour is darker.
-
Based on WACG 2.1 Level AA and colour theory, text size is larger, and colour is darker.
-
Customer service: According to help and documentation in heuristics, it is vital to provide help function. Make customer service into a bigger button.
-
Product description & shipping: switch description and shipping. Make the description more obvious. Based on perceivable design in accessibility, elements should be in a meaningful sequence. More important information in description is displayed in the first, others are put at the bottom.
-
Customer reviews: based on efficiency of use in heuristics and top tasks results, it is essential to have customer reviews.
-
Shopping bag: based on user control and freedom theory and efficiency in heuristics, users can edit or remove items in shopping bag with pop-up status instead of clicking into pages.
Checkout
-
Shipping & return: based on recognition rather than recall theory in heuristics, users are not willing to remember things. Shipping and return policy are added in checkout page for users to consider well before purchase.
Redesign Testing (Performance and Satisfaction)
Performance
Setting and Sample
Based on the convenience sample, 6 participants evaluated the Gucci redesign website through figma.
Data Collection
When users were completing user tasks, researchers needed to record the following data: time, completion, errors, and efficiency.
Data analysis
Based on a small sample size, non-parametric statistics were applied. Mann-Whitney U Test was used because of different participants. The researcher chose one-tailed to calculate. This is because one-tailed can show whether one group is better than another one (Birkett 2019).
Result
Task success:
Mann-Whitney U Test (one tailed) was used to investigate if Gucci redesign has higher possibility of completing tasks than Gucci original.
The result revealed that there was no significant difference in the task success between Gucci redesign (Md=100 n=6) and Gucci (Md=100 n=16), U = 42, z=-0.40547, p=.3409
Gucci redesign and Gucci have similar performance in task success. Both websites are easy to use.
Time:
Mann-Whitney U Test (one tallied) was used to study whether Gucci redesign has shorter task time than Gucci.
The result revealed that there was statistically significant difference in the task time between Gucci redesign (Md=120 n=6) and Gucci (Md=162 n=16), U = 20,
z=2.02733, p=.02118 The test achieved a moderate effect size; r=0.43
Gucci redesign has better performance in task time.
Error:
Mann-Whitney U Test (one tallied) was used to find out if Gucci redesign has fewer errors during the tasks than Gucci.
The result indicated that there was no significant difference in the task time between Gucci redesign (Md=0.5 n=6) and Gucci (Md=1 n=16), U = 36, z=0.84779, p=.19766
Gucci Redesign and Gucci have similar performance in errors.
Efficiency:
Lostness is a calculation when analysing effort data. Mann-Whitney U Test (one tailed) was used to find out if Gucci redesign has a less lostness than Gucci's. The result pointed out that there was a statistically significant difference in the efficiency between Gucci redesign (Md=0 n=6) and Gucci (Md=0.22 n=16), U =9, z=2.83826, p=.00226 The test achieved a large effect size; r=0.61
Gucci redesign has better performance in efficiency.
Conclusion
The results indicate that Gucci Redesign and Gucci have similar performance in task success and task errors. However, Gucci's redesigned website is more effective than Gucci's due to the moderate effect size in time and a large effect size in efficiency. It seems that the Gucci redesign has slightly better performance, but a small sample size and limited testing of website pages may lead to imprecise results.
Satisfaction
Setting and Sample
Based on a convenience sample and limited conditions, 6 people took part in the satisfaction evaluation of Gucci redesign through Figma, including think-aloud and SUS.
Data Collection
Participants completed tasks in redesigning Gucci websites and an online survey with 10 questions (Brooke 1996). They expressed agreement or disagreement by choosing 1 to 5. Participants talked about their feelings, action and thinking when completing tasks. The microphone was used to record what they said.
Data analysis
Using the SUS calculator to get the SUS score in the redesigned Gucci website and compare it with the original one and the other 3 websites mentioned above. Based on a small sample size, non-parametric statistics were applied. Mann-Whitney U Test was used because of different participants. The researcher chose one-tailed to calculate. This is because one-tailed can show whether one group is better than another one (Birkett 2019).
Thematic analysis is divided into 3 parts: Do, Think, and Feel.
Result
Think aloud:
Combined with the Do, Think, and Feel code table, and compared with the original Gucci website data, it indicates that:
-
Product description is simpler to read but still has too much information
-
Customer reviews are needed
-
Structure is understandable
-
Text size is more suitable to read
-
Estimated shipping information is helpful
-
Shipping and return information at checkout is helpful
-
The navigation bar function needs to be emphasized
Gucci redesign has high satisfaction on text content, logical order, useful elements like customer reviews, shipping information. Although product description is displayed in larger areas and useful information in the first, there is still filled with useless information.
The average SUS score in the Gucci redesign is 74.2 (acceptable). The redesigned website has a similar SUS score to the original one. A Mann-Whitney U Test indicates that no significant difference in the Gucci redesign (Md=72.5, n=6) and Gucci (Md=75.0, n=78), U = 216.5, z= 0.29526, p= .38209. Through think aloud method, some problems have been solved, but SUS scores indicate there are no differences. It seems that the Gucci redesign has similar satisfaction to the original one. However, the sample size in the redesign evaluation is too small to get reliable results.
Discussion and Conclusion
Gucci's website has a simple and elegant design style, but this leads to negative impacts on usability and accessibility. Based on design aesthetics, some text sizes are small, and images are too attractive. Although the website design looks good, it is difficult for some disabled people to use it. Also, some essential information like washing instructions is displayed in an unimpressive place. Gucci chooses to emphasise green products because it needs to build its brand image.
The top tasks result suggests that customer review is one of the most important elements on the website. Think aloud result motioned that customer review can help them make a decision. A lot of retail websites including Gucci have no customer reviews. This is because customer reviews may affect sales.
In the evaluation of Gucci, the sample size is not big enough to draw a reliable conclusion, especially in the redesigned test. Besides, participants with experience or no experience in using the Gucci website can affect results. Redesign website pages are limited, but it is reasonable it has good performance.