Central Market
Benchmark Study
Report

Overview
PROJECT GOAL
During the COVID-19 pandemic, mobile ordering and grocery curbside pick-up and delivery services saw an increase in customer use. This project consisted of multiple stages to learn how the Central Market mobile app compared against competitors. The final report is a benchmark study between Central Market and Walmart, a lead competitor in the Austin, TX, detailing high-level findings and recommendations. 
ROLE
Project Lead, UX Researcher
TEAM
Pei-Syuan Jhang, UX Researcher
Yiran Su, UX Researcher
Aris Wells, UX Researcher
METHODS
Competitive Analysis, Heuristic Evaluation, Evaluation of Task Inputs, Recruitment, User Interviews
Methodology
Understanding user needs and expectations
Why do shoppers use grocery apps?

Through conversation with grocery app users in Austin, TX, we noticed that they shared similar goals when it came to using a grocery app for pickup or delivery:

 

  • It saves time.

  • It allows users to avoid in-store crowds and long checkout lines during the COVID-19 pandemic.

  • It’s easy to locate special deals and sale items. 

What do users expect from a grocery app?

From what we observed, users expect these qualities from a grocery app:

 

  • Ease of use through an intuitive interface.

  • Efficient search and browse features in which relevant items are retrieved and desired item can be found quickly.

  • Clarity in departmental/categorical names and documentation of user progress and activity.

We tested the Central Market mobile app with the Walmart mobile app as the competitor in this benchmark study.
Eight participants were tested in this study. Within our recruitment criteria, all participants had either primary or shared shopping responsibility in the household and no experience with the Central Market app. (One out of the eight participants did shop at the brick-and-mortar store within the last 12 months.) All participants had little to no experience using the Walmart app.
Session duration: Each session was approximately 1 ½ hours per session, with the participant spending 25-30 minutes on each app.
Outline of tasks:
  • Task-M: Lactose-free milk, 1-gallon [search or browse]

  • Task-C: Chicken thighs (skin-on, with bone, 2-3 pounds) [browse only]

  • Task-G: Fresh Ginger (small piece, 3-4 quarter-inch slices) [browse only]

  • Task-B: Whole loaf of French Bread (example images, ok to find similar) [browse only]

  • Task-5: Review order for accuracy

  • Task-6: Place order (confirm delivery and total)

  • Task-X: Add additional item to get to $35 delivery threshold (Walmart only) [search]

  • Task-8: Cancel the order

Process:
  • Counterbalancing: Each session alternates between whether Central Market or Walmart is the 1st/2nd app used to reduce learning effects across apps. Task order for Tasks C-G-B is randomized to reduce learning effects between tasks.

  • During-tasks: Time-keepers record start and end task times. Note-takers record navigation on app and think-aloud comments from the participant.

  • Post-tasks: Participants are asked a series of questions concerning reflections, measure of ease/difficulty, ratings of confidence levels (depending on the task). 

  • SUS: After going through all the tasks on one app, the participant fills out a System Usability Scale (SUS) survey using a URL link.

Key Objectives
We were interested in task objectives that would be able to evaluate how well Central Market’s mobile app meets customers’ needs and expectations in the areas of ease of use, efficiency, and clarity, compared to Walmart.
What we were interested in testing:
  • Comprehensive navigation of app interface - ease of completing task (easy, hard, or somewhere in-between), success rate on task (success, partial, failure, or abandon), and time-on-task
     

  • Search functionality - generation of relevant results and finding specific items 
     

  • Browsing features - intuitive organization of departmental categories/labels and diverse filtering options
     

  • Modifying cart - review, remove/edit item, adjust quantity, and add substitution note/marker
     

  • Confidence - placing and canceling shopping order with appropriate feedback from app

Executive Summary

Grocery apps should help users save time and add convenience to their lives.

Searching, Browsing, Filtering

The Central Market app should be easy to use and where desired items can be found quickly, whether users are searching or browsing. This requires clear naming of department/categories and a robust filtering system.

Central Market’s search feature tends to overwrite the user’s previous query. When the user is quickly trying to add to their existing query, this has caused frustration. Although some users have identified the lack of search prediction at the time of the study, it has recently been added to bolster the search feature.

Having clear labels, abundant subcategories, and robust filtering options are helpful when a user is browsing. Central Market’s current filters are limited to price, on sale, and brand options.

Ensure that department/category names are clear and intuitive to the user. If it’s too broad, like “Fresh Market” and “Fresh Produce,” then the trade-off may be clarity and usability.

Reviewing, Ordering, Canceling

Central Market should present concise, comprehensive information to the user and notify users of the current system status, especially when sales are involved such as placing or canceling an order.

Items in cart should be displayed with simple, concise information for easy review. It should be obvious where users can add a substitution note to personalize their order.

Users’ preference for Central Market comes from an efficient checkout process and low delivery fees. Here, Central Market outperforms Walmart, which presents extraneous information/steps to the user and has higher delivery fees.

The cancellation process for Central Market took longer because the screen would freeze. Canceling orders should be easy and users should received quick confirmation of their order status.

Full report including in-depth findings, analysis of quantitative data, and recommendations is available upon request.

 

Other reports related to the Central Market project (Competitive Analysis, Heuristic Evaluation, and Evaluation of Task Inputs) are also available.