SHERPAAn application that helps cyclists perform maintenance and repair tasks
Role I conducted user experience research to guide design decisions and provided technological expertise to judge feasability of features.
Team Benton Humphreys, Varnit Jain, Kevin Key, Yunfei Wang Timeline Aug 2019 - Nov 2019
The first step in our research process was to talk to users about their bicycle riding habits. We interviewed seven users and were interested in their frustrations and motivations related to performing bicycle repair and maintenance tasks.
We analyzed the interview data to finalize the research direction. We created an online survey to get data from a wide demographic. We had three interest areas:
1. Bicycling riding frequency and purpose
2. Repair/Maintenance experience
3. Common frustration or pain points
We circulated the survey on online bicycling communities and received 67 responses.
After concluding our two research methods, we decided to use affinity modeling to combine the information we had gathered to find common themes and pain points. We turned the survey data and interview answers into high quality notes for modeling and created the affinity model found below. After completing the affinity model, we added our own design ideas and questions to areas we found relevant to help with concept generation.
While we liked the ideas and thought were very relevant, we realized we had not given enough importance to our user’s pain points by going straight to design ideas. We decided to go back to the affinity model to find common problems found from our research and then developed those into seven pain points that represent our user group.
1. Users lack the resources (e.g. tools, space) to perform bicycle maintenance
2. Users unfamiliar with maintenance rely and depend on others with more experience
3. Community resources are not readily available
4. Users feel helpless when they are unable to perform maintenance tasks themselves
5. Instruction guides do not directly correlate to the user’s own bike components or situation
5. Users have difficulty performing maintenance tasks for the first time
5. Users can be apprehensive to visit bike shops due to perceived cost
After noting down the pain points, we weighed each design idea based on relevance, feasibility, and creativity to weed out ineffectual ideas. Once we had a list of features, we decided to group our design ideas into broader categories based on similar topics. We came up with the following themes:
1. Online resources pool
2. Diagnostic tools
3. Integrated Technology
4. Community repair stations
5. Gig economy
Post analyses of all themes, we focused on three design directions which solved most of the pain points. We ensured that each design direction was varied and targetted a unique yet overlapping set of user groups.
After we had all of our themes, we further analyzed them based on how many pain points they tackled. Once we had the top three themes decided, we further refined them into concepts by naming them, determining the specific features and functionality of each, and creating brief scenarios to explain their particular use cases.
Concept 1: Sherpa
Sherpa is a mobile application that provides the user with step-by-step guidance for diagnosing and fixing common maintenance issues. The app will use the phone’s camera and computer vision to recognize parts and prompt the user with questions to help them discover and fix the problem. If the user cannot diagnose or fix the problem themselves, they can use a paid function to connect with an expert, who will be available by video stream and an onscreen “drawing” feature.
Concept 2: NorthStar
Northstar is a concept that redefines what a public-use bicycle repair station can be. It provides users with free-to-use tools, resources, and has the added element of an interactive screen that can guide users through general repair processes. It is designed to be integrated with Google Maps to make discovering repair station locations more accessible. Further, it leverages community input in the form of users reporting issues that inform others of the available tools/resource.
Concept 3: LifeCycle
Life Cycle is a “gig-economy” service that connects users with nearby bicycle repair technicians. It is a pay-per-use service that can help a user who does not know how to repair their bike connect with someone who does. In the event of an emergency maintenance situation, the user has the option to request a pick up from a certified bicycle technician, who will attempt to fix the bicycle on location, or transport it (and the user, if necessary) to a more suitable location.
To test our design ideas, we created another survey to reach out to a wider audience. To achieve the same, we shared the survey not only to bicycling communities but also on famous facebook, reddit pages. This survey focused on Learnability, Efficiency, Memorability, Errors, Satisfaction as well as feature level evaluations. We wanted to know which features seemed attractive and useful to users and which ones felt obsolete to them. We received 44 responses and evaluated which features overcame initial pain points and were appreciated by the users.
Our final solution contains most parts of Concept 1 - Sherpa and one part of Concept 2 - NorthStar. We realized that not a lot of users will want to pay for an expert's advice, they would rather take their bicycle to the shop; so we removed the expert advice feature. We also noticed that users do not carry tools all the time so we integrated the Repair Station Locator feature from NorthStart to display public repair stations.
1. User Profile
The user can set up a custom profile that will allow them to track their use of the app, maintain a maintenance schedule, and connect to the activity app “Strava.” This fits into the idea of Sherpa being an integrated product, and leverages Strava’s capabilities for activity tracking.
2. Bicycle Selection
This details the type of bicycle(s) that the user has, and allows them to input custom components. As repairs might be different based on the type of bicycle a user has, a “bicycle profile” can be created. These bicycles will be displayed on the home screen, and the user can easily cycle through them.
3. Diagnostic Tool
The diagnostic tool is a feature which helps users determine the problem with their bicycle. Once the option is selected, the user will be prompted with a series of questions. The question flow will depend on the answers which will help the user narrow down on a specific problem. The users who already know the issue with their bicycle can choose to skip the diagnostic tool.
4. Repair Guide
Once the user knows the problem with their bicycle, they will use the repair guide to perform repairs. The repair guide is a step by step process that tells users what to do. For each step, the user will see a small visual of the repair. We understand that the user might have to use both their hands to perform repairs and it is often not feasible to physically touch the phone or even look at it. Owing to these concerns, the repair guide has two audio elements: (1) An audio guide which speaks the step that is to be performed (2) Once the step is completed, the user can speak “Next” which moves the system to the next repair step
5. Part Identification
The part identification tool is used when the user can not recognize a certain part mentioned in the repair guide. Sherpa uses augmented reality to help the user identify the part. The user scans their bicycle using their phone camera and the selected part is highlighted when hovered upon. The feasibility of the augmented reality tool is supported by the fact that all bike manufacturers have 3D models of their bicycles which can be leveraged to support this tool.
6. Repair Station Locator
The repair station Locator is used in cases when the user doesn’t have the required tools to perform the repair process. They can use this tool is used in combination with Google Maps to direct the user to the nearest public repair station. The application also shows a list of tools available at any given repair station so that users can decide if a particular repair station has all the tools they need or not.
In our evaluation plan, we planned three evaluation methods: Usability Expert Review, Bike Mechanic Review, and Usability Testing. However, we had to perform a discount evaluation, we chose to do the Usability Expert Review, but with the inclusion of task-based evaluation to make the review more holistic. Each session involved two to three people: the evaluator, the facilitator, and possibly an additional note taker if available. We created a high fidelity prototype to test our proposed solution. We split our evaluation into two parts, the usability expert review and task-based evaluation.
Participants- Five HCI cohorts who have been trained to evaluate a design considering usability specifications
Usability Expert Review
We decided this evaluation method should be completed first because it is a hierarchical process that will quickly reveal obvious flaws in our design. Also given the scope of this project, it is much less resource-intensive compared to usability testing with our user group.
Initially, the facilitator gave the evaluator an overview of the system and provided context including the user goals, concept features, and relevant scenarios. Once the facilitator felt that they had a good understanding of the concept, the evaluator was asked to freely navigate through the home, profile, and bike selector sections of the system. We asked the evaluator to think aloud while going through the design to understand their thought process and probe better insights. Here we wanted the evaluator to cover different features each time they went through the system and become familiar with the layout.
Each evaluator had to consider the system for the usability specifications- Consistency, Customizability, Recoversability, Observability, and Responsiveness. For each usability specification we would like the evaluator to rate it with the following Likert scale of 0-4.
After the usability expert review section was complete the evaluators moved onto the task-based evaluation. The evaluators went through each task described above and again were instructed to think aloud to describe their thought process. We answered questions from the participant after each task and avoided giving instructions during the task. After the final task was completed the participants were asked a few questions to further evaluate the design.
The task-based evaluation was included to make our evaluation more holistic. We used the usability experts we recruited to go through tasks that did not involve repairing a bicycle and was considered the second part of our evaluation session. Including this helped to evaluate the logic of our system and its features, as well as the information architecture. We came our participants three tasks that a typical user would perform:
Task 1: Diagnose your flat tire
Task 2: Find a nearby repair station
Task 3: Go through the repair process
The quantitative data is from the usability expert evaluation of consistency, customizability, recoverability, observability, and responsiveness for our design as a whole. From this, we should be able to determine what aspects of the design are reworked first, based on how high the average score is. For each usability specification the evaluator rated it with the following Likert scale of 0-4:
0 - No issues, 1 - Visual/cosmetic issue, 2 - Minor issue, 3 - Major issue, fix when possible, 4 - Fix as soon as possible.
We received the following scores:
1. Consistency- 1.6
2. Customizability- 0.8
3. Recoverability- 1.6
4. Observability- 1.2
5. Responsiveness- 1.2
Once we compiled the qualitative data, we could see the changes that needed to be made on a whole as well as individual level. We saw that some areas (like Consistency) required a more work than others. We were able to get task and even screen specific feedback. The data from the evaluation of the prototype was then analyzed to inform ongoing refinement of the design. Overall, we found participant reception to our prototype to be positive, and most issues could be solved with simple changes.