Zillow Usability Case Study
ROLE
Researcher, Usability Study Facilitator
DURATION
10 weeks (Oct - Dec 2018)
DELIVERABLES
-
Research Findings
-
Surveys
-
Affinity Diagrams
-
Usability Observations
-
Cognitive Walkthrough
-
Heuristic Evaluation
-
Usability Testing
-
Severity Rating
-
Preliminary Analysis
-
Usability Study Report
TEAM MEMBERS
Reksha Rathnam
Kelly Xu
Monica Posluszny
Context
Zillow is a real estate website that can be used to find homes to buy, rent, or sell. Not only does it have important information regarding the residences but also interactive features such as saving listings and booking tours that attempt to create a user-friendly experience overall. For this study, we specifically look at the renting page of Zillow. We plan to exclusively test Zillow’s website in meeting the needs of college students in finding rental homes because they make a big part of the renter population.
We conducted a usability test with 4 tasks to test how users were able to narrow down search results based on preferences, find desired housing in a specific location, save housings, and contact the housing lister through the website. We assessed the effectiveness, discoverability, and satisfaction of the site features through analyzing how well users were able to accomplish goals. Findings and recommendations from the study are intended to help the Zillow product team improve current features and increase the retention rate of users in using Zillow to rent homes.
Research Questions
We were interested in testing the Zillow website because there are many components that are part of the housing search journey and we plan to focus on the renter perspective. The goal of this usability study is to test how the Zillow website is meeting the needs of college students in their attempt to find housing to rent. Below are our research questions:
-
Effectiveness/User Control: How easy is it for users to input their criteria for their desired home? Can the user find a residence that fits their criteria?
-
Discoverability: Can users utilize Zillow’s features to narrow down and rediscover their searches?
-
Satisfaction: How can we improve the overall satisfaction of Zillow users when looking for rental housing options?
Overall Findings
Through our usability study, we found successful elements of Zillow as well as features that could be improved. The major findings from the study are issues related to discoverability and effectiveness of site features, such as the Draw and Save features for apartment listings. Participants had trouble finding and recognizing such features, and therefore were unable to fully utilize all of Zillow’s features and functionalities. Some features were ineffective due to being inconsistent with others, not visible enough for users, or not intuitive to be used for the right purposes. Additionally, participants had to use external sites in order to complete some tasks. Overall, the site could be improved by reducing confusion, miscommunication, and inconsistencies, which would boost the overall user experience in using Zillow to find rent. We also proposed steps to take in the future that would help improve the above areas and provided more in-depth design recommendations.
Participant Profile
Environment/Testing Conditions/Roles
Our team reserved a study room in the Odegaard library for the ease of location and capacity to comfortably seat 3-4 people. We used UserZoom to help conduct the tasks. We had two laptops present, one for the participant to perform tasks and the other for the notetaker and facilitator to begin, view, and end the UserZoom test. During the test, we had one moderator who facilitated the tasks and one note-taker from our team who took notes as well as filled out our data logging tables. If we had an additional team member available, the member would act as another observer and take general notes.
Tasks
The below are tasks with scenarios completed by our 8 participants during the usability testings.
Task 1: Find homes in a city with basic requirements
You are interested in renting a place in the future with a roommate, and you want to have a basic idea about the current renting rates of houses and apartments in the city of Seattle. You also own a cat. Using Zillow.com, look for available house/apartment rentals in Seattle that allow pets with a price range of $1,000-3,000 and 2 baths.
Task 2: Filter out homes around a specific location
You want to specifically live near the Seattle University. You are open to homes in all around the campus. You want to narrow the results to just show homes around the Seattle University.
Task 3: Save desired homes and be able to find them
You have found a few listings that you are interested in and you want to keep them for later viewing. Save at least 5 different listings and locate them within the Zillow site.
Task 4: Contact lister
You have found a home that really suits you and would like more information about it. Find contact info of the lister and contact the lister by sending a message to ask if it is possible to move in next week.
Results
Successes
-
Filters allow participants to narrow down search results.
Our participants overall found that the filter bar, which follows global website filter standards and practices, was fairly easy and simple to use. All 8 participants were able to use this feature to put in home requirements and narrow down search results for their first task. Having this feature reduces information overload and shortens the home search process. The average task duration for doing this task was the shortest out of all tasks. Overall, the filters greatly improve the efficiency of finding a home that meets all user needs.
Interaction with filters
-
The Save feature is helpful for going back to view previously saved listings.
8 out of 8 participants agree or strongly agree that they found saving homes useful. They indicated that it is a simple process to save listings by clicking on the heart icon. It is also convenient to find saved listings in the upper right corner as a global feature, which also updates the number of saved listings. Additionally, having saved listings as hearts that show on the map is helpful for locating housing locations, which improves the discoverability of saved homes.
Using the save feature
Areas for Improvement and Recommendations
Below are the severity ratings of the problems that may occur during the usability testing. We categorize each problem to a severity level based on discoverability, effectiveness and satisfaction to prioritize issues and determine the order in which these issues would need to be addressed.
Priority 1 = A critical problem that needs to be resolved immediately, is preventing the user to accomplish tasks.
Priority 2 = This problem is confusing/troublesome for the participant when doing the task, but the participant was still able to complete the task.
Priority 3 = A minor problem that doesn’t impact the user’s ability to complete the tasks, but needs to be fixed to improve the overall experience in using the site.
Key Finding 1: Poor discoverability of the "Draw" feature icon on the webpage
Severity: Priority 1
The draw feature is available on the map, but it is not noticeable by many participants. For task 2, the feature is meant to help participants narrow down their choices of listings by area without the need to use the search bar, but many chose the search bar instead. Most participants did not understand what this feature could do from its appearance, thus they did not click on the feature’s icon.
The draw feature was easily skimmed over
by most participants
Recommendation
For the draw feature to be more discoverable, our recommendation is to change the naming convention it has, the placement of it, and make it appear more uniform to the buttons next to it. To begin, “draw” is not a clear explanation to what the feature can do. Having a more descriptive name would clarify the function of this feature as well as give a brief explanation to the user on how it can be helpful. The placement of the button can also go above the “Schools” button, as the “Draw” button is more closely related to the zoom in and out features in terms of functionality. By grouping these features together, it creates more cohesion that these are meant to manipulate the map compared to showing/finding information. Lastly, Zillow can change the default color of the button. The button is highlighted on default which makes it easily missed or seemingly daunting to the user. Typically when the button is highlighted in blue, it means the feature is in use. This breaks the mental model users have of how the buttons on Zillow operate, so creating uniformity may benefit this feature to become more discoverable.
Key Finding 2: Poor discoverability in the "Draw" feature's functionalities
Severity: Priority 2
Functionalities within draw feature were also not communicated effectively to our participants to make the most use out of this feature. It was not intuitive in using it to select areas and narrow down search results. When using this feature, many participants did not realize they need to click on “apply” after drawing a region. Additionally, they did not know that they could draw multiple regions before clicking on “apply”.
Draw feature functionalities are not effectively communicated, as most participants had trouble clicking on the "apply" button to confirm selected regions
Recommendation
When the user goes into “draw” mode, the only instructions available were in a gray banner at the top of the map, which was unnoticeable by most of our participants. Changing the coloring of the banner will improve noticeability as well as bring the user’s attention to the text that explains how to use the feature as well as the “Apply” button that is needed to complete the task. Another improvement that can be implemented is to include a quick tutorial if it is the user’s first time using this feature. A small animation of a cursor circling an area of the map and clicking apply should repeat in a loop until the users clicks on the map to draw themselves. This allows for a visual instruction to prompt users to successfully make the most out of the feature.
Key Finding 3: Inconsistency and poor discoverability in the save feature
Severity: Priority 1
Overall, the "save" feature was liked by all participants. They found that it was very easy to click on the heart next to each listing to save them. However, on the results page, they recognized that some listings did not have a heart icon to let them save. After further digging, only one participant was able to figure out how to save these types of listings.
Inconsistent location and appearance
Supporting Data
Chart 1: Average task duration without outlier
Task 3 (saving an apartment/multilistings) ended up being the longest on average. The longer length of time stemmed from users who encountered multi-listings and could not find save option for it.
Chart 2: Participants found task 3 useful and majority would repeat. When it came to finding information needed, some participants found it easier than others because they did not encounter multi-listings
The users who disagreed or were neutral with the statements “It was easy to find the information I needed” and “I found it easy to find every feature need for the task” are the ones who encountered multi listings. It was not easy to find the save option for the listings that did not have a heart available in the search results.
Recommendation
We recommend that all listings have the "heart" feature available on the surface of the results page. For listings that have multiple units, sub listings such as apartments or multiple houses showing in one result, we still recommend to have the heart feature available. Once the users click on it, they should be prompted to pick which unit or have the option to save the entire listing. The user should also have the option to “save all” which would allow them to save all the listings they have filtered down to.
See the full report here.
Conclusion
We hope that the Zillow design team would find our findings helpful for site improvement and redesign. As for next steps for this project, we would like to test with a wider participant group to gain more perspectives and confirm our current findings. For the continuation of this project, we would like to have a more diverse group of college students coming from different backgrounds and with different levels of computer literacy. It would also be desirable to have a fairly even number of male and female students and a bigger age range of participants in our usability studies. We would learn more about our user and their experiences through interacting with participants. Lastly, for our next step, we would like to conduct competitive analyses of market competitors to help us find more insights and provide Zillow with stronger recommendations to enhance current features, which would in turn retain customers.