Zillow Usability Case Study

zillow.png
ROLE

Researcher, Usability Study Facilitator

DURATION

10 weeks (Oct - Dec 2018)

DELIVERABLES
  • Research Findings

  • Surveys

  • Affinity Diagrams

  • Usability Observations

  • Cognitive Walkthrough

  • Heuristic Evaluation

  • Usability Testing

  • Severity Rating

  • Preliminary Analysis

  • Usability Study Report

TEAM MEMBERS

Reksha Rathnam

Kelly Xu

Monica Posluszny

Context

Zillow is a real estate website that can be used to find homes to buy, rent, or sell. Not only does it have important information regarding the residences but also interactive features such as saving listings and booking tours that attempt to create a user-friendly experience overall. For this study, we specifically look at the renting page of Zillow. We plan to exclusively test Zillow’s website in meeting the needs of college students in finding rental homes because they make a big part of the renter population.


We conducted a usability test with 4 tasks to test how users were able to narrow down search results based on preferences,  find desired housing in a specific location, save housings, and contact the housing lister through the website. We assessed the effectiveness, discoverability, and satisfaction of the site features through analyzing how well users were able to accomplish goals. Findings and recommendations from the study are intended to help the Zillow product team improve current features and increase the retention rate of users in using Zillow to rent homes.

Research Questions

We were interested in testing the Zillow website because there are many components that are part of the housing search journey and we plan to focus on the renter perspective. The goal of this usability study is to test how the Zillow website is meeting the needs of college students in their attempt to find housing to rent. Below are our research questions:

 

  • Effectiveness/User Control: How easy is it for users to input their criteria for their desired home?  Can the user find a residence that fits their criteria?

  • Discoverability: Can users utilize Zillow’s features to narrow down and rediscover their searches?

  • Satisfaction: How can we improve the overall satisfaction of Zillow users when looking for rental housing options?

Overall Findings

Through our usability study, we found successful elements of Zillow as well as features that could be improved. The major findings from the study are issues related to discoverability and effectiveness of site features, such as the Draw and Save features for apartment listings. Participants had trouble finding and recognizing such features, and therefore were unable to fully utilize all of Zillow’s features and functionalities. Some features were ineffective due to being inconsistent with others, not visible enough for users, or not intuitive to be used for the right purposes. Additionally, participants had to use external sites in order to complete some tasks. Overall, the site could be improved by reducing confusion, miscommunication, and inconsistencies, which would boost the overall user experience in using Zillow to find rent. We also proposed steps to take in the future that would help improve the above areas and provided more in-depth design recommendations.  

Participant Profile

Environment/Testing Conditions/Roles

Our team reserved a study room in the Odegaard library for the ease of location and capacity to comfortably seat 3-4 people. We used UserZoom to help conduct the tasks. We had two laptops present, one for the participant to perform tasks and the other for the notetaker and facilitator to begin, view, and end the UserZoom test. During the test, we had one moderator who facilitated the tasks and one note-taker from our team who took notes as well as filled out our data logging tables. If we had an additional team member available, the member would act as another observer and take general notes.

Tasks

The below are tasks with scenarios completed by our 8 participants during the usability testings.

 

Task 1: Find homes in a city with basic requirements
You are interested in renting a place in the future with a roommate, and you want to have a basic idea about the current renting rates of houses and apartments in the city of Seattle. You also own a cat. Using Zillow.com, look for available house/apartment rentals in Seattle that allow pets with a price range of $1,000-3,000 and 2 baths.

Task 2: Filter out homes around a specific location 
You want to specifically live near the Seattle University. You are open to homes in all around the campus. You want to narrow the results to just show homes around the Seattle University. 

Task 3: Save desired homes and be able to find them
You have found a few listings that you are interested in and you want to keep them for later viewing. Save at least 5 different listings and locate them within the Zillow site.

Task 4: Contact lister 
You have found a home that really suits you and would like more information about it. Find contact info of the lister and contact the lister by sending a message to ask if it is possible to move in next week.

Results

Successes

  • Filters allow participants to narrow down search results.

 

 

 

 

Our participants overall found that the filter bar, which follows global website filter standards and practices, was fairly easy and simple to use. All 8 participants were able to use this feature to put in home requirements and narrow down search results for their first task. Having this feature reduces information overload and shortens the home search process. The average task duration for doing this task was the shortest out of all tasks. Overall, the filters greatly improve the efficiency of finding a home that meets all user needs. 

Interaction with filters

  • The Save feature is helpful for going back to view previously saved listings.


8 out of 8 participants agree or strongly agree that they found saving homes useful. They indicated that it is a simple process to save listings by clicking on the heart icon. It is also convenient to find saved listings in the upper right corner as a global feature, which also updates the number of saved listings. Additionally, having saved listings as hearts that show on the map is helpful for locating housing locations, which improves the discoverability of saved homes.  

Using the save feature

Areas for Improvement and Recommendations

Below are the severity ratings of the problems that may occur during the usability testing. We categorize each problem to a severity level based on discoverability, effectiveness and satisfaction  to prioritize issues and determine the order in which these issues would need to be addressed.

Priority 1 = A critical problem that needs to be resolved immediately, is preventing the user to accomplish tasks.

Priority 2 = This problem is confusing/troublesome for the participant when doing the task, but the participant was still able to complete the task.

Priority 3 = A minor problem that doesn’t impact the user’s ability to complete the tasks, but needs to be fixed to improve the overall experience in using the site.

Key Finding 1: Poor discoverability of the "Draw" feature icon on the webpage
Severity: Priority 1


The draw feature is available on the map, but it is not noticeable by many participants. For task 2, the feature is meant to help participants narrow down their choices of listings by area without the need to use the search bar, but many chose the search bar instead. Most participants did not understand what this feature could do from its appearance, thus they did not click on the feature’s icon. 

The draw feature was easily skimmed over

by most participants

Recommendation

For the draw feature to be more discoverable, our recommendation is to change the naming convention it has, the placement of it, and make it appear more uniform to the buttons next to it. To begin, “draw” is not a clear explanation to what the feature can do. Having a more descriptive name would clarify the function of this feature as well as give a brief explanation to the user on how it can be helpful. The placement of the button can also go above the “Schools” button, as the “Draw” button is more closely related to the zoom in and out features in terms of functionality. By grouping these features together, it creates more cohesion that these are meant to manipulate the map compared to showing/finding information. Lastly, Zillow can change the default color of the button. The button is highlighted on default which makes it easily missed or seemingly daunting to the user. Typically when the button is highlighted in blue, it means the feature is in use. This b