• By teaching computers how to understand the visual world, MLS photos will soon unlock a treasure trove of listing data.
  • Companies like RealScout and Trulia have already begun using computer vision to add new functionality to home search.
  • In the future, computer vision may be used to automatically predict marketability of listings and sale price.

Homebuyers don’t measure “home” in square feet. I learned that during my time as an agent with Keller Williams Realty.

Instead, my buyer clients described their home preferences in terms of large backyards, open floor plans and lots of natural light. But the language they used to describe their ideal house didn’t translate into any online search experience available at the time.

This frustration was, in part, the impetus for founding RealScout.

Last year, we took a major step in realizing this central mission by implementing a powerful new technology called computer vision.

Real estate listing photos carry vast amounts of valuable information for homebuyers — information that, until recently, was not understandable by computers and thus not easily searchable.

[Tweet “They say that a photo is worth a thousand words. In the MLS, it’s worth zero words.”]

Computer vision can analyze listing photos and automatically identify hundreds of features, giving consumers the ability to filter searches with this and other buyer-centric information.

Computer vision technology: All around us now and in the future

The application of computer vision technology is familiar to most of us and will become increasingly so. It’s what Google’s self-driving cars use to distinguish pedestrians from bike riders and countless other factors in a dynamic road environment. Facebook uses it to suggest that the woman next to you in the photo you recently posted is your college friend.

Google Self-Driving Cars Use Computer Vision

Google’s autonomous vehicles use computer vision to identify objects

Trulia’s been using it, too; the company announced that it has recently moved from “explicit search filters” to “implicit search filters.”

This capability provides agents with information such as consumers’ neighborhood-specific feature preferences and trends, based on true interests and preferences discovered from search behavior.

This technology, when paired with an agent’s expertise, is setting a new standard in the consumer homebuying experience. Only human agents using the best technology and machine intelligence can most successfully guide their clients through the complex homebuying process.

What is computer vision?

Computer vision is the science of teaching computers how to understand the visual world. Since it’s so easy for people to glance at photographs of a home and accurately identify and describe what they see — room type, open layout, natural lighting, new appliances — it might be surprising that computers have only recently begun to do the same.

Recent advances in computer vision have been driven by innovations in deep learning, a technique by which computers learn to solve problems and improve over time, similar to the process experienced with the human brain. Just as parents teach children by pointing out objects in pictures and saying “this is a cat,” so too are programmers training computers by providing examples.

[Tweet “The same tech that powers Facebook’s face-tagging is starting to change the face of home search.”]

Teaching computers how to understand pictures is a lot of work. In real estate, it’s currently accepted that almost 100,000 manually tagged photographs are needed to help computers identify the nine different room types in a house — kitchen, dining room, living room, bedroom, bathroom, etc. — as accurately as humans can.

Additionally, each subcategory — oven, refrigerator, countertop material and so on — requires thousands more hand-annotated photos.

Delighting clients with computer vision

In 2012, we started manually creating the types of experiences that we’d eventually scale using computer vision. In the same way Google and Facebook have trained their models with mind-boggling amounts of data, we built a vast data-set tailored to real estate. We have used human taggers to label more than 7 million listing photos with dozens of elements ranging from room type to features, attributes and other key information.

[Tweet “Computer vision can determine a listing photo’s room type with the same accuracy as human taggers”]

Sample Use Case #1: Compare Listings Side by Side, Room by Room

Compare is one of the unique applications of computer vision. What makes this feature special is the ability to display photos of particular rooms side by side, enabling buyers to easily compare properties.

Compare Rooms Side by Side

Room by room property comparison enables homebuyers to quickly view rooms of their choice side by side, without needing to flip through multiple photo carousels.

Sample Use Case #2: Reinventing the Photo Carousel

Portals will likely upgrade photo carousels to cluster a listing’s photos by room type, allowing homebuyers to quickly navigate between the rooms most important in their search.

Feature Carousel

Today, homebuyers can quickly access a listing’s photos by room type. Image courtesy of RealScout.

Computer vision and the future of real estate search

Whether you find it helpful or creepy when Facebook’s face-tagging system suggests the name of a friend or family member in a posted photo, this same technology is starting to change the face of real estate and home search.

The use cases above represent only the tip of the iceberg of innovative user experiences that will be made available through computer vision. Fundamentally, computer vision unlocks a treasure trove of visual information and makes it searchable, manipulatable and consumable, not only increasing the overall quantity of property data, but increasing its quality.

[Tweet “Computer vision unlocks a treasure trove of listing data.”]

This means that soon, agents can expect computer vision to enhance every part of their business.

For example, property insights extracted from photos can power highly detailed CMAs for pricing properties, or enable high-accuracy look-alike search for agents looking for homes with similar features. Even information like square footage, construction quality and neighborhood conditions can be extracted through computer vision techniques, increasing the transparency of information for agents and clients.

What does the future of computer vision hold for real estate search?


  • What are the fine-grained room classifications: wine cellar/poker room/nursery/deck/patio?
  • What are the fine-grained feature classifications: granite/marble/concrete/tile countertops?
  • What is the exact size of every room?
  • What will the dining room look like if I add french doors to the back patio?
  • Has the house been professionally staged?

 Exterior and neighborhood

  • What is the architectural style?
  • When will I need to repair/replace the roof?
  • What direction is the house facing? What is the solar orientation?
  • Is the house on a cul-de-sac? Is there a sidewalk?
  • How green is the surrounding neighborhood?
  • How large is the front yard? Backyard?
  • How big is the driveway? How much snow will I have to shovel?

Answers to those questions will help our industry:

  • Provide the best online exploration experience for home buyers
  • Predict days on market and sale price
  • Make suggestions to listing agents on how to optimize the sale price
  • Make suggestions to buyers agents on how to craft compelling offers

What does this mean for agents and brokers?

Does computer vision represent a threat to the agent value proposition? Absolutely not.

Increased property information, in fact, empowers agents to be even greater domain experts. Making sense of data and, more importantly, making informed decisions through data, are subtle and highly personal skills machines cannot acquire. Agents and brokers who effectively integrate these technologies will be able to better cater to their clients’ needs, and will naturally be in higher demand.

Computer vision is still in its early days, and the full impact of the technology is unknown. But one thing is for sure, pioneering the use of this technology is a rare opportunity for our industry to take the lead in adopting bleeding-edge innovation that drives real value for our clients.

Thanks to Helena Winkler, Arthur Kaneko and Chris Conley for their contributions to this article. 

Andrew Flachner is the CEO at RealScout, the collaborative home search platform that empowers real estate agents to close more deals faster. He is also author of the Offer Generation Playbook.

Email Andrew Flachner.

Show Comments Hide Comments
Sign up for Inman’s Morning Headlines
What you need to know to start your day with all the latest industry developments
By submitting your email address, you agree to receive marketing emails from Inman.
Thank you for subscribing to Morning Headlines.
Back to top
Log in
If you created your account with Google or Facebook
Don't have an account?
Forgot your password?
No Problem

Simply enter the email address you used to create your account and click "Reset Password". You will receive additional instructions via email.

Forgot your username? If so please contact customer support at (510) 658-9252

Password Reset Confirmation

Password Reset Instructions have been sent to

Subscribe to The Weekender
Get the week's leading headlines delivered straight to your inbox.
Top headlines from around the real estate industry. Breaking news as it happens.
15 stories covering tech, special reports, video and opinion.
Unique features from hacker profiles to portal watch and video interviews.
Unique features from hacker profiles to portal watch and video interviews.
It looks like you’re already a Select Member!
To subscribe to exclusive newsletters, visit your email preferences in the account settings.
Up-to-the-minute news and interviews in your inbox, ticket discounts for Inman events and more
1-Step CheckoutPay with a credit card
By continuing, you agree to Inman’s Terms of Use and Privacy Policy.

You will be charged . Your subscription will automatically renew for on . For more details on our payment terms and how to cancel, click here.

Interested in a group subscription?
Finish setting up your subscription