How computer vision is reshaping real estate search

Homebuyers don’t measure “home” in square feet. I learned that during my time as an agent with Keller Williams Realty.

Instead, my buyer clients described their home preferences in terms of large backyards, open floor plans and lots of natural light. But the language they used to describe their ideal house didn’t translate into any online search experience available at the time.

This frustration was, in part, the impetus for founding RealScout.

Last year, we took a major step in realizing this central mission by implementing a powerful new technology called computer vision.

Real estate listing photos carry vast amounts of valuable information for homebuyers — information that, until recently, was not understandable by computers and thus not easily searchable.

[Tweet “They say that a photo is worth a thousand words. In the MLS, it’s worth zero words.”]

Computer vision can analyze listing photos and automatically identify hundreds of features, giving consumers the ability to filter searches with this and other buyer-centric information.

Computer vision technology: All around us now and in the future

The application of computer vision technology is familiar to most of us and will become increasingly so. It’s what Google’s self-driving cars use to distinguish pedestrians from bike riders and countless other factors in a dynamic road environment. Facebook uses it to suggest that the woman next to you in the photo you recently posted is your college friend.

Trulia’s been using it, too; the company announced that it has recently moved from “explicit search filters” to “implicit search filters.”

This capability provides agents with information such as consumers’ neighborhood-specific feature preferences and trends, based on true interests and preferences discovered from search behavior.

This technology, when paired with an agent’s expertise, is setting a new standard in the consumer homebuying experience. Only human agents using the best technology and machine intelligence can most successfully guide their clients through the complex homebuying process.

What is computer vision?

Computer vision is the science of teaching computers how to understand the visual world. Since it’s so easy for people to glance at photographs of a home and accurately identify and describe what they see — room type, open layout, natural lighting, new appliances — it might be surprising that computers have only recently begun to do the same.

Recent advances in computer vision have been driven by innovations in deep learning, a technique by which computers learn to solve problems and improve over time, similar to the process experienced with the human brain. Just as parents teach children by pointing out objects in pictures and saying “this is a cat,” so too are programmers training computers by providing examples.

[Tweet “The same tech that powers Facebook’s face-tagging is starting to change the face of home search.”]

Teaching computers how to understand pictures is a lot of work. In real estate, it’s currently accepted that almost 100,000 manually tagged photographs are needed to help computers identify the nine different room types in a house — kitchen, dining room, living room, bedroom, bathroom, etc. — as accurately as humans can.

Additionally, each subcategory — oven, refrigerator, countertop material and so on — requires thousands more hand-annotated photos.

Delighting clients with computer vision

In 2012, we started manually creating the types of experiences that we’d eventually scale using computer vision. In the same way Google and Facebook have trained their models with mind-boggling amounts of data, we built a vast data-set tailored to real estate. We have used human taggers to label more than 7 million listing photos with dozens of elements ranging from room type to features, attributes and other key information.

[Tweet “Computer vision can determine a listing photo’s room type with the same accuracy as human taggers”]

Sample Use Case #1: Compare Listings Side by Side, Room by Room

Compare is one of the unique applications of computer vision. What makes this feature special is the ability to display photos of particular rooms side by side, enabling buyers to easily compare properties.

Sample Use Case #2: Reinventing the Photo Carousel

Portals will likely upgrade photo carousels to cluster a listing’s photos by room type, allowing homebuyers to quickly navigate between the rooms most important in their search.

Computer vision and the future of real estate search

Whether you find it helpful or creepy when Facebook’s face-tagging system suggests the name of a friend or family member in a posted photo, this same technology is starting to change the face of real estate and home search.

The use cases above represent only the tip of the iceberg of innovative user experiences that will be made available through computer vision. Fundamentally, computer vision unlocks a treasure trove of visual information and makes it searchable, manipulatable and consumable, not only increasing the overall quantity of property data, but increasing its quality.

[Tweet “Computer vision unlocks a treasure trove of listing data.”]

This means that soon, agents can expect computer vision to enhance every part of their business.

For example, property insights extracted from photos can power highly detailed CMAs for pricing properties, or enable high-accuracy look-alike search for agents looking for homes with similar features. Even information like square footage, construction quality and neighborhood conditions can be extracted through computer vision techniques, increasing the transparency of information for agents and clients.

What does the future of computer vision hold for real estate search?

Interior

• What are the fine-grained room classifications: wine cellar/poker room/nursery/deck/patio?
• What are the fine-grained feature classifications: granite/marble/concrete/tile countertops?
• What is the exact size of every room?
• What will the dining room look like if I add french doors to the back patio?
• Has the house been professionally staged?

 Exterior and neighborhood

Answers to those questions will help our industry:

What does this mean for agents and brokers?

Does computer vision represent a threat to the agent value proposition? Absolutely not.

Increased property information, in fact, empowers agents to be even greater domain experts. Making sense of data and, more importantly, making informed decisions through data, are subtle and highly personal skills machines cannot acquire. Agents and brokers who effectively integrate these technologies will be able to better cater to their clients’ needs, and will naturally be in higher demand.

Computer vision is still in its early days, and the full impact of the technology is unknown. But one thing is for sure, pioneering the use of this technology is a rare opportunity for our industry to take the lead in adopting bleeding-edge innovation that drives real value for our clients.

Thanks to Helena Winkler, Arthur Kaneko and Chris Conley for their contributions to this article. 

Andrew Flachner is the CEO at RealScout, the collaborative home search platform that empowers real estate agents to close more deals faster. He is also author of the Offer Generation Playbook.

Email Andrew Flachner.