At the Inman Real Estate Connect conference earlier this month in San Francisco, one of the coolest new sessions was something called Connect Create.
Developers were challenged to create something cool and useful within 48 hours and two teams — from real estate tech companies Diverse Solutions and Real Estate Webmasters — stepped up to the challenge.
Diverse Solutions came up with an online tool dubbed "Agent Scouting Report," and I believe it has some great potential (see related article).
The folks at Diverse Solutions wrote up a great post explaining what they did, why they did it, and what they think it means; I recommend reading it in full. I can’t imagine being a Realtor today without thinking through what a tool like this means for my business, and what I have to do to prepare for it.
What it is
In brief, Agent Scouting Report is an agent rating system that uses multiple listing service data rather than purely subjective opinions and client surveys. Diverse Solutions used 10 years’ worth of MLS data to compute things like "Salesmanship" (average days on market), "Experience" (tenure in the MLS), "Knowledge of Market" (how many times did the price drop from the initial listing date), and the terribly misnamed "Popularity" (how many homes sold in last 180 days, the past year, and the past two years).
The team took all of the data, weighted it, and came up with a star-based ratings system similar to Yelp and other rating Web sites: click here to view a screenshot.
There is an element of subjectivity involved in the algorithm, but it’s hard to argue with black-and-white data. Either you sold 10 homes in the last 180 days or you did not; either the listing you represented did drop the price from initial listing or it did not. There just isn’t much of a gray area there.
This is the equivalent to the baseball trading cards that have obsessed boys — whether athletically or intellectually inclined — for decades. In one tiny piece of paper, you get a player’s career up to that point summarized in a neat chart that looks something like the one pictured here.
You can see at a glance that this ballplayer was a monster. Look at 1927: .373 batting average; 47 home runs; 175 runs batted in (RBIs); an on-base percentage of .474, which means he got on base roughly half the time he came up to bat. And you also see that this was not an aberration — a freakish one-year anomaly. From 1927 to 1934, this guy had at least 126 RBIs every year and averaged 157 RBIs. That’s a model of consistency.
Numbers aren’t everything, but they are something
These stats do not tell the whole story of one Lou "Iron Horse" Gehrig. They do not tell us what kind of a teammate he was in the locker room. They don’t tell us whether he liked to party after games, or went home for a scene of domestic bliss with his wife.
The numbers don’t tell us that one possible reason for such godly stats may be that he played with Babe Ruth and the "Murderers’ Row," and that 1927 was one of the greatest seasons of any baseball team in history. …CONTINUED
Opposing pitchers couldn’t pitch around him, with Earle Combs and Babe Ruth hitting ahead of him and Bob Meusel and Tony Lazzeri hitting after him. These numbers don’t tell you about Gehrig’s 2,130-game consecutive start streak — a 14-year run and a record unbroken for more than a half-century.
So the numbers aren’t everything, especially when it comes to judgment calls. This is the heart of virtually every sports talk radio show and the subject of infinite bar conversations: "Who’s better?"
"Who’s better: Lou Gehrig or Babe Ruth?" "Who’s better: Joe DiMaggio or Mickey Mantle?" And they get weird, too: "Who’s better in cold weather, facing a left-handed pitcher, with a man on first and third with two outs: 1927 Gehrig or 2003 (Albert) Pujols?"
These arguments never end, and no one can actually win them, because words like "better" or "best" contain value judgments that are different for each individual.
And yet, these arguments always, always, always invoke statistics. No one disagrees on the statistics — they can’t, because those numbers are an objective record of achievement: number of at-bats, number of hits, number of home runs, number of runs batted in (RBIs), batting average, etc. These are not up for debate; it is a fact that Pujols hit .359 in 2003.
What is up for debate is interpretation. Even if the data is objective, interpretation is always subjective. Is it better to have hit for average or for power? Can you put Pujols, playing in the modern era of millionaire athletes, on the same page as Lou Gehrig, who was playing during the Great Depression and made a modest living?
So … real estate
Let’s bring it back to real estate. The data-based agent performance reports are coming. Agent Scouting Report is but the latest offering, and by no means the last. Even the people at Diverse Solutions know that they could do so much more with access (and permission) to use all of the MLS data — especially if they had more than 48 hours to build the thing.
The trend toward transparency and openness is real; there’s likely no way to put this genie back in its bottle. The agent ratings are coming, like it or not.
The real question, then, is how you prepare for it.
If your stats are amazing, like a Gehrig or Pujols, then preparation is simple. Talk about how great your stats are, and point to them as an indicator of why someone should use you to sell a house. Big surprise, right?
If your stats suck across the board — your days on market (DOM) are far too long, you don’t know how to price a listing, and you can’t do many transactions — consider alternative careers. Run for Congress, maybe.
But if you’re part of the vast middle with so-so stats, some better than others, then what? Seems to me the answer has to be adding context and shaping interpretation. For example, which is better: to have shorter days on market, more transactions, but show multiple price drops? …CONTINUED
Or is it better to have all sales over the listing price, but with eight months of days on market and six transactions every year? The answer likely depends on your market, your target client, etc.
The second agent may be perfect for super-luxury properties; the first may be fantastic for low-income work.
Some analytic tools to aid you in your quest for interpretation:
1. Segmentation: Make sure that your data is being compared with the data of others who operate in a similar market area, work with similar property types, etc. Diverse Solutions did some of this by going into a bit of geographic segmentation.
Where you might want to do more segmentation is by tenure (Does it make sense to compare someone with three years of experience with someone who has 30?), by market segment (distressed vs. conforming vs. luxury, for example), by further geographic segmentation (agents in my neighborhood vs. my "market," which may be too large), etc. The point is to interpret your statistics with others who are comparable to you.
Warning: Don’t get silly with segmentation — "No. 1 among left-handed female German-speaking Polynesian Realtors under 45" is definitely particular, but may not be particularly impressive.
2. Identify anomalies: If there is some specific event that is really out of the norm (and of course, you’d have to define what this means), then try to eliminate such anomalies when interpreting data.
One could argue, for example, that any data from 2002 to 2006 should be tossed out because that was the height of the bubble. Or more locally, perhaps a major new manufacturing plant opened up in your area, resulting in a flood of newcomers two years ago … and you started in the business last year. Point out such anomalies in interpreting your data.
3. Trending: If your statistics are average, but you can show a consistent trend of improvement, make use of that. Maybe your DOM stats are below average, but you’ve been improving every year for the past three years.
Use the trend to your advantage, and tell a story about how you’re learning in a tough market, making adjustments, and how you can serve the client’s needs better today than you did yesterday.
There are other ideas, of course. The point is to explain, contextualize, and interpret — while remaining truthful, transparent, and honest so that you can either use the data to your advantage or minimize the damage the data might do to the full picture of your value as a trusted real estate adviser.
You might as well get prepared now, because objective agent scorecards and ratings are coming. Don’t get caught off guard when they arrive.
Robert Hahn is managing partner of 7DS Associates, a marketing, technology and strategy consultancy focusing on the real estate industry. He is also founder of The Notorious R.O.B. blog.
***
What’s your opinion? Leave your comments below or send a letter to the editor.
