
How Bad Are Zillow “Zestimates”? - duvora
http://blog.duvora.com/exactly-how-bad-are-zillow-zestimates-case-study/
======
mcherm
I am quite disappointed in the research that went into this article.

(1) Author discovers that Zillow has a specific claim about accuracy: "around
90% of the listing Zestimate is “Within 20% of Sale Price” of the home".
Author then gripes that this is "a really large margin for error". Really?
Because I was prepared to praise Zillow for such honest and clear reporting.
It only costs a few hundred dollars _per home_ (Zillow covers nearly every
home in the US) to get a professional to produce an assessment, and these
assessments are usually only a little more accurate than what Zillow is
claiming.

(2) Author pulls up tables from Zillow showing a median error. Which is an
excellent way to analyze this and is (I think) extraordinarily small. Author
then proceeds to do other stuff.

(3) Author decides to verify the values by checking a total of NINE houses. Is
the accuracy of that test expected to impress us? Furthermore, one of these 9
is rejected because "(obviously a data glitch or input error. Dismiss this
result)" \-- but that's the whole POINT of using median error; it isn't
affected by outliers.

(4) Author finds that all of the values (all 3 of them) in Staten Island were
pretty far off. Author asks professionals who say that the local market o
Staten Island is behaving quite unusually at the moment. Reporting on this is
the only thing I think the author did right.

~~~
DennisP
A long time ago I got a real estate appraiser's license. Here is the procedure
for appraising real estate, as taught by my state:

1) Pick three homes that you feel are comparable.

2) Pick out the material differences between those homes and your client's

3) Make up a value for each of those differences, and adjust accordingly

(Unofficially speaking) If you don't like the number you come up with, just
make different decisions in steps 1-3 until you get the number you want.

~~~
robbyking
In the late 90's I worked for an online mortgage broker, and 9 times out of 10
when we ordered an appraisal, the appraiser would ask us what the selling
price was and return an identical figure for their estimate.

~~~
leereeves
Amazing that people still pay for that.

~~~
URSpider94
"People" don't. It's a required part of the due diligence to get a mortgage.
No appraisal, no loan.

For the most part, the banks and brokers re-sell those loans immediately, so
they don't stay on their books. All they need to protect them from lawsuits
later is proof that they followed the process to the letter -- which involves
getting a written appraisal.

No mortgage broker or Realtor ever got rich by picking a fight with the
appraiser to lower the appraised value ... there is literally nobody present
at the table who has an incentive to push the value down.

~~~
leereeves
I understand that it's required, and that organizations often follow processes
long after they're revealed to be foolish.

But it's been a decade now since the housing bubble revealed that appraisals
are meaningless.

Why has no one eliminated them?

~~~
mcherm
They are required by law or by long-standing policy in a government-run body
(which is practically the same thing). Laws DO change to keep up with the
times, but they do so rather slowly. In 50 to 100 years, if there is
continuous clear evidence that home assessments are highly unreliable and not
particularly profitable to anyone, then you should expect them to be
eliminated.

------
jannotti
I don't have an opinion on whether zestimates are accurate. But this blog post
has 9 samples from only 3 cities. And the author seems to be making claims
based on that. This is a lame "thought leadership" post as a subtle ad for his
company.

At least give me 10-20 homes in each of 10-20 cities if you are trying to
convince me of something systematic.

~~~
Bartweiss
The only really interesting thing I saw in this post was the discussion with
realtors. It's significant that Zillow uses long-term averages, and therefore
can't price in abrupt market shifts even once they're common knowledge. If
nothing else, it implies that "Zestimates" are a bit dubious in tight urban
markets where high variance is more possible. Of course, it also looks like
Zillow is aware of that challenge.

The ostensibly data-driven part was by far the least convincing section - as
soon as I saw the sample size I decided it was nothing more than a starting
spot-check.

~~~
cestith
Perhaps Zillow is more interested in showing the longer-term market value of a
home so buyers know if they're being asked for a high demand-driven price. A
tight market like that can spring up quickly and fade away just as quickly. At
some point most people are going to look to resell or to at least not be
forever upside down because they bought during a tight market.

------
donmaq
SF Bay area, I've had 3 formal house assessments, & each time they "start
with" Zillow et al to look at "reasonable values in the neighborhood". Which
seriously affects your ability to refi, etc.

So Yeah Zillow has an effect. It's worth it to create a login & enter in your
upgrades, so your ZESTIMATE goes up.

~~~
exhilaration
_It 's worth it to create a login & enter in your upgrades, so your ZESTIMATE
goes up._

Very nice, thank you, my Zillow estimate is now $5965 higher. We have no
intention to sell but if we do, at least I can start the negotiations at a
higher price point by pointing to Zillow.

~~~
teuobk
Incredible -- I just did the same, and my house's "zestimate" went up about
$20,000.

Of course, if I compare my house to actual sales in the neighborhood over the
past few months, that's probably 10% high, so correcting Zillow's data about
my house made the zestimate less accurate, not more.

~~~
Tempest1981
Try changing the number of bedrooms. That used to have an effect, even for the
same square footage. In one case, 2 bedrooms came out higher than 3.

------
20years
Realtors who feel threatened by Zillow love to complain about Zestimates.
Zillow has made it clear time and time again that they are not 100% accurate
and even allow homeowners to edit them.

Zestimates are a tool that homeowners like to use. Zillow knows that. Good
Realtors won't feel threatened by it but instead leverage it to provide
prospects with a much more complete and accurate estimate.

Realtors need to stop blaming Zillow & Zestimates for their own weaknesses. It
is getting old.

~~~
acdha
Zillow deserves a fair share of the blame because they prominently feature a
single number with dollar-level precision and bury the caveats multiple clicks
away on a separate page.

Some basic UI changes could avoid all of that: e.g. instead of showing
$123,456, show a range like $115,000-135,000, and keep the last few digits at
zeros based on their estimated error rates. Similarly, their graphs could use
shading to make it clear that they're showing the mid-point of a range rather
than a discrete value.

~~~
brentm
> Some basic UI changes could avoid all of that: e.g. instead of showing
> $123,456, show a range like $115,000-135,000

So true. When you know that you are not displaying an exact dollar amount you
should not use exact dollar values. Exact dollar values afford exactness even
though the branding is an estimate.

~~~
pigpigs
This might very well backfire on them. The buyers will want to buy at the
lower price and the seller would want the higher price. Both parties end up
being unhappy even if the end price was within the given band.

------
drewbuschhorn
I bought a home last year, and each realtor I talked to brought up how
terrible Zillow was before I ever mentioned it. If you look at the WaPo
article all the comments are by other realtors, so I'd take those comments
with a grain of salt. Oddly, my mortgage guy never looked at Zillow (that I
saw) for figuring out values, so _shrug_.

I'm sure the data is inaccurate, but it's somewhere to start the conversation
and let people feel like they have some power in the process of buying and
selling homes.

And as people like to say about Uber, realtors are welcome to make a competing
site that's as free and easy to use.

~~~
joeax
My bro-in-law became a real estate agent a couple years ago. Recently we were
discussing Zillow, and he went into a tirade about how terrible it is. I just
mentioned that it's a good tool to gauge the price trends of my neighborhood,
but he began countering with canned, carefully scripted responses that sounded
like indoctrination from his broker/employer. I just sighed, before the topic
turned to how important it is to pay the full 6% when listing your house.

The real estate industry is one of the most protectionist industries out
there, and any threat of disruption, no matter how minor, is met by an army of
vitriol.

~~~
shostack
What were the arguments for the 6%? There is a broker in the Bay Area who
advertisers that they take half that but the complaints against them see to
point to the fact that you'll only get offers from other affiliated agents,
and many agents won't share the listing with clients if they can avoid it
because they want their full commission.

I'm the Bay Area I feel like a seller's agent is grossly overpaid for what is
often a weekend of open house showings and then fending off multiple above
list price zero contingency offers.

~~~
joeax
I've sold houses using discounted brokers and he hates that. He said that
buyer agents will dissuade buyers from looking at houses offering less than
"standard 3%" to the listing agent (another protectionist tactic).

I countered that I don't care. In this day and age buyers are savvy enough to
jump on Zillow and Redfin and find my house, and they will demand they see it.
If I hired a buyer agent and they tried to talk me out of showing a house over
a commission I'd fire them right there.

------
mi100hael
Zillow's Zestimates are particularly obnoxious because they provide little
detail regarding why the values change and they retroactively revise them. I
bought a house this summer that had an inexplicable drop in estimated value of
about 15% a month or two before the house was listed. I paid closer to the
previous estimated value than the post-drop value. Now when I look at the
house on Zillow, the drop is nowhere to be seen and the past year is a smooth,
barely sloping line. Never was there any indication of _why_ they had a 15%
drop, which was particularly frustrating when trying to determine how much to
offer for the house.

~~~
talloaktrees
This has happened to me as well.

------
dvdhnt
Interesting. While I agree the results are likely to be inaccurate, this case
study targets only the three largest cities in the US. Urban centers probably
aren't the best population from which to sample.

In a case like this, we'd probably want to use locations based on number of
home sales; and even then, we'd need to account for the amount of time between
homes changing hands i/e houses that are flipped. There's likely something to
be said about the difference between the nation's average sale price and each
area's average sale price; assuming that may affect Zillow's estimations.

Anyways, I like that you're doing studies like this, regardless.

~~~
keithly
I'd guess part of why those cities were chosen were to have enough data points
to draw conclusions, since all the examples are from a narrow time frame.

------
ahulak
There are ~100M homes, ~3k counties, ~27k zipcodes, and ~300k neighborhoods
and subdivisions that all have different market dynamics. A sample size of 9
make this analysis completely meaningless.

Equally important, is that the author of this article doesn't appear to know
much about how the real estate data industry works (especially in regards to
market trends and how they are used to generate the Automated Valuation Models
(AVMs for short). A bit surprising given the nature of the business he is
representing.

The Zestimate is one of a handful of Automated Valuation Models (AVMs in
industry speak) - it's the only one with a consumer-facing brand, and as a
result sees the most scrutiny despite the fact that the other AVMs on the
market are the ones actually used by the banks during the appraisal process.

If you think the Zestimate is bad, you should see some of the other commercial
AVMs.

The reality is that creating an automated valuation for the ~100M properties
in the US is an incredibly difficult task given the availability of data (or
lack thereof!). Zillow does a pretty darn good job and has a bit of a data
advantage given its incredibly high coverage of for sale listings (though they
lack the experience - some firms have been producting AVMs for decades!).

Another thing to note, is that AVMs (like any statistical model), usually aim
for a sweet spot that covers as many homes as possible. As a result, ultra
high-end properties are often incorrectly valued. E.g. that $25M mansion down
the street will be very hard for the model to price. Additionally, certain
luxury features, like say, a tennis court, are almost impossible to
incorporate into these models on a national scale.

------
intrasight
Better question is "how bad are the compared to those of professional
appraisers who look at half a dozen comprables?" An even more important
question is "how biased (based on who is paying) are those professional
appraisers?"

~~~
teuobk
Isn't it strange how the appraisal always seems to come in just a little above
where it needs to be in order for the financing to be approved?

~~~
zrail
An appraisal is telling you what the market will bear for a particular
property. What better indication of the market value than the most recent
price agreed to?

~~~
intrasight
You're missing the point. May appraisers are under the thumb of the mortgage
brokers. See [http://www.nakedcapitalism.com/2014/12/bill-black-
mortgage-a...](http://www.nakedcapitalism.com/2014/12/bill-black-mortgage-
appraisal-fraud-baack-bank-execs-profit.html)

[I]n 2007the New York State attorney general sued First American: relying on
internal company documents, the complaint alleged the corporation improperly
let Washington Mutual’s loan production staff ‘’hand-pick appraisers who bring
in appraisal values high enough to permit WaMu’s loans to close, and
improperly permit WaMu to pressure …. appraisers to change appraisal values
that are too low to permit loans to close” [FCIC 2011: 92].

------
kosei
Considering a Zestimate doesn't take into account anything about the
aesthetics or actual character of the house, being off by 20% doesn't seem
like it is that much. People respond to more than just the pure square footage
and neighborhood. The Zestimate should be taken as nothing more than a
directional starting point for someone who doesn't have the time to actually
look at neighborhood comps.

~~~
Neliquat
Isnt that what it is marketed as? I found it useful as a starting point when
shopping.

~~~
kosei
Exactly. I think the writer is taking issue with trying to use it as a more
exact science, and calls it out for being within 20% of the sale value 90% of
the time, which I think is pretty reasonable considering the high amount of
subjectivity in home purchasing.

------
morganvachon
I looked my house up on Zillow and it is showing an estimate of about $80k. My
homeowner's insurance has it covered up to $75k, the county tax assessor lists
the FMV as about $50k, and I owe $37k on it. My own estimate is about the same
as the tax assessor's, and I wouldn't ask more than $50k when it comes time to
sell it, given the work it needs to get it up to the Zillow estimate in my
eyes.

Looking at other houses in the area, my house appears to have a much higher
perceived vs actual value, based on Zillow estimates alone. For example, my
neighbor's house on Zillow is supposedly worth $90k, but I'd put it closer to
$100k based on her lot size, house size, overall condition, and age of the
house (I'm friendly with my neighbor so I know a little about her property's
history and makeup). In other words, her Zestimate is a bit lower than what a
reasonable person would put it at, whereas mine seems greatly inflated.
Perhaps this is due to Zillow "grading on a curve"; it would seem odd if one
house was rated at half the value of its neighbors, so mine gets brought up to
the lower end of the average range even though it doesn't deserve it. That's
purely speculation on my part though.

~~~
ryandrake
Offtopic, but I had to chuckle a little: Bay Area house prices have gotten my
mind so clouded that I instantly thought you were missing an extra zero on
each of those dollar values. I had to re-read it twice until it dawned on me
that there are other places out there...

~~~
slackstation
Same thing happened with me. Coastal housing prices skew your perception of
reality so much, it's hard to imagine <$100K even existing.

------
mholmes680
While we're at it, lets also figure out why my Zestimate dropped $40k
overnight two months ago. If I was planning to sell, I would have a huge
problem on my hands. On top of it the Zestimate chart shows no indication that
they dropped it $40k. Aiming to fix the number is a fool's game : my "private
home estimate" on my owner dashboard is 23% higher.

Its a worthless number, and they should stop trying to manipulate the market.
Having a secret measurement formula that works for 70% of the houses but not
the other 30% needs to either have a disclaimer that its all fantasy, or just
outright be removed. I'm guesstimating those 70/30 numbers, but their site
gives some locations 2 stars on their own scale:
[http://www.zillow.com/zestimate/#acc](http://www.zillow.com/zestimate/#acc).
If its not 4 stars, then don't publish it.

~~~
Finnucane
From time to time, Zillow rejiggers (that's a technical term) their algorithm,
resulting in sudden readjustments of estimates. Given that the Zestimate
itself is presented as a fairly wide range, the error margin is hardly hidden.

Also, it is important to remember that no appraiser from Zillow has actually
looked at your house; the data is drawn entirely from sales data, public
records, and volunteered information. So there's no way it can really account
for every factor, nor does it even try. And there's a lag; the market can
change faster than Zillow can. It is, at best, a rough ballpark estimate of
what's going on in your neighborhood.

~~~
mholmes680
point taken, but two observations:

1) its ridiculous to also "fix" the zestimate history. Put the old one in the
history, and explain near the chart what rejiggered.

2) The market in my immediate area, as evidence by the comps from zillow and
the sales prices in the newspaper, has only increased. The conservative comps
are near the old zestimate - this is a suburban area. The wild ones are higher
than my private estimate.

~~~
tmarthal
I think what you're actually seeing is the output of a Kalman Filter temporal
model. The zestimate is the mean of the filter output, but the error bounds of
the filter are more important than the actual mean/reported value.

What happens is that the model uses trends to extrapolate from each "real"
data point (in this case, a house sale in your neighborhood). The problem is,
and what Kalman Filters help manage, is the uncertainty propagation between
each house sale. When it has been a long time from when a sale has occurred,
it is unclear what the real/actual price is of a home. This means that on the
estimated price the error bounds are large, and zestimate still just reports
the mean value of this huge uncertainty.

What then happens is that a house is sold in your area, a new data point is
recorded, and the filter re-adjusts itself and collapses its uncertainty/error
bounds in the time of that measurement around the measurement. And you get
correction. This is why the "old zestimate" is updated.

------
EpicEng
More comprehensive studies have ben performed:
[https://www.washingtonpost.com/news/where-we-
live/wp/2014/06...](https://www.washingtonpost.com/news/where-we-
live/wp/2014/06/10/how-accurate-is-zillows-zestimate-not-very-says-one-
washington-area-agent/)

I just bought a home myself and looked this up, but the takeaway is that the
Zestimate is within +- 5% 50% of the time. So, not that accurate. Are people
really using it in negotiations though? I'm sure people do, but you'd have to
be pretty uninformed to do so, and I'm also sure you're real estate agent
would tell you exactly that up front.

~~~
ensignavenger
Yeah, but how accurate are appraisals? Several years ago I was reading about
an analysis that found that Zestimates were about as accurate as appraisals
are 80%+ of the time.

~~~
honkhonkpants
The job of the appraiser is to write down a number as large as or larger than
the amount to be financed. They know this and they know the number in advance.
What they do is not rigorous.

~~~
r0m4n0
Definitely not true... there are many checks in place to ensure this doesn't
happen

[http://www.freddiemac.com/loanadvisorsuite/loancollateraladv...](http://www.freddiemac.com/loanadvisorsuite/loancollateraladvisor/)
[http://www.freddiemac.com/loanadvisorsuite/faq/loancollatera...](http://www.freddiemac.com/loanadvisorsuite/faq/loancollateraladvisor.html)

This doesn't even take into account the process for appraisal management
companies (third parties).

~~~
honkhonkpants
You seem to think these industry controls are effective. Maybe they prevent an
appraisal from being, say, 10x higher than the recent sale prices on either
side, but they don't prevent the equally bad event of a series of 10 sales on
the same block each being 10% higher than the previous. Every one of those
houses in Las Vegas that turned out to be worthless in 2007 was appraised for
at least the sale price the year before. The appraisers in those cases didn't
do a damn thing, for example they didn't come out to the house and write down
"this neighborhood is in a desert with no access to water where the
unemployment rate is 20% and the employed people make minimum wage therefore
this cannot be a neighborhood of $850000 houses." They just said "the last
house sold for $800000 and this market seems pretty hot so $850000 okey
dokey."

------
joshuaheard
I use Zillow to give me a ballpark number for prices in an area I don't know
well. My experience is that they are usually within 20% of the market price,
which coincidentally they say on their website. For a free automated service,
Zillow is great. If you want a better number, do your own comp analysis, or
hire an appraiser.

------
baus
I've been working in the mortgage industry for the past few months and have
learned a bit about the business.

What Zillow does with its Zestimate product really isn't that novel. There is
a whole class of products available called AVMs (Automated Valuation Models)
which are used in the industry which pre-date Zillow's Zestimates. What Zillow
did which was innovative was brought this type of product to the general
public, for free.

In some cases an AVM is all that is needed to secure certain types of loans
such as HELOCs.

------
jotux
Redfin does a much better job with estimates and track their error rate:
[https://www.redfin.com/redfin-estimate](https://www.redfin.com/redfin-
estimate)

------
blisterpeanuts
I find Zestimate _over time_ to be useful. I've been watching my Arizona
property climbing up since 2010 when it was estimated at rock bottom, or about
50% of its purchase value (sigh). Now it's estimated to be about 90% of its
pre-crash value. But as with all private transactions, bottom line it's worth
what someone's willing to pay for it.

------
organsnyder
In my experience, Zillow does a terrible job compared to local knowledge. My
zip code is weird—our ~$275k house backs up to houses worth $50-70k (the
disparity is due to build quality, lot size, and the streets they're on). Not
surprisingly, Zillow handles this very poorly, and has consistently valued our
house at much lower than it is actually worth (based on city assessments and
sales of nearby comps).

Interestingly, Zillow recently bumped up our Zestimate from $190k to $230k.
What's puzzling is that they also bumped up the _historical_ Zestimates—there
isn't a big jump shown on the graph on their site (though there is a big jump
shown in my Personal Capital account, which uses Zestimates as part of their
net worth scores). Either they modified their algorithm in a way that affected
values retroactively, or they're trying to disguise the fact that their
previous Zestimates were way off.

~~~
cbr
> What's puzzling is that they also bumped up the historical Zestimates

There are two things a "historical estimate" could mean:

1) What did Zillow say the value was on day X?

2) What does Zillow now think the value was on day X?

You and others seem to be thinking of (1), but I think Zillow actually is
doing (2). Since the goal is to figure out what the house is worth and how its
value has changed, I think (2) is actually the right metric here.

~~~
tyingq
(2), of course, has the advantage of obscuring past mistakes and producing a
nice smooth chart.

~~~
danans
Unless you think they should never update their algorithm, approach (2) is
just making the chart consistent with itself.

The useful part of these charts isn't really the current Zestimate, but rather
the trend line of the home values, especially at the neighborhood level.

If you don't recalculate the historical values, then you break the ability to
see the trend line.

If you really want to see the results of previous models, then the way to do
that is to plot a separate line for each model, but you can see why they
wouldn't want to do that from a product perspective.

~~~
tyingq
Every county in the US that publishes historical valuations seem to get by
without overwriting history.

~~~
danans
Yes, but those are actual valuations based on sales prices, not predicted
valuations based on an algorithm. Different thing.

------
ivankirigin
5M home sales yearly, so this shouldn't be done by anecdote.

~~~
ctrl-j
That was my first thought as well. Analyzing 3 houses in 3 markets... It's an
interesting look, but I wouldn't consider any takeaway message to be
necessarily accurate.

Algorithms have outliers, and quantitative analysis requires adequate sample
size.

------
acconrad
Whether or not the data collected in this article is questionable, there's a
pretty easy way to evaluate the price of your home: re-finance / hire an
appraiser. If you believe your home to be worth more, wouldn't it be worth the
$200 or so to increase the value of your home by _thousands_ by having an
accurate market research from a qualified appraiser? I got mine as part of my
refinance package, so I have a very accurate estimate of what my home is
worth, comped out against other homes in my area that sold recently at a
similar location/sq ft/etc. Not sure why you feel the need to use questionable
data points like Zillow in the first place.

------
Spooky23
In New York Zillow seems to be very dependent on property tax assessment data,
at least in my region. They also don't have a good way to distinguish between
neighborhoods well.

That said, they do a pretty decent job overall. Property valuation is art and
science, and is probably a lot easier in a place like LA with lots of tract
homes built in similar timeframes to similar spec.

I live in a city founded in the 1600s. Housing within a 5 block radius is
built between 1890 and 2016. It's not a market with a lot of flipping, so it's
hard to value as a house that looks like mine (reasonably modern internals)
may have 1960s kitchens and bathrooms.

------
jedberg
A very smart statistician who worked for Amazon once said, "estimates are
useless without error bars". The problem with Zillow is they really should
just show a range, not a fixed number that appears to be accurate to the ones
place.

Edit since I wasn't clear: I know they show a range in tiny text, but they
main number they show should be the range if they want to avoid all of the "is
Zillow accurate" articles.

~~~
lutorm
Except Zillow actually _does_ show a range. If you look at the Zestimate
details on a house, you see something like this:

"Zestimate $472,603", and immediately to the right: "Zestimate range $425k -
$506k". The little question mark reveals:

"The Value Range is the high and low estimate market value for which Zillow
values a home. The more information, the smaller the range, and the more
accurate the Zestimate. See data coverage and accuracy table."

------
dbg31415
Zillow isn't accurate for Austin.

Using two identical homes in my neighborhood about 3 doors apart from each
other, mine / neighbors.

Zillow - $387,546 / $384,326

Redfin - $417,808 / $420,737

TCAD - $368,876 / 372,802

(Recent Appraisal) $427,000 / (Recent Sale) $429,000

What's worse, is that when I contested my taxes Zillow used the TCAD numbers
to lower the value of my home on their site. Uh... Zillow, you know it's in
our best interest to contest our taxes, right? Has nothing really to do with
the value of our homes... it's a mix of us wanting to pay less and the City /
County wanting us to pay more.

I'm sure there are more scientific studies, but for my house I'd say Zillow is
~10% under-market. When I went in to contest my taxes this last year, I was
told by a friendly TCAD employee that Redfin numbers were better and that I
should use Redfin when trying to cite examples of comparable properties with
lower taxable values.

Keep in mind most of the value depends on your realtor's ability to show the
house, how well you stage your house, and the number of of interested buyers.

------
Tempest1981
Thought exercise: if you were Zillow, what additional data would you need to
make the estimates more accurate? And where would you get it?

------
SoftwarePatent
This article deserves the heavy criticism it has received here. I'll go
further and say it is part of a trend you might call skepticism gone wrong.
This is a intellectual movement that "throws the baby out with the bathwater"
because they are so skeptical that they don't believe anything!

Many authors in this vein have read about, for example, cognitive bias, the
many invalid results in scientific journals and the hoax/fake news/infowars
trend. They see _every_ human action as a result of bias, every result as
invalid, and every news article as part of a misinformation campaign.

Having the most accurate possible understanding of the world we live in is
only possibly by trusting information sources which deserve it, understanding
the limits of what "quick and dirty" statistical analysis is capable of, and
recognizing when people are influenced by bias, and also when people are
acting with full awareness of bias.

------
sytelus
As far as I can tell, Zestimate works by taking prices of recently sold nearby
houses and then estimate gain over previously sold prices or the property tax
estimates. If Zillow wasn't in the picture, you would likely do the same
thing. This obviously would generate inaccurate estimates if house wasn't in
market since many years or if nearby houses are too different or too few.
Another source of inaccuracy is that they don't seem to take in account school
zoning very well. So house in good school area can inflate price of nearby
house which doesn't have same school.

In my experience, Zillow has played big part in house pricing. Seller in hot
market will almost always set the max of Zestimate and price in their mind.
That prices decreases only slowly over time if buyers don't show up. In hot
market usually some sucker will show up anyway and Zestimate becomes self-
fulfilling prophecy.

------
dpandey
Zestimates are only reliable as an 'in the region of $X' indicator. Seen
multiple houses where this issue exists. I'm sure those are not the only ones.

I know Zillow doesn't make a claim about being super accurate, but not
everyone is an intellectual buyer doing math in their head. For a lot of
people, they treat zillow as a roughly true indicator and are surprised when
that doesn't match reality. Or they completely dismiss it in the very
beginning and rely on their agents or intuition.

You'd be surprised how seriously people who aren't quant inclined can take a
prediction or estimate. Well, given that all the quant inclined among us
believed Hillary would win, it's obvious that humans tend to put trust in
other people's estimates if they essentially trust those people would do a
good job unless proven otherwise. It reduces the cognitive burden of
calculating it ourselves.

------
jedberg
> If you excluded trust sales and probate sales or family-to-family type Quit
> Claim transfers (where the property is transferred at a considerably lower
> value from its market value) shouldn't the Zestimate reset to the exact
> amount of the property's last sold price?

No, it should not. My personal example: When I bought my house, I bought it
for 20% less than what Zillow said. But Zillow would have been right on the
money _if my house were in average condition_. But it wasn't, it was
unliveable. The first thing we did, before we moved in, was spend 9 weeks
renovating it. Even then we didn't even bring it up to the local average. It
took us five years to finally finish all the work.

Zillow estimates assume the house is in average condition, because they have
no way of knowing otherwise.

~~~
Tempest1981
I wonder if you could estimate the condition of the house based on a street-
view picture of the outside. There is probably correlation between outside
upkeep and interior condition.

~~~
jedberg
Maybe. Ours looked ok from the outside, was definitely worse on the inside.

Incidentally we bought it from the original owners who had owned for 55 years.
I wonder if there is a correlation between condition and how long the current
owner had it.

------
iamthepieman
I bought my home for 139,000 4 years ago. The bank appraised it at 155,000
while i was buying it (also 4 years ago). The town has it appraised at 178,900
since 2011 Zillow's "Zestimate" has it at 134,500

If we go by the "market" rate of what it would/could actually sell for then
Zillow is the most accurate out of all those estimates.

Now I've done quite a bit of interior work to the house and Zillow's is most
likely off by quite a bit now that my house has a new kitchen and bathrooms
but I'm not sure how it could take that into account since there's no way for
that data to reach Zillow.

------
geekamongus
I don't know how accurate these things are, though my gut tells me not to
trust them. That said, as a recent home-buyer, I can state without uncertainty
that real estate agents hate Zillow, and Zestimates in particular.

~~~
famousactress
I concur. Though I tend to find this attitude pretty ridiculous since the
status-quo home valuation process is just as bullshit at the zillow strategy
is.

Status Quo in my experience goes:

1\. List your house (usually at slightly above whatever the realtor thinks it
might sell at)

2\. Someone's willing to buy at listing price +/\- 5%

3\. Bank sends an appraiser whose job it is to figure out what a house is
worth (guess what: see #2 for relevant demand evidence), and magically the
number comes remarkably close to the price sorted out in #2 massively most of
the time

I've never heard a real estate agent complain that zillow's numbers are too
low, mind you... only ever that they're too high, which makes sense to be
frustrated by if your biggest challenge is controlling your time-costs.

------
Terretta
Really dumb but fixable flaw in the algo: I bought my home from a mother and
daughter who each owned half. Town hall filed the two transactions on the same
afternoon.

Zillow counts it as two data points at half price!

It's the only sale on the property so they have two half price "historical"
inputs to the Zestimate, and that makes the real listing price seem nuts to an
Internet home buyer.

They refuse to correct the data, insisting it sold twice at full price on the
same day instead of two halves, and saying it's on me to get the town hall to
reflect it as a single transaction, which it was not.

------
harrumph
Given that the company's own CEO didn't use his own company's "Zestimate" when
it was time to zell his house, I think the accuracy of the number is fairly
perceived to be zub-optimal.

"You gotta eat your own dog food," goes the old zaying.

[http://www.inman.com/2016/05/18/zillow-ceo-spencer-
rascoff-s...](http://www.inman.com/2016/05/18/zillow-ceo-spencer-rascoff-sold-
home-for-much-less-than-zestimate/)

------
ffjffsfr
It's not suprising that estimates based on "user-submitted data points" are
not accurate. Everyone who ever tried to sell a house knows how difficult it
is to estimate real market price of property. Usually people will submit
higher valuation of their house so that they can negotiate it down in case
their valuation is not realistic. It is better to give higher value so that
you have room for negotiation. Transaction prices will always differ from
listing prices.

------
camhenlin
Just over two months ago, I sold my house within $1,600 of the "Zestimate". I
then turned around and purchased another house over 10% below its "Zestimate"

------
cinosan
If you have a SFH try this out. Way better than zillow.
[https://www.housecanary.com/](https://www.housecanary.com/)

~~~
yoloswagins
Which house canary product does free home valuations?

~~~
jungrothmorton
We have free trials of our valuation product, Value Report:
[https://www.housecanary.com/product-value-
report](https://www.housecanary.com/product-value-report)

~~~
r00fus
When do you plan to add support for multi-units? I have a duplex...

~~~
jungrothmorton
We're working on getting all of our single family residence products where we
want them before we move into multi. I don't have any timeline for when that
would happen.

------
hoov
I bought a house a year and a half ago; Zenstimates are consistently
ridiculously low for my area. I was looking for single family homes in
Somerville, MA. There is so little supply that every home goes immediately
after listing and most likely with a bidding war. If you look at the
Zenstimate for my property, it looks like I over-payed by almost 30%, but I
guarantee I could flip for a profit tomorrow.

------
sixdrum
The Zestimate for my house is incredibly close, within $2k of a recent
appraisal. There are a lot of comparables in the area, so they have a decent
amount of data to work with.

I have a friend who just bought a 3,000sf house on 1/2 of an acre in
Lexington, VA for $31k. Zillow has it at $400,000. He's either incredibly
lucky and should buy me a beer, or Zillow is the worst. Leaning towards the
latter.

~~~
jsjohnst
Like the previous reply, I'm going to assume you meant $310k, not $31k, as
$31k for that property doesn't make sense based on the factors you listed
unless the place is a dump.

As to the disperity, this could be easily explained by seller factors. Maybe
the house is actually worth $400k, but they _needed_ to sell it, even if less
than it was worth (had a contract on another home, home was a foreclosure, etc
etc).

~~~
sixdrum
I did not drop a zero. He paid cash.

~~~
hnal943
is it a nuclear waste dumping ground? What's the mitigating factor?

~~~
sixdrum
In the country and needed about $10k of renovations. Sold as-is. It _may_ end
up being worth $200k, if he's lucky. Zillow is just incredibly far off.

------
pbnjay
At some point my home was converted to a duplex, and converted back into a
single family home before I bought it. I've tried multiple times to get Zillow
to fix the listing but they've been "unable" to do even such a simple thing...

I suspect their data quality is pretty terrible if they're unable to do any
curation, so it's no wonder the zestimates are all over the place.

------
paulddraper
Anecdotally, I've noticed that Zestimates are biased by location.

According to Zillow, 90% of the houses in one region were overpriced and
selling like hotcakes. And 90% of houses in another region were underpriced
but staying on the market.

I think most of the value is not an absolute measure, but as a relative one. I
can compare Zestimates to each other in the same city with fair accuracy.

------
camiller
I don't know about "zestimates", but they seems to not be able to do something
as simple as email address verification properly. Admittedly, based only on
the five email's I have received from realtors in North Carolina just this
morning (I live in Nebraska and have no plans to move any time soon).

------
amyjess
I'm still scratching my head and wondering what Zestimates mean for rentals.

For example, I'm giving serious thoughts to moving to SoCal, and I see a place
that looks pretty nice... it's listed as being $2000/month, but then when I go
to the listing it says Zestimate $2650/month. What's up with that?

~~~
r00fus
Is $2000 within the rent range? Is it a duplex? Seems like Zillow data is far
more accurate for single family homes.

------
mundo
Wait - is the author just measuring Zillow's accuracy the same way Zillow
does? As in, "Here's a poll that asked N people something. Let's check the
poll's accuracy by calling 3 of those people and asking them the same question
again."

That seems... not useful in any way.

------
motor_racer
Zillow is beyond useless. My property consists of 4 legal "lots", two of which
the house physically occupies. Each lot is assessed separately by the gov't
for tax purposes. The "Zestimate" considers only the lot associated with the
street address. So about 40% of the property valuation is ignored.

Second, my house is lakefront. Lake frontage is probably the single biggest
factor in valuing the property. Yet Zillow ignores lake frontage as a feature.
In fact a recent check of so-called "comps" included no lakefront homes at
all. A complaint to Zillow was met with a yawn and a "well its just a number
people can use as a starting point, and real estate professionals in the area
will know the particulars." So basically they told me the "Zestimate" has no
value and they know it.

~~~
pcsanwald
I have a similar situation with my condo in brooklyn, which has a deck and a
great view, which doesn't count towards square footage. So the zestimate is
way lower than what we paid.

We recently moved, and rented our condo, and we were able to rent it for 2k
more than the rental zestimate. So, I agree that it's to be taken with a grain
of salt, and not that important.

------
m0llusk
This article failed to compare Zestimates to any of the numerous competing
estimation services and also neglected to mention that the primary competition
for these computed estimates is intuition of Realtors and brokers.

------
kylebenzle
Ohio, residential home, Zillow = $310,000, sold for $210,000

------
notadoc
In the hot west coast markets, Zestimates are usually fairly accurate but
often end up low compared to closing due to bidding wars and mania.

