Hacker News new | past | comments | ask | show | jobs | submit login
Why Zillow Couldn’t Make Algorithmic House Pricing Work (wired.com)
66 points by Kydlaw on Nov 12, 2021 | hide | past | favorite | 69 comments



There was a great episode of The Indicator from Planet Money about this.

One aspect not mentioned in this article is that when you offer an average price for a home, you are only going to get the owners of below-average homes taking you up on the offer, i.e. the Lemons. This is a tricky problem for Zillow to resolve given the mismatch in knowledge between the existing owner and the iBuyer.

https://www.npr.org/2021/11/08/1053689886/ibuyers-zillow-and...


I disagree. Real estate lemons are quickly identified by home inspectors which is why there is a variable % in the offer to buy. There are plenty of people that want to sell quickly for various reasons if not at market and at a reasonable discount. This is not the problem.


Home inspectors, in my experience, are pretty useless. They're running through a checklist, not actually thinking much.


A checklist is precisely what inspectors use when verifying a rocket for launch. It is far and away the best system to ensure consistency and thoroughness.

It might be a poor checklist, but that's another story. I suspect there are good checklists you can download that you can use yourself and give to an inspector who is well meaning but inexperienced.

No list will fix dishonesty however. They can simply check off boxes.


Verifying rocket launches involve multi-million dollars worth of equipment run by technicians and engineers (hopefully) intimately aware of the rocket about to be lifted up into space.

As the sibling post said, most inspectors aren’t using NDT tools to check if there is wood rot behind that fresh coat of paint


Inspectors can’t rip apart the walls. A fresh coat of paint can hide a lot of damage. I would know because this happened to me, and I had two inspections done.

If you’re slightly decent with houses and construction, you’d see most of the things they see anyway.

Sadly, I’d place home inspections closer to useless to useful for me because I am familiar enough with housing codes to do a visual inspection.


They don't need to be thinking much, they need to be doing exactly that; running through a checklist of _major_ structural (foundation, roof, walls) or infrastructural (pipes, HVAC, electrical) issues with a house.


Not sure what kind of inspector you got?

* No general inspector I have gotten was qualified or willing to touch the roof. If I wanted to look at the roof condition, I needed to pay a roofing company separately to come out.

* The electrical inspection consisted of taking the lid off the electric box, and looking for loose wires, and putting a "does this have ground" meter in all of the power outlets. Most of the interesting electrical issues that could exist are burred in walls and not accessible.

* General inspectors are not HVAC techs, and can't really say much about HVAC beyond "The furnace is 14 years old based on the install sticker, and appears to turn on"

* Structural issues were like.. look for visible cracks in foundation, but much is likely behind drywall and unable to be seen. Maybe put a marble on the floor and see if it rolls.


> Most of the interesting electrical issues that could exist are burred in walls and not accessible

Assuming the electrical work was done by electricians and not total amateurs, there should be absolutely no issues inside walls/ceilings, every splice or termination must be ‘accessible’, which means in a junction box that is not buried in a wall. If a box is inside a ceiling or wall, an access panel would make it accessible.

Electrical issues are almost always at the point of termination, unless you have animals chewing on your romex insulation. Bad splices and terminations cause arcs, which cause fires, which is why AFCI breakers are now mandatory for almost (unsure if it’s all or not, I work on the commercial side) every receptacle circuit inside a residence. Aluminum wiring caused fires because aluminum expands and contracts with heat faster than the terminal screws that were used in the wiring devices of the time. Now we have AL-CU devices that eliminate this problem.

You can’t always count of electrical work being done properly though.


This might depend on where you live, in my area every inspector will check the roof (unless for some reason it's unreasonably dangerous to get up there).

Obviously they can't see hidden problems but the really bad stuff (cracked foundation with seepage issues, water damage, mold, etc) is worth looking into. My inspector pointed out a number of things like old cast iron plumbing, some unidentified venting, electrical that wasn't to code, insufficient ventilation to the attic, etc.

It didn't affect the price but it did give me a good checklist for what to fix in what order.


Thanks for this.

I’m looking at starting a campaign to force mega rental companies to get home inspections and provide results to tenants.

A large apartment building with a known history is very different then random houses where incompetent people did who knows what to the internals.

Sounds like I may need to push more then a standard home inspection.


The algorithm doesn't care. Realtors bought houses only to flip them to zillow a few weeks later for an instant 10% profit while doing no work on the property.


I think you are only going to get highly motivated sellers and that in of itself is a red flag in a sellers market. The trick is understanding why they are so motivated.

Why would someone want to sell a home fast? Generally due to a relocation urgency or financial challenge. If it’s financial, there is a decent likelihood that proper care and maintenance may have been missed and that could lead to a lipstick on a pig kind of situation.


I'm not sure there's such a thing as a housing lemon (except maybe a house so infested with mold it has to be torn down, or something built in a place where nothing should have been built in the first place). There are houses that require massive investments (foundation, roof, plumbing, sewer) but those are all knowable. The simpler answer is probably that your models don't work if your model causes you to take action that changes the market. See the SciFi paradox of predicting the future causing you to take actions that change the prediction of the future. For example what happens to prices when you add an algorithm that is REQUIRED to make a certain number of purchases a month? (I'll give you that one for free. Prices go up A LOT) What happens to the cost and availability of contractors when demand increases by 20% (same answer as above) Now feed those things back into the model. Add in the fact that market rate changes based on the number of aggressive algorithmic buyers operating in the area and that you're not going to bid on your own sale (I hope!) CRAP! Now those actions we took before are predicted not to be cost-effective! Obviously. Remember the first lesson of thinking ahead, kids. If I do X, Y will likely happen.


Doesn't that beg the question?

Zillow can have better information on the market value of a house that it has worse access to information about than the owner.


This whole issue is fascinating to me and no doubt will be studied in schools in the coming years. One of the lead product designers on Zillow Offers wrote a retrospective yesterday that outlines, broken user journeys, greed and the attempt to algorithmically one-size-fits-all the whole market. He lays a lot of the blame squarely at the feet of incompetent executive leadership

From his notes: “They valued old-school growth hacks from 2007 over taking user-centered, data-informed, research-backed approaches. They spent more time flexing and building decks that drove an internal narrative of Sellers being happy, than actually addressing real and persistent product problems that ultimately led most sellers down a path where they didn't know what to expect, didn't understand the process and were inundated with calls and scheduling.”

https://www.linkedin.com/posts/bardlavens_zillowoffers-zillo...


I read it and while I appreciate the speaking up, this retro does not look nearly as good as the poster thinks it does.

First, it is written from a perspective of "I/we know better, they didn't listen to us, they're incompetent", as in here:

> We knew back then how many home sellers felt jilted, offended, etc. by this product.

Then he doesn't clarify anything about why sellers would feel this way, what does "many" mean - is it all? A significant majority? 0.3%? Quantifying this would have brought his point across better.

Then there's an assertion that they "should have taken a nuanced approach engaging with local communities" again without any justification as to why they needed to do it, with a mention to failure in new markets vs. success in markets already served by Opendoor without any context or even reason to be there. What does that even mean?

The descent into the "toxic, abusive, etc.", "meritless leaders" name-calling is poor form and takes away from the best paragraph of this, which is where the author starts hinting at some of the real problems - sellers inundated with calls - he wants to discuss.

Obviously the Zillow leadership screwed up on this, but this rant does little to illuminate why and frankly hits at some political ideology disagreement in addition to bad product design.


I came away from this post with a strong sense that I wouldn't want to work with the author.


I felt the same way and find it somewhat ironic that this guy now works at...wait for it...Facebook. You know, the world's most non-toxic, consumer-centric charity.

It's also amusing that he has this in his LinkedIn bio:

> Previously, he led design for Zillow’s newest venture, Zillow Offers—building the team and product vision of an end-to-end experience for buying, selling, financing and managing a home, all within the Zillow ecosystem.

So he takes credit for leading the design for Zillow Offers and building the team and product vision, but then posts a retrospective criticizing it all and disavowing any responsibility in the product's failure? Strange, but perhaps explains why he's a good fit for Facebook.


Yep he's likely trying to put a good face on it and rationalize away the failure. It's human nature especially when career/rep/money is on the line. I'm sure there were problems outside his control but it's highly unlikely this is an accurate and objective account of the project's history


> It's human nature

I think this is only true to an extent. In my own career, I've worked for and hired people who were honest about their roles, taking a reasonable amount of credit where appropriate while also owning the things that didn't go so well.

In tech, there's really no downside to this because the industry is pretty damn forgiving. Failure is expected, and a lot of super successful people had multiple failures under their belt before they succeeded.

It's really, really cringey to have a LinkedIn bio where you basically position yourself as the owner of a product and then elsewhere make it sound like you not only had nothing to do with said product's failure but in fact knew how to make it successful but couldn't because everyone else was flawed in some way.


I have been in the position of leading products in what turned out later to be bad directions. It does not matter if you are “in charge of a product” when management shoves bad ideas down your throat and ultimately derails any attempt to keep the train running smoothly on its tracks. Unfortunately, this is no way to address such a situation. If you have already moved on to greener pastures, consider yourself lucky and let it go.


Exceptional detective work here.


I ran Opendoor's pricing unit for the years leading up to IPO. I can say that this designer has a fundamental misunderstanding of the housing market and the Seller mindset which is likely why leadership ignored him.

In housing, EVERY OWNER BELIEVES THEIR HOME IS WORTH MORE THAN IT IS, and will be offended by a fair offer. Period. The failure of their lead designer to realize this after all these years is just one signal into why Zillow couldn't succeed where Opendoor did.


Every owner who sells their house obviously thinks they have a fair offer other wise they wouldn’t have sold in the first place. In my area OpenDoor bought the same quirky weird homes as Zillow that were ugly, had shoddy repair work done, or were in a very weird lot. The reason these sellers sold to OpenDoor or Zillow at slightly below asking was because it was an easy quick sale without all the questions a homeowner would ask themselves before buying.


> Every owner who sells their house obviously thinks they have a fair offer other wise they wouldn’t have sold in the first place.

Because no one ever has a deadline to sell or any of a thousand other reasons one might take a lowball offer?


The genius of Eric Wu is that he created a premium product for home sellers wanting convenience. Even when we give you an offer, we tell you up front to go test the market with an agent if you want to price optimize.


He adds:

Many of these leaders, were leaders not on their merit, success or competency, but simply because they stayed at the company long enough that their titles grew as the company did.

Curious how people at FAANG and other midsized to large technology companies feel about this. Can we look at a company like Google and find senior people who rose through the ranks not so much because of talent or intelligence or leadership qualities but rather because they stuck around long enough? Or is this something found mainly in toxic companies like the author described Zillow to be?


That post strikes me as wildly off the mark. Zillow didn't fail because "many home sellers felt jilted, offended, etc. by this product." They failed because they systematically overpaid for assets. (Anecdotally, the one person I know who sold to Zillow is very happy with his experience; not because the process was perfect, but because they gave him more money than his house is worth.) This post is about the wrong battle ("Why does our conversion rate suck?")

If I were to steel man his argument, I'd say that maybe the bad user experience meant they had to overpay to get people to actually use the product. But he seems very narrowly focused on "this wasn't a good experience for sellers" when that's at most an indirect factor in the failure.


If I didn't know he was talking about Zillow in that quote I would have thought he worked at my company.


Isn't another problem that people who would use their automated service to sell their houses will skew towards those whose houses have some value-diminishing problem (which their algorithms have a hard time accounting for) or for other odd reasons have lower than usual market price?

People who can fetch a premium for their houses probably wouldn't use an automated algorithm, which tends towards the mean?

E.g. a self selection of those who would use this service tends to give Zillow a bunch of houses of lower value than average.


Without question this extends to the other iBuyers in the market.

Zillow exiting because they could not make it work should be a massive alarm bell. Not only are they no longer buying new properties, they are liquidating their existing assets. You would not do that if you thought the market would come back to you.


So I would think too, yet Opendoor has popped back up within 5% or so of its level prior to the Zillow news:

https://www.marketwatch.com/investing/stock/open?mod=over_se...


If social media is to be believed, firing 2,000+ employees is also a bit concerning.

At a bare minimum, that does not show much confidence in the organizational structure they put together, either.


This does not mean the zestiamte estimate is flawed, but that flipping homes is inherently risky in independent of the price estimate algo.

The huge, v-shaped recovery of the housing market and also stock market and GDP caught a lot of experts and pundits by surprise. The expectation was that, even with stimulus, that things would take years to recover as the US economy clawed its way out from the depths of the pandemic, but then Game Stop stock went up 100x and suddenly a FOMO unlike the likes ever seen took hold. Glad I didn't sell, and also I added to some stock positions in April 2020. Suddenly everything went from extreme over-supply to extreme shortages in a year.


The "extreme shortage" media narrative is misleading. Supply in many areas is at or above pre pandemic levels. What's changed is a massive demand spike as society reshuffles and realigns to post pandemic life.

It's a subtle and important point because it means inflation is not transitory. You should buy all and any risk assets to protect against inflation including homes and GME.


It is absolutely not misleading.

In fact I would say it's understated. We lost a half of builders, a third of tradesmen, and saw a massive drop in people training for the trades after 2008.

We only caught up to the peak in new home builds a couple of years ago.

Sure, there was a demand spike, but supply is about a decade behind in a market where it takes a very long time for supply to ramp up.


I believe you are correct [1][2] that wage workers in these occupations have declined since 2008.

Is that related, however, to physical good supply chain disruptions in the past 12 months? And if you're referring to labor supply - we are basically at the same or better unemployment rate than we were in 2016... which was already a pretty good time in the economy.

[1] https://fred.stlouisfed.org/series/LEU0254505500A [2] https://fred.stlouisfed.org/series/LNU04032231


We have an extreme shortage of housing in places where there is demand.

We have a shortage of builders and tradesmen to build more.

This problem will take time to resolve.


> The huge, v-shaped recovery of the housing market and also stock market and GDP caught a lot of experts and pundits by surprise.

It is not recovery but a price inflation fueled by relentless government printing and injecting money into the system. Since the actual economic activity is still at the standstill (cannot get anythings from cars to Thanksgiving turkey) this new money has low velocity and needs to be parked somewhere. Hence, inflated housing and stock markets. Very sad state of affairs.


The Zestimate is seriously flawed to the point of it being utter crap.

The fact they used it as a basis for flipping homes makes me literally laugh. They were that delusional about their own shitty tech to just outright burn bails of money


Technically, they weren't using the Zestimate, it was a different algorithm developed for the iBuyer program (or so I've read).


The markets behavior is largely disconnected from GME though. We were on the upswing long before Jan 2021 because of all the money in the world looking for a place to go.

That said, I agree that many were surprised by the quick recovery (yay QE?).


Interesting topic but this article is very short on the important details.

“Buy and hold for a few months” would have been a good strategy for most of the pandemic, but Zillow appears to have overpaid substantially to the point where that didn’t work. Their predictions must have been badly off. (Also: this is a great example of the “winner’s curse” in first-price auctions!)

The article quotes a figure of $65,000 “over market price”, but where “market price” comes from for those houses it bought is unclear.

The claim of “market manipulation” by Zillow is ludicrous on its face without substantial evidence, which I’m guessing did not appear in a tiktok video.

If all iBuyers in the Phoenix market _combined_ were buying 5% of homes, that doesn’t suggest to me that Zillow _alone_ would be able to exert any pricing pressure.

I would love to read a more detailed article than this one if any are available.


are you just ignoring that zillow is a website that people search for their homes and zillow would be, like facebook, like google, like windows, the pureposefully distort results and dark patterns towars their own wares, skewing the actual perception of the msrket?

seems odd to give them any good faith here, given what we have seen with every other xearch platform.


I'd recommend this article for a deeper look at how Zillow and its competitors exhibited different behavior in Phoenix as the market cooled:

https://www.mikedp.com/articles/2021/10/19/ibuying-is-hard-z...

> As the market cooled between August and September, Opendoor and Offerpad purchased fewer houses, while Zillow purchased more.

> The iBuyers also adjusted to changing market conditions by paying less for houses. The median purchase price in Phoenix peaked in August. Opendoor and Offerpad's median purchase price also peaked in August before tracking the market and declining in September. But Zillow kept paying more and more.


Lots of general contractors, with enough capital to buy a home, do some repairs, and sell it; They have a very good knowledge of the local market. Can do the repairs themselves.

Larger companies probably focus on commercial developments and/or condos.


The network that GCs need is astonishing. I did a renovation this year and my GC had of course his crew, and also hired a plumber, an electrician, a plaster guy, a flooring guy. The electrician and plumber both had their own crews. The network is such a big part of the battle.


Because models <> reality.

That’s it. Full stop. Anyone who thinks otherwise will eventually face this same fate.

For instance, from the sub-heading, “But the pandemic messed up its predictions”

should really be this instead, “but their algorithms couldn’t handle a situation they didn’t anticipate”.


House price volatility is localized to the point that sample sizes are too small. Prices are generally neighborhood-based and the sales in a neighborhood are too infrequent. One quick divorce sale, or one over-eager buyer can skew the data enough to foil this approach.

As an anecdotal example, my development has 115 homes. In the last 12 months, 3 have changed hands. The first sold at 5% over the Zestimate, the second, 10% over, the third 15% over. How would the Zillow algorithm predict the price of the next sale?


It's a very complicated algorithm of weighted distributions of /dev/urandom


The next would be 20% over.


After the failure everyone is able to say why it wouldn't work.


These types of restrospectives are often very interesting examples of human capability to rationalize away culpability. Coldly I think it's good business skill as often people who are good at crafting these types of failure retrospectives seem to fail upwards


first as tragedy, then as farce


They have to offer the seller more than the seller thinks the winning bid would’ve been if they sold it without Zillow. And it has to be less (including their transaction and improvement costs) than the actual winning bid will be, when they turn around and sell it themselves.


>They tend to offer lower prices than traditional buyers, but attract sellers by promising faster, all-cash deals.

That was the deal?

They thought they'd get a lower price by offering cash and quick closing?

Even in a down market (last time I bought) I offered quick closing and all cash.

Sellers didn't like it. It took around 9 months before I got a call back from one house I wanted saying "Is that offer still good?"

In the meantime they had accepted 3 other/ better offers ...but pending the other buyer selling their house, all of those feel through.

And that was in a down market...


They were acting as their own realtor, so they also had some margin compared to people paying for a realtor.


Why didn’t sellers like it?


They did like getting offers lower than expexted and seemed to be happy to wait for the possibilty of a better offer, even if less reliable ( not cash,not a quick closing).


Yep, sellers always want that extra $10k especially if not in a hurry. However, I’d bet their agents would love to sell that home for $10k under asking just to get rid of it fast rather than having to market it across months. That $10k to them is $300 on a split commission.


2020-2021 was a very weird period. The fact that a given predictive model failed during these years is not really a strong indictment against the model. It makes sense to suspend the algorithmic trading during that time until the situation goes back to normal.


Any metric, once it becomes a target, ceases to be a good metric.

Seems to me like the guys at Zillow never heard of this one.


Rumor has it that this tiktok video[0] was the cause of the pause and eventual disbandment of iBuyer program.

[0]https://www.tiktok.com/@seangotcher/video/700785597830984832...


No matter how popular, it’s hard to see a single video doing that.

The financials from last quarter are probably enough.


I think Matt Levine's take was pretty interesting: https://www.bloomberg.com/opinion/articles/2021-11-03/zillow...


Lack of liquidity.

Algorithmic pricing works in liquid markets. Housing is not a liquid market.


ctrl-F “adversarial environment” - no hits.

Markets are not physics - there is no neutral POV.


The simple answer is they didn't fact check with local realtors.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: