
I scraped all of Fandango's movie listings - chadmhorner
https://readypipe.com/blog/fandango-ticket-prices/
======
brianbreslin
I can vouch for their blurb on CMX in Miami. Truly rapacious experience. $28
and the popcorn isn't free, and service was awful (food delivered in the last
15 minutes of a 2-hour movie after ordering before the movie started). /rant

~~~
ekianjo
how do they stay in business?

~~~
brianbreslin
my theories are: no other super close theater (though one opened less than a
mile from them at 60% of their price), or grandfathered in super cheap rent
(they opened with the new building), or tourists don't know better, but never
return.

------
deanclatworthy
This is really interesting, even as someone who is not from the US.

I find it astonishing that it's only 13 bucks for a movie ticket over there
(around 11 euros). Here in Finland tickets can easily cost over 20e if you see
a movie on the weekend.

~~~
pesfandiar
How much do you pay for an oversized soda drink and a family-sized bag of
popcorn for one? I believe the concession stand is a major source of revenue
for theatres compared to the ticket price itself.

~~~
applecrazy
Yes. I read on here a while ago that theaters actually don’t receive much
revenue from movie tickets, due to the studio negotiating a 60% or so share of
ticket sales. So most of the money has to come from expensive concessions.

~~~
bscphil
Hmm, if that were true as it's said to be, you'd think all theater chains
would offer cheap subscriptions, since getting people in seats is way more
important than maximizing their revenue from ticket sales.

------
hunter23
Is this the same technology that was used by Yipit to aggregate daily deals
back in the Groupon/Living Social days? What languages do you support and how
are you different from scrapinghub.com ?

~~~
spulec
Yes, we've been improving our scraping technology for the past 8 years as
we've worked on YipitData (the #1 provider of web data to wall street).

Python is the only supported language right now. Scrapy is an awesome project,
but we have a very different approach. We strive to be Flask instead of
Django.

If you want to, you can use ReadyPipe entirely in the browser through jupyter
notebooks instead of needing to setup a local environment. This is especially
helpful with more complicated systems using Selenium and Puppeteer. We discuss
a lot more of the features that differentiate us on our homepage:
[https://readypipe.com/](https://readypipe.com/) and the docs
[https://docs.readypipe.io/](https://docs.readypipe.io/)

Feel free to reach out (email in profile) with additional questions.

------
omarchowdhury
Great piece of natural content marketing and demonstration of product. Keep it
up.

~~~
sh87
Given the effort that's clearly gone into this article, it would seem
appropriate to mention/link to the underlying product near the top of the
article. Good job, marketing in a way that didn't annoy.

------
hathawsh
Is the author aware that the prices depend on the day of the week? Competition
in Utah has driven all the big theaters to charge only $5 for all movies every
Tuesday, making the movies cheaper than even Wyoming. I'm curious to learn
whether the same thing has happened elsewhere.

~~~
chadmhorner
Yep! We scraped on both weekends and weekdays, and also limited our analysis
to "Adult" tickets for "Standard" class. Typically the special Tuesday tickets
(they are often on Tuesdays) will have a different designation.

Possible we didn't catch everything, but in aggregate it should be a
reasonable estimation.

------
gammateam
very cool! I always wanted to know these things more often

I've done the SF/NYC/Miami circuit, completely skipping the heartland like
everyone else, and these ticket prices are baked into my budget.

Oh you want 3D/Special Sound System/random perk? Prepare for $22 and I'm okay
with that

For the actual residents of these cities, there isn't often a place for you to
watch something you chose to see in a loud usually spacious environment. These
are still undervalued entertainment experiences, amongst the sea of
entertainment choices, for actual city residents.

I wonder what the industry's own pricing models show

------
tlow
Is this a violation of Fandango's TOS?

~~~
minimaxir
Most likely, although there isn't a practical issue unless they redistribute
the raw data.

~~~
daef
seems like they are advertising a scraping service -> "Want to get the data
yourself?"

~~~
minimaxir
Yeah, that's a legal gray area. (also, from looking at the example code, it's
unclear what the service is value-adding anyways; you still need to do most of
the legwork by figuring out what/how to scrape).

~~~
gmiller123456
It is not gray at all, it's a violation of their terms plain and simple. They
only gray part is whether or not it's enough of a bother to Fandago to
actually initiate legal action.

~~~
edm0nd
While it is a TOS violation is this data being scraped really hurting
anything? I dont think so.

~~~
minimaxir
That’s why I said in practice nothing would happen. However, if the data is
used to build a competitor or it contains PII, it could be problematic.

~~~
Reedx
Plus it consumes server resources and bandwidth.

And some scrape so aggressively that it's like being DDOS'd.

------
whatever_dude
Tangent, but I wish someone would disrupt the movie ticket industry.
Fandango's mobile app is one of the worst apps that I've ever seen: it's very
unresponsive and full of advertisements, spams and interstitials I do not care
about. And yet it's the only way to buy tickets in my area.

I mean I'm already using the app to spend money, why not offer me a good
experience instead of bombarding me with crappy ads?

~~~
ThomPete
I assume it's because they want to avoid paying fees to Apple no?

~~~
omarchowdhury
I don't see how that transfers. Paying fees to Apple also might not apply with
Fandango, as it delivers a service fulfilled physically and not digitally. I
would suspect the ad bombing in the Fandango app is due to razor thin margins
in the movie tickets purchasing business.

~~~
ThomPete
Hmm good point.

If one was to take on Fandango would you have to take over their contract with
the movie theaters or could I be a competitor to Fandango?

~~~
omarchowdhury
They definitely have direct contracts that are not easily replicable.

------
bertylicious
You might want to clean up that messy prototype code. It's annoying to have to
scroll horizontally and the nesting level is just insane.

------
bob_theslob646
How did you verify that you got all the theaters?

Having looked at the code in regards to only using zip codes that end in 1 in
order to eliminate overlapping neighborhoods, it seems like you missed some.

Why not grab all of them and clean the data? Maybe it is too intensive, but
insisting you have "all of Fandango's movie listings" is actually false.

------
GauntletWizard
This is a really cool example of ReadyPipe - It's a great demo of scraping a
site that's got some intent not to be scrapeable :D

On the other hand, your data just seems to follow
[https://xkcd.com/1138/](https://xkcd.com/1138/) \- What happens to the prices
if you adjust for cost of living index? What about if you adjust just for
population? It looks to me just like a Cost-of-Living heatmap.

------
kumarm
>> Only cities with 500,000 residents or more (per Wikipedia) were included in
this analysis

So 35 cities? Interesting read none the less. Good to see independent theaters
are still doing well.

~~~
chadmhorner
Yep! Just those 35. Main reason being that it’s actually a pretty manual
effort to assign theaters to cities, because there could be a bunch of
different “cities”, as Fandango classifies them, that are all really the same
city - like for Miami as an example, you can have Miami and Coral Gables and
Miami Beach, which you’d really want to classify all as Miami.

Long story short, didn’t want to go to that process for too many more cities
haha.

~~~
dajohnson89
Maybe theres a mapping somewhere, between List<Zipcode> <-> Metropolitan area.

~~~
rhcom2
I believe the US Census offers that [https://www.census.gov/geo/maps-
data/data/ua_rel_download.ht...](https://www.census.gov/geo/maps-
data/data/ua_rel_download.html)

------
albertgoeswoof
Why don’t you summarise what you found in the title, and give me an indication
of whether the I should read your article or not

~~~
chadmhorner
Honestly planned to but was character-limited

~~~
minimaxir
The HN guidelines
([https://news.ycombinator.com/newsguidelines.html](https://news.ycombinator.com/newsguidelines.html))
favors the article title w/o numbers, in this case "Insights From Analyzing
Fandango Ticket Prices"

HN specifically discourages clickbaity titles. (in this case, your title "I
scraped all of Fandango's movie listings, here's what I found" is also
misleading; you didn't scrape all the listings)

~~~
chadmhorner
Ah, noted. Will keep that in mind in the future

