
Opinions I have formed about the “geospatial industry” - luu
https://twitter.com/mouthofmorrison/status/1265635034939248640
======
Waterluvian
Two geography degrees meant I saw a lot of ESRI. In fact I won a national
award and was flown out by them to meet Jack in San Diego. Everyone at ESRI I
interacted with were all wonderful people. But their software is just brutally
corporate and slow to respond to market changes.

FOSS GIS software is pretty good but suffers from the usual lack of budget.
Nevertheless, I’ve been able to push robotics + GIS packages for QGIS with
ease. Reaching out to my ESRI contacts about “where’s your indoor and outdoor
robotics suite offerings” received a response of only crickets.

As for Google Maps. The biggest tell is that there’s almost no geospatial
capability in the software. They don’t want you to analyze and understand.
They just want you to consume “where is X”. It’s a standard Google vector for
ads approach.

I’m going to rant for a long time if I don’t stop here so I’ll leave it at:
stop teaching so damn much ESRI and start teaching way more programming. The
biggest regret of my education was that I was mostly taught how to use
Geospatial tools rather than how to make my own.

~~~
twelvechairs
> FOSS GIS software is pretty good but suffers from the usual lack of budget.

FOSS GIS is incredible these days. You can do everything in open source - it
just sometimes requires a bit of work. Between qGIS (+SAGA/Grass/etc.),
postGIS, python tooling like GeoPandas, leaflet/cesium, etc. there's plenty of
tools for different things outside the ESRI ecosystem, you just have to learn
to use them. This is a long way from the dark ages 10+ years ago when much of
this capability was closed source or via some ridiculously complex text
interface of GRASS.

ESRI's market dominance on GIS _analysis_ is still a big thing though but only
really a hangover from the days when ArcGIS had much more capability and lock
in coroporate training was a thing. Now qGIS has essentially most of the same
capability, but ESRI's corporate training structures etc. are still there to
some extent. It is losing its stranglehold though, especially amongst GIS
'dabblers' who don't want to sign up and lock in to a massive corporate
account for whom the ESRI older-style value proposition isn't really there.
qGIS is even an accepted option for local government these days in many
places.

Geospatial is both complex and diverse though. There's a lot of industries
there and while there's big easy simple 2d GIS markets in mining, government
zoning and military, there's also a lot of other stuff going on and its all
moving fast. 3d, temporal data, drone photography, lidar and satellite etc.
There's lots of innovation happening there right now and new ESRI monoliths
being built.

~~~
ScalaFan
What does FOSS GIS software have to do in order to compete against ESRI? Or
which packages have the best chance to?

For example, few consider going with Oracle anymore now that you have
PostgreSQL but that took a long time. So, looking perhaps as far as 10 years,
which FOSS GIS software might even have a shot to start displacing ESRI in the
future?

~~~
twelvechairs
From my perspective at least theres a few main sides. From least to most ESRI-
dominant:

\- Web - ESRI never had a major horse in this race. Google Maps is still there
but more common now is Leaflet and Cesium for 3D is basically the only option.
Both Leaflet and Cesium are FOSS.

\- Server - As you say postGIS is now the industry standard

\- Programming - There's some people using ArcPy if you are in the ESRI
ecosystem but the plethora of FOSS libraries out there are probably bigger in
scope and fit together better than ESRI. PyQGIS is another option but in my
experience not as heavily used as other open source libraries (all of which
are based on GDAL, OGC standards and the Java Topology Suite)

\- Desktop analysis - This is ESRI's core stranglehold and I think qGIS is now
a competitive option for most users (certainly it has most of the core
functionality) however there's a lot of inertia to move from big companies
especially those that want ongoing external support/training.

------
campchase
Hey, this is Joe from twitter thread linked. I didn’t post this to HN but it’s
nice to see it on here. I was wondering why a weeks-old tweet suddenly started
popping off...

One thing I’d like to note—-I have learned a tremendous amount from other
people responding to the opinions enumerated in that thread. Some things I’ve
even started to question. So I’d love to hear your ideas—-especially if you
disagree with me. Just please do it constructively. Sometimes seeing your
ideas thrust onto HN without being prepared for it can be a harrowing
experience and I’d rather this time around be a productive one.

Much love to the brilliant minds of HN -Joe

~~~
terravion
The take that I had the most curiosity about was #4.

I get where it comes from, but what I observed when I was in the military was
if you asked staff in the operations center about data they would say,
something like "we've got two drone feeds, GPS trackers, field reports.. we're
drowning in all this data we can't look at it all..." Okay, great we got some
money from the theater, we can get anything you want, what do you want?
"Another drone feed." It seems that even if you can't use all your data right
now, the long run answer to today's unanswered questions is usually more data,
or did I not understand something about that take?

~~~
michaelt
Some problems people don't try to solve because they don't realise there's a
solution available - or don't conceive of the fact that, if they spent enough,
one could be developed.

If you take a time machine back to the year 2000 and talk to people who
collaborate on documents and ask them what would make their lives easier, very
few of them will say "invent an online browser-based real time collaborative
document editing system" \- many will just want more time, and consistent file
naming when e-mailing back and forth modified word documents.

------
perrygeo
I started my career as a GIS analyst and was steeped in the "geospatial
industry" for a few years. As I grew my career into software development, I
found an entire world of innovative practices and technologies that had been
largely hidden from view. Linux, Databases, HTTP, legit programming languages,
software best practices... these things are largely absent from the GIS
curriculum. In their place are bespoke, proprietary systems that solve the
Geographic part well but are painfully isolated from the Information Systems
that the rest of the world are working on.

My biggest realizations came using tools like PostGIS, GeoPandas, Rasterio,
and Numpy where geography is just another data type (albeit with some
specialized functions to work with them) and well integrated with the rest of
the software ecosystem.

You can see it reflected in the salaries - A "GIS Programmer" will often make
a fraction of what a "Software Engineer" makes. There's a reason for that -
for most use cases, geospatial isn't unique enough to warrant a walled garden
of technology.

~~~
newyankee
R also has some decent spatial libraries.

------
jandrewrogers
As someone deep into the cutting edge parts of the geospatial industry that
few ever see, I’d quibble on some of the points but it is a broadly accurate
description of the current state. The industry has been comfortably stagnant
for many years.

A critical distinction that explains much of what we see today is that
geospatial analytics has almost no relationship to making maps at a technical
level. There is a superficial similarity because they sometimes share
presentation layer. All of the popular infrastructure and tooling was
designed, either implicitly or explicitly, for making maps.

If you want to analyze some physical world behavior using multimodal sensor
data, the paucity of viable data infrastructure is insurmountable for most
companies. Unfortunately, this is what most organizations want to do these
days.

------
csteubs
I'd argue another point; satellites are a major bottleneck to GIS as we're
familiar with. With only ~300 or so commercial imaging satellites in orbit,
current revisit rates are sufficient for most use cases. Maybe one or two
images/day for X point depending on weather. Low earth orbit is getting
crowded, fuel to raise and lower orbit is limited onboard, expensive
maintenance cycles, etc.

I'm working on a startup trying to solve this problem now. The idea is to
"crowdsource" aerial imagery using a network of camera arrays on 80,000
commercial flights/day. We stream the video feed back to earth using in-flight
wifi and overlay the visual diff to create a map that shows minute-level
changes. There are huge built-in cost savings that come with not having to
develop, launch and maintain a satellite constellation, and with a vastly
higher revisit rate (as high as 4,532/day in some areas). We're at
notasatellite.com

~~~
sgt
Streaming the video feed using the onbord wifi? I think you are overestimating
the quality of the Internet connections available on flights.

~~~
notahacker
Having worked on inflight connectivity apps and Earth Observation
respectively, for a couple of startups I'm inclined to agree overall.
Workarounds would be:

[i] ground sync, which would be fine for a lot of use cases where higher
revisit rate and sometimes less cloud cover is sufficient to make the imagery
valuable, and open up non-wifi flights. Ground sync can be over regular 4G or
even swapping flash drives [some W-IFE boxes update this way, though
infrequently].

[ii] Getting dedicated bandwidth from the satellite/a2g internet provider, who
can potentially allow you a lot more than is made available to an individual
consumer, but obviously at a hefty charge.

[iii] using streaming codecs designed for low kbps rates - though this is
obviously lossy, and pixel perfection matters for a lot of EO use cases.

~~~
csteubs
Great points! We're incorporating aspects of all three workarounds. Ground
sync is already possible (wifi isn't available on many regional short-hop
flights so we eat the latency), and we're discussing dedicated bandwidth with
in-flight wifi providers. We've been paying the $14.99 market rate for
customers testing the MVP who receive free in-flight wifi for their efforts,
so the resulting cost would be much cheaper. To your last point, we load the
last-known tiles expected to be captured on the flight in order to take a
visual diff in the air, and then send that diff back down to earth instead of
the entire image. This results in a roughly 80% savings in data but requires
more hardware.

------
littlestymaar
> The most successful and ambitious mapping project of all time, Google Maps

Google map is indeed a commercial success, but it's far from “the most
successful and ambitious” when it comes to mapping itself, quite the opposite.
State projects (even those developed before satellites where a thing[1]) are
way better, and even openstreetmap achieves a better coverage and higher maps
quality.

It's the “most successful and ambitious” _commercial_ project, which is
coherent with the rest of the thread.

[1]: for instance: [https://www.sovietmaps.com/](https://www.sovietmaps.com/)

~~~
ramraj07
Do you have a citation for OSM? My experience in first world countries
especially where Google had ground truth project are still much better in
Google than osm.

~~~
morsch
Casually browsing OSM in my area, I see individual park benches, garbage cans
including type, road surface material, doorways to buildings, power lines,
location of springs. And that's just what Osmand is rendering, I'm sure I'm
just scratching the surface.

Google doesn't even attempt to map most or all of these things here, much less
expose this information. OSM attempts to build a much better map, and for my
region, easily succeeds.

That's not to discredit Google Maps. Almost all of the time, I don't need the
best map, I need the kind of yellow pages, yelp, sightseeing guide, live
traffic and transport, navigation mashup Google is projecting onto their map
with great skill. I think it's their best product.

------
jld
As a mid-career software engineer who would be interested in getting into the
"geospatial industry",

A) What technologies should I learn to allow me to get my foot into the
industry? I have historically focused on being a UX focused web engineer. I'm
sure that experience could surely transfer, but am interested in learning the
technologies and/or understanding that is more core to geospatial problems.

B) As someone who loves maps and imagery, but does not want to work for the
military industrial complex, how do I avoid point #3. "In geo, you either die
a hero or live long enough to make the majority of your revenue from defense
and intelligence."

~~~
campchase
A: Break it into two parts, vector and raster. For the former, focus on Mapbox
and associated libraries/standards (VectorTiles, Leaflet, GeoJSON, etc.). JS
is the name of the game in vector land—it’s amazing what you can do all on the
front end now. For backend/DB tech, look at PostGIS.

For the latter, I’d focus on Python to start. Look at GDAL (and it’s various
Python interfaces/wrappers like rasterio), Cloud Optimized Geotiffs, and the
SpatioTemporal Asset Catalog spec.

In both cases, watch what the leading companies are putting out (Mapbox,
Azavea where I work, Development Seed, etc.) and not what OGC is doing. One is
a leading indicator, the other is a lagging one. Sorry OGC, I still love you
and you play an important role.

B: Get comfortable with the idea of working with Defense. For better or worse,
they’re the ones with the budget to take on humanitarian work, disaster
response work, even climate change adaptation work. That’s my advice. Possible
to avoid it altogether, but only if you work on an app that is domain-focused
(eg Trulia) and incidentally geospatial.

------
artificialidiot
I agree to all points. For point 6. Mapbox is a big contender for ESRI's
current position.

For 8. Openstreetmap, GDAL and associated open source ecosystem offers a
solution to proprietary format and API of the month problem to a degree. It is
not surprising that there is no money flowing if mapping is not a core
business of the users.

Aerial and satellite imagery is the heavily regulated and lucrative part of
any mapping solution. If you are allowed and able to obtain that, you can
probably get by just selling the raw imagery without geospatial processing.

~~~
jaakl
Mapbox tries to replace Google maps, ESRI is different category which is only
slowly challenged by enterprise vendors (Azure and other clouds) and FOSS:
PostGIS and other sptially enabled databases which have more and more geo
features, making specialized GiS to a niche.

------
geogra4
I used to be heavily involved in geospatial and a whiz with the esri toolset.
I worked at a software company that made a custom GIS tool for visualization.
The problem is that it's underpaid and underappreciated. I had little
transferrable experience and most GIS related jobs don't pay that much.

For me my GIS job was mostly a time to learn programming and get a real it job

------
zeku
I entered college to get a Human Geography B.A. and go on to get my PhD. After
realizing this meant I must leave my home area and my family(I live in a major
US Metro but not one of the biiiig ones) AND the only jobs were Professor or
Think Tank(very cool, very walled garden and I went to a state school) I
decided to switch to Physical Geography B.S. My curriculum had 3 GIS classes
and a Remote sensing class among other physical geo classes. After almost
finishing this degree I realized it would leave me with only GIS Tech and
hopefully GIS analyst positions to have which is a smallish market and might
get "automated away". Also it doesn't pay super great and I was worried about
my prospects if I didn't like it. So I dug bag in and also got my B.S. in
Computer Science.

I wanted to be that Geospatially focused programmer. So I graduate with my
B.S. in Geography and another in CS. I get interviews with a couple of random
defense contracts and directly for federal government, but I turn them down
because honestly I don't want my work going to wars I might not agree with
_and_ I would need to leave my family. I also happened to get an interview at
ESRI for fun when I still was deciding if I should leave my metro or not and
they laughed at me because I didn't have an internship--something that is very
common in Toronto, LA, NYC, but not common in my mid-US metro CS scene.

So I now have a nice programming career solving "big data" problems completely
away from geospatial, because the industry is tiny. I don't really know where
I was going with this but this industry lacks opportunities for programmers
compared to all the other stuff they could be doing. I'm someone who put years
into wanting to break into the industry and do the tech work, if I wasn't the
prime candidate for a junior geospatial programmer role then I don't know who
is and the industry still couldn't get me in it.

~~~
vharuck
The disconnect is from one of the beliefs listed in the Twitter thread: "There
is no 'geospatial industry,' only industries with spatial problems."

I'm a statistician with a state health department and do some GIS work. If I
were more skilled with geospatial analysis, there's no doubt the decision
makers I work with would be delighted.

If somebody wants to use GIS skills, they should keep an eye on IT or analysis
roles. They might not specifically ask for it, but you can still sell it in
the interview if it'll help solve their problems.

If you're hoping to make the big bucks or work on cutting edge problems, then
staying local will be difficult.

------
jhoechtl
ESRI is essentially the Oracle of GIS. To big to fail and to powerful to be
wiped away. On the sweet spot of acceptably behind the trends. Yet nobody go
ever fired because of acquiring ESRI Software.

------
jonny383
I really wish this trend of writing blog posts in 240 character chunks on
Twitter would pass. It's so damb frustrating to read them.

~~~
simonw
Weirdly I found I really like them.

The thread format forces people to break their thoughts up into chunks - a
little like bullet points on slides but 280 characters makes the chunks much
more information-full than just a bullet.

I can like or retweet specific interesting points as I read them.

Individual tweets in a thread can include images or link previews or link to
other tweets.

What platform are you reading them on? I find them fine to read in the Twitter
iPhone app - almost indistinguishable from a article elsewhere. Is that not
the case outside of the app?

------
TheSpiciestDev
I've enjoyed working on an enterprise, GIS-focused application over the past
few years (with no prior, formal geographic background) and one of the take-
aways I always enjoy sharing with family and friends is the "point in polygon"
question [0] and it's simplicity. I'm fun at parties. I have certainly learned
how much more intuitive data is for users to explore when you offer (if you
can) spatially.

[0]
[https://en.wikipedia.org/wiki/Point_in_polygon#Ray_casting_a...](https://en.wikipedia.org/wiki/Point_in_polygon#Ray_casting_algorithm)

~~~
CyberDildonics
Are you really doing the test in the wikipedia article over and over? Why not
convert a complex polygon to triangles? Things like this were established in
computer graphics in the 70s.

------
ReactiveJelly
I mostly work on desktop programming, and I've dabbled in web stuff...

And I don't know what GIS is. I see it talked about _a lot_ but I don't
understand how there can be so much money and interest in geography.

I understand web stuff being huge, because I'm online almost all day. But what
_is_ GIS? Is it something businesses do and consumers don't see?

~~~
thinkingemote
[https://en.wikipedia.org/wiki/GIS](https://en.wikipedia.org/wiki/GIS)

GIS is everything location. From modelling pandemics to rainforest loss, to
logistics, to Tinder and Pokemon Go.

~~~
Crespyl
Uber, Lyft, etc are other useful examples.

I think some of the sort-of-obscurity/confusion comes from the fact that, as
the OP tweet gets at, you don't really see a very visible "GIS
Industry"/products most of the time, you just see tools that happen to have
maps embedded in them, and those often require a lot of backend GIS work to
power them.

Finding a rideshare and planning your next oil well don't, at first, seem to
have a lot in common, but on the backend there may be a surprising amount of
overlap.

------
edleranalytics
I have spent a fair amount of time learning GIS in undergraduate (urban
studies program) and applying those skills in local government/real estate
development. I have to agree with a lot of what he is saying. Many of the GIS
skills I learned in ESRI during University pigeonholed me into the same few
glorified data entry/basic spatial analysis roles. There are also a lot of
people with this niche skill set, which makes getting these positions really
challenging. There is an emerging field called Civic Technology that focuses
more on building out geospatial applications for the public good. I hope more
public policy/urban studies degree programs shift in this direction of
building out tools for social change rather than learning the same few
applications of ESRI ArcGIS.

------
notagoodidea
During my academic history with GIS/remote sensing/Geospatial (and still going
on), I have seen how the same tools around the various fields in got plugged
to in, mostly for the data pre-processing, processing and analyst.

The sacro-saint trilogy, for me at leat, is GDAL/OGR, PROJ and GEOS. They are
the building blocks behind QGIS, PostGIS and most of spatial libraries in
various programming languages (Rasterio and Fiona in Python for example).

On a more GIS analyst perspective, most of it can be done with ESRI or QGIS
and if you can get you hand dirty PostGIS is an amazing tool that I which I
had the time to invest way more (or even Spatialite, sqlite flavour of GIS in
database). I really which than Shapefiles goes away for good and be superseded
by GeoPackage coupled with GeoJSON.

The web had a hard time to be a proper tool for visualisation on GIS/remote
sensing due to the big size of the various files but Cloud-optimized GeoTiff
and Vector Tiles are the game changers but came too late.

Commercial vendors tends to go for the cloud and platform lock down as Planet
and their super-powered jupyter notebook and access to the catalog. Same for
Google Eart Engine.

We see also stuff like CartoDB and the Database As A Service for geodata.

In a world when Geodata are highly available with better resolution than
never, they are so big to store somewhere than it can be crazy. Working on
time series at country scale (or even world scale) require now a lot of store
space and calculation power for high resolution images (Even just Sentinel or
Landsat at 10m and 30m resolution respectively). Small-scale is easier than
ever and lower resolution can be managed on a laptop. I remember when I had to
work with time series dataset of Sentinel-1 and -2 images when we were
preprocessing 10-12 Tb of data with a pretty long chain of operations to do
before handing it out to another team for furter data analysis. Some academic
lab are storing Petabytes of remote sensing data gather during the time.

"Geospatial Industry" is way more than looking at ESRI or Google Maps/Earth. A
lot of things can be and are done with various research and industry projects
than require high integration with the spatial field and the temporal
analysis. And the quality of what you have as tools vary highly between field,
actually I am working for a PhD on a Bayesian spatio-temporal modelling with
lattice data. It is the wild west and you need to cure a lot of R libraries to
get what you need. I don't know how much you can do in Python because we had
to avoid any MCMC method due to constraints on processing power available :
low in fund and no easy access to calculator, we settled for R-INLA who is
very interesting on its own.

------
stickfigure
This seems to have been written by someone unfamiliar with real estate, local
government, natural resources, or any other traditional line of business that
depends heavily on GIS. I don't know what my county pays for ArcGIS but it's
probably substantial, and I suspect there are multiple IT employees devoted to
maintaining the system.

Replicate that across nearly every single political locality in the US,
probably the world. The industry is huge.

Also, "long form twitter" is dumb.

~~~
campchase
Hi stickfigure, this is Joe from the dumb long form twitter thread you just
wrote five sentences critiquing. Thanks for your feedback, I really appreciate
it.

------
atrocious
My academic and work history is with GIS, but I've spent the last 4 years in
fintech. What does the cutting edge of GIS look like in tech nowadays?

~~~
growlist
Take a look at what OGC are working on.

