Hacker News new | past | comments | ask | show | jobs | submit login
Maps.earth – free and open-source web maps (maps.earth)
696 points by ellenhp on Aug 22, 2022 | hide | past | favorite | 111 comments
Hey everyone, a while back I posted about an open-source maps project I was working on called Headway. I just wanted to share an exciting update. I made a few architectural improvements with the help of some really awesome contributors and Headway can now run full-planet instances. I'm really excited with what we've managed to come up with.

Right now the full-planet demo server is in Germany and lives at [1]. A brief explanation is at [2]. I'm hopeful that the project will continue to improve with time and eventually end up being a real contender in the web maps space. I'm talking with folks about the potential for getting some more robust hosting options too which is very exciting.

For the moment though, Headway has some usability issues. It's missing ETAs, has no steps list for directions, or proper geocoding search results page, etc. I'm going to get to these issues with time, but please drop me a line if you're interested in helping out. Especially interested in help with web frontend, design, or internationalization. Development takes place on GitHub [3] and discussions tend to happen in the matrix room [4].

[1] https://maps.earth/

[2] https://about.maps.earth/

[3] https://github.com/headwaymaps/headway

[4] https://matrix.to/#/#headway:matrix.org




Link to the previous post: https://news.ycombinator.com/item?id=31536217

Congrats on all the progress in 3 months!


Hey, this looks great, thanks for working on it! I'm the creator of Mapzy[0], an open-source and self-hostable store finder. We're trying to focus on privacy as much as possible, but currently use Mapbox since it's better than Google Maps and we didn't find any other alternatives.

Do you think Headway could be a replacement for us? We currently use the Mapbox JS library to create the pins etc. - could we use Headway with Leaflet JS for example? We also use the Mapbox geocoder, mainly for reverse geocoding of addresses. How does Headway's geocoder compare?

0: https://github.com/mapzy/mapzy/


I’d recommend taking a look at Maplibre GL JS[0], which is a fully open source map library. It was forked from Mapbox GL JS just before licence change, therefore it’s also fully compatible with the style spec etc.

It’s also actively maintained and has a strong community behind it.

0: https://github.com/maplibre/maplibre-gl-js


If you are in this space deck.gl [0] is well worth cehcking out. It does scale at speed, 3d (more kinds), motion and effects extremely well.

[0] https://deck.gl/


Thanks, I'll check it out!


Headway was designed more for consumer use-cases than anything else. There's been some talk of adding the ability to customize it in the future but I don't have time for that because I need to focus on the basics first. You could fork it though, or just use some of the data automation (respecting the Apache 2.0 license). I replied to a child comment here but my recommendations for geocoding would be Pelias or Photon, and I like maplibre-gl-js. You can (should!) use planetiler for generating your mbtiles files and I've had generally good experiences with tileserver-gl-light, but many people recommend using PMTiles instead. It kind of just depends what your requirements are. For privacy-first stuff like Headway I can't require that people make requests out to S3 or Cloudflare, it has to be self-hostable which means a traditional tileserver is the only way to go.


Thanks for the info! I'll look into your recommendations.


We're in the same boat using Mapbox, we've been debating leaving for the OpenStreetMaps container since we only use them for static images and map display. https://github.com/Overv/openstreetmap-tile-server

Our bill last year was $80k and honestly its right on the cusp of being worth the effort to switch.


> Our bill last year was $80k

You might be able to pay someone from the project to help you switch. A nice way to give back and it will probably be a smoother transition.


Have you looked into Nominatim [1] for reverse geocoding?

[1] https://wiki.openstreetmap.org/wiki/Nominatim


Nominatim requires a lot of resources for full planet installs. I'd recommend looking into Pelias or Photon instead. They're both amazing. Headway has used both of them at various points. Photon is definitely easier to set up for the planet, though, you can just download a pre-computed search index from GraphHopper and you're good to go.


I haven't. Thanks for the recommendation!


Try MapTiler. It works fine with what you're using. For geocoding, I dunno, we run our own show.


Thanks! MapTiler looks great, but a bit on the pricey side for our use case.


We picked it specifically because Mapbox was too expensive. We pay like between 20 and 30 euros each month for tens of thousands of transactions. You can also buy tile sets and serve your own vector tiles, but it isn't worth the hassle for us.


Have you ever compared it to https://github.com/Overv/openstreetmap-tile-server ?


No. We no longer want to bother with monthly updates and the overhead of maintaining our own tile server over <30€/month. We tried to make our own raster tile server several years before vector tiles were a thing, using tilecache, then tilestache and just gave up because of the storage and hardware requirements. Plus there's a lot of work involved in customizing your maps - we didn't want to show the standard OSM map built for mappers in mind but a transport oriented map for fleet tracking. MapTiler has almost everything we want except good Africa coverage.


Makes sense, thanks for your explanation.


You can spin up an own instance in a docker container easily (if you self host everything anyways).


Yeah just saw that after making the comment, thanks!


Very impressive!

Is it just me (Firefox on Android) or does the UI really not allow tapping landmarks (shops, restaurants, subway stations)?

Also, directions don't seem to work right now (infinite loading) but I suppose this is due to the server receiving the HN hug of death right now.

By the way, for anyone looking for the mobile (non-web) version of this: Organic Maps is a fantastic frontend for OpenStreetMaps (and of course there's the good old OsmAnd, though I like Organic Maps better these days).


OsmAnd doesn't seem to get many updates nowadays. I get that it's open source and we can't demand anything for free and all that, but they have some monetization options that should bring in some money, but aren't really worth the price IMO. The issue tracker is full of bugs and feature requests marked as nice to have, but almost nothing happens.


Recently switched from Osmand to Organic and am quite impressed. Rendering and routing are much faster and there are no crashes even with very long routes.


Agreed, it's so much more pretty and the UI is slick. With Osmand I always seem to end up with stuff on the map from a previous search/navigation that I don't want any more, but can't figure out how to get rid of.


Thanks for Organic Maps!

The website doesn't seem to work for me either on mobile.


I'm a little confused as to what this actually is.

Is this a route planning built on top of OSM? Does this bundle OSM data "all in one" so that you can more easily host an OSM server?


OSM doesn't have a server per se (other than the OSM API server). It's a dataset for the most part. Headway lets you bring up a server that will render maps tiles, allows you to search for places and get routes between places. It's trying to be a user-friendly UI for various pieces of the OSM ecosystem. There are a lot of amazing pieces of software out there that work with OSM data, but there isn't an easy way to bring them up, and there's no unified frontend for all of them other than Headway (that I know of).

Edit: To the comments telling me I'm wrong about this, remember that Headway has a geocoder and routing system, along with a transit trip planner (currently disabled on maps.earth though). These are not things you could get with off-the-shelf solutions that include a GUI until now. I also want to self-host my mapping software for privacy reasons, and Headway makes that easy too. I probably need to be better about explaining what all Headway does that's different than just bringing up an OSM tileserver though. Thank you all for the feedback :)


There are services hosting OSM data, often times requiring paid API access for anything other than personal use.

OSM is amazing, for sure, but standing up even a small OSM service that only services a map in your local town/city/community is too daunting for most people (myself included).

Considering there are many other comments in this thread that mirror my main question, I would try and figure out messaging. Is it a particular tile rendering on top of OSM data? Is it trip heading information? Is it helping people stand up self hosted OSM servers? Is it helping with just a few components of that process? If so, which components? etc.

It's hard to know what the "value add" is here and a lot of this is my own ignorance on how the OSM ecosystem works but that's kind of what I would recommend trying to convey to people.

I wouldn't assume deep familiarity with OSM. I would like to hear more, especially concerning tools to help stand up a self hosted OSM server more easily, as I would suspect many other people would, but it would be nice to have some basics spelled out instead of assuming I have deep familiarity with the sprawling and intricate OSM ecosystem.


> tools to help stand up a self hosted OSM server more easily

I think that’s already the wrong way to think about it. For some reason that I do not comprehend, OSM comprises at least 3 different file formats, and 5 or 6 different ways to store that data.

When you ‘set up an OSM server’, what you really do is pick one of the storage methods, and then import the data with one of a variety of tools that were built over the years (and are probably compatible with only one or two of the storage methods each).

Seriously, the OSM ecosystem is a clusterfuck. Anyone that aims to standardize it into something easy to deploy has my full support.


Thanks.

It would be great if this could be stated on the page.

I kept hackernews open in order that someone would explain to me what this actually does. Your comment thankfully fullfilled this.


Thank you for the feedback :)


I built something along the same lines many years ago, with routing but not geocoding.

https://github.com/stevage/saltymill

Yeah, it's a pain in the arse getting all the bits together. And I wasn't attempting full planet scale.


There no frontend to bring up an OSM? Literally you need a single line to load one up:

  (new map('map_div')).addLayer(new TileLayer('http://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png'))


Using tile.openstreetmap.org is subject to the OSM tile usage policy: https://wiki.openstreetmap.org/wiki/Tile_usage_policy


To add onto this, those are raster tiles and the industry is moving away from raster tiles for a multitude of reasons. Headway doesn't bulk-download tiles from OSM, by default it takes an OSM extract from BBBike (with explicit permission) and brings up a server based on that. It generates vector tiles with planetiler and serves them with tileserver-gl-light. I've put a lot of thought into the way it all works in an effort to minimize load on the OSM ecosystem's infrastructure, provide attribution where required, etc.

I can promise you that there's no one-line replacement for the work I've done here unless you want to go out and pay for a closed-source product.


Their data is also subject to this policy: https://wiki.openstreetmap.org/wiki/OpenStreetMap_License

They also heavily discourage bulk downloading tiles so I'm not sure if you're that much better off building your own tile server.


If you are building your own tileserver then you take one of the OSM dumps for your area.


Note that the {s} part is for backwards compatibility, and that https://tile.openstreetmap.org is enough, or maybe even a tiny bit faster, now that we have http 2.


OK, the GitHub page spells it out better [0].

> Self-hostable maps stack, powered by OpenStreetMap.

In addition to listing out some of the required specs (for a localized area) [1]:

> ... generation ... needs ... 8GB of memory ... running ... around 4GB ...

> ... recommend ... 50GB ... disk space

[0] https://github.com/headwaymaps/headway

[1] https://github.com/headwaymaps/headway#system-requirements


could we.... uh, have our server for a specific city or town for example if the use case is around that area only? that would save a lot of resources


Yup! That's how most people run Headway right now. You can probably get by with around 2GB of RAM for a medium-sized metro, possibly even less.

Also for full-planet requirements, the OVH box running maps.earth has 64GB of RAM and 6 physical cores and is really struggling right now. Load average is about 40, most its CPU time is spent on the Valhalla service. Really enjoying watching the metrics right now :)


You may have chosen Valhalla for other reasons, but FWIW I’ve found Graphhopper can handle 1-2 orders of magnitude more qps than Valhalla on the same hardware.


I wasn't happy with cycling directions in GraphHopper and I was having trouble getting route shapes from its routing endpoint which made it really difficult to plot route polylines on the map. I've also been looking forward to exposing some customization of routing parameters since valhalla allows you to do this fairly trivially. That is good to know, though. Valhalla really is getting hammered by all this traffic. :/


What problems did you face with cycling directions?

Regarding the route shapes I can only guess: we use an encoded polyline as default for a very compact response. You can either disable this or use our JavaScript library, or there is a new UI https://github.com/graphhopper/graphhopper-maps

Regarding routing parameters: our custom_model approach is more powerful than simple parameters and still not complicated: https://www.graphhopper.com/blog/2020/05/31/examples-for-cus... See e.g. the several bike examples and see how to get an world wide instance up and running: https://www.graphhopper.com/blog/2022/06/27/host-your-own-wo...

In recent master you can additionally control how elevation influences your route (e.g. prefer or avoid them entirely or steep sections or similar)


could you put up munim or other similar metrics site for public? it would be nice to see such a high load


Wow this is pretty amazing. For those that haven't tried, it's really difficult to piece this sort of thing together. So the fact that this is all provided as a Docker Compose configuration is really awesome.


How do you handle map data updates? Do you regenerate everything from scratch from the full data set, or can you update it incrementally with patches? Is it possible to put the tiles on a CDN like cloudflare?


This seems really neat. I have a question that I hope isn't something I'm missing, but is there an easy way to change the rendering ruleset? I ask because I've been wanting to build some custom maps for mountain bike trails in the state where I live and have them served up similarly.

Whenever I've looked into other projects for this it's either focused on rendering tiles from Mapbox data (where not all the OSM tags -- especially not MTBing ones -- are available) or really old info or presumes that someone else will be generating tiles.


This is tracked here: https://github.com/headwaymaps/headway/issues/85

It's going to be somewhat tricky because of a few mediocre decisions I made early on in the development process, but if you were up for helping out I'd love to get a bring-your-own-style experience working in Headway. Long-term, I want to add the ability to easily brand Headway and customize it for your own needs without forking it.


Thanks for pointing me there. I'll give it a read and see what I might be able to do.


I tried the hosted app at https://maps.earth but it isn't really usable at present. How do you actually search for and navigate to places?


The search box in the top left should bring up a bunch of autocomplete suggestions when you type stuff into it. Clicking on one brings up a place card view [1]. If my demo server falls over that will fail to populate. Once you have a place card you can hit "bike there" or something and then populate the "From" field that appears again in the top left to get a directions view [2]. Not ideal, the frontend was built in a few weeks by someone with very little frontend experience (me) so I did my best.

[1] https://maps.earth/place/openstreetmap%3Avenue%3Away%2F12903...

[2] https://maps.earth/directions/bicycle/openstreetmap%3Avenue%...


I guess it may be failing due to the request spike. Even otherwise, searching for common places like "San Francisco" or "New York" shows a dozen results all over Europe and South America, not the one place most people would be looking for. Street address search doesn't seem to work either.


Not working for me either. No results show up, even for common places like LA.


Yeah, I've noticed that at least the way Pelias is configured in Headway, searches for administrative areas don't seem to work, like at all. I'm not sure if that's a config issue or an issue with Pelias though. Address search should definitely work as long as there's an appropriately tagged OSM node for your address, but the server's load average is 40 on a 6 core OVH box in Germany so you might have to be (very) patient to get anything back until the load drops to a level it can deal with.


Hi! Maintainer of Pelias here and co-founder of Geocode Earth where we continue to fund Pelias development through our SaaS and consulting.

You need to bring Who's on First data in to support administrative areas, it won't use admin areas from OSM. This is a bit of a technical choice (WOF has some nice features like stable IDs and a rich hierarchy, and a data format that's much easier to work with), but also the result of the Who's on First project and Pelias being developed together while we were at Mapzen.

Don't hesitate to reach out to us if you need help getting things set up.


Is there a contact point you'd prefer me to use for reaching out? I'm pretty sure I'm using WOF for bringing up Pelias but it's entirely possible I'm configuring things wrong. I'm also having a lot of trouble getting good performance out of the viewport biasing on the autocomplete endpoint (`focus.point`). Photon has `location_bias_scale` which allowed me to do very strong viewport biasing for zoomed-in queries and eliminate it almost entirely when zoomed out.

Is this something that could be added? I don't mind doing a bit of legwork if need be. This is important enough that I'm willing to throw a few weeks at implementing it.

The pelias.json for this instance is at: https://publicdata.ellenhp.workers.dev/planet-v1.16.pelias.j...

I'll go double check that the placeholder container is operating correctly, actually. :)


Is it possible to access it via less js heavy frontend? Owner of Nokia N900 here :D Would great to have Ghost in the Shell style super lightweight maps


Unfortunately no. maplibre-gl-js requires webgl for vector tile rendering. That's how you get snappy zoom performance, reduce the load on the end-user's cellular connection, etc.

Sick phone though I'm a little bit jealous :)


I can also imagine requesting route via terminal and getting just list of directions which is many times enough


I have seen (but cannot find now) a command line tool that you give two addresses, it queries OpenTripPlanner for itineraries and displays the result using walking, train etc. Unicode emojis.


thanks! it works via REST/GQL apis so should be easy to write a cli


You wouldn't be as jealous if you knew how hard it is to find batteries these days.


I found a 1600mAh one from AkkuKing on German Amazon, I thought it will be harder,


Time to upgrade to Librem 5 ;)


no keyboard, bigger size :(


This is very interesting (and impressive work). I've been thinking for quite some time that it would be very useful to have a local map server for offline use.

A lot of applications need to show data related to location (e.g.: a photo app might want to show a map of where a photo was taken). Having a local service to resolve these would be superb.

I know this product has a different goal, but I wonder if the backend side of it could actually fill in this role. Essentially exposing location/mapping services to local applications.


This is amazing, thank you very much! It is really tempting to spin up a small instance for hyper local maps.


If you do spin one up, feel free to join the matrix and share the experience or just email me with feedback. I'm trying to get this as frictionless as possible to set up because I want people to own their own location data.

Headway is much much more responsive on a local scale than it is in https://maps.earth/ and the search results are more relevant too. My own personal instance for Seattle has largely replaced Google Maps for me. :)


Thanks for this. I'm curious to know: why did you include two routing engines (OTP and valhalla)? Wouldn't e.g. OTP sufficient and use much less resources compared to 2 setups?

And why not reuse one of the existing open source map or directions front-ends?


OTP is a resource hog (for good reason, it solves a harder problem than valhalla). Covering a US state is very much pushing it for OTP, and covering the planet wouldn't work at all on any hardware.

I didn't use OTP's frontend because I found it unpleasant to use. Valhalla's frontend wouldn't work well with OTP and I don't find it terribly pleasant to use either.

Neither work on mobile, and neither work with vector maps, it seems? I haven't tried out GraphHopper's UI though.


Looks like they use either mapbox gl is or maplibre gl js to render the map


I meant there are many ready-to-use map UIs out there already, like the UI from OTP or valhalla itself. No need to reinvent the wheel.

E.g. see https://valhalla.openstreetmap.de/

Or https://github.com/Qwant/erdapfel

Or https://graphhopper.com/maps2 & https://github.com/graphhopper/graphhopper-maps (disclaimer: I'm a contributor there)


A few things that I notice in the area where I'm somewhat active as an OSM contributor:

- You probably shouldn't render guideposts or route markers on the map, or at least not with generic "info" icon.

- Things with access=private are likely not terribly relevant to people looking at the map. This will include things like parking spots or gardens.

- Lots of shots and amenities have icons, but doctors don't. Neither do fire stations.

- Sports pitches are shown as nodes, but may be better shown as areas, since they often cover quite extensive swaths of ground that's otherwise empty with just an icon in the middle. Same with playgrounds, schools, and kindergartens, gardens, and parks.

- For buildings and building parts, if you don't make use of the parts in any way (such as showing roofs, height, etc.) perhaps just disregard the parts and only render the building outlines. This, combined with the fact that geometries seem to be simplified, even at the highest zoom levels, results in a bit of a mess of lines at times where some outlines are simplified differently from others that should actually share parts of the same edges and thus overlap perfectly. (e.g. https://maps.earth/place/openstreetmap%3Avenue%3Away%2F11120... or https://maps.earth/place/openstreetmap%3Avenue%3Away%2F11120...). Same thing with the geometry simplification also happens with adjacent geometry that should actually not have gaps (e.g. semidetached houses or garages next to houses). Lots of buildings that are actually rectangular also get quite weird angles.

- Tunnels are quite subtle. And while at least roads in tunnels don't overlap buildings that are actually above them (like with Carto) it's still hard to make out what's above and what's below in such cases (e.g. https://maps.earth/place/openstreetmap%3Astreet%3Apolyline%3...). Culverts for waterways are handled better than roads in tunnels in this case.

Overall though, this seems like a quite nice map style. Closer to Google Maps in that it emphasizes roads, but some people like the less detailed look (and it works better with vector maps, I guess, to have fewer polygons to render).


I'm using OSM Liberty [1] so as far as I know I should be just tracking upstream with most of these issues, but I think some of them might actually be introduced by planetiler [2]. Thanks for giving it such a detailed look though.

[1] https://github.com/maputnik/osm-liberty

[2] https://github.com/onthegomap/planetiler/issues/324


Ah, [2] indeed looks like the same issue. I'll try to raise issues on osm-liberty for the other things. Not that they don't make sense to display in certain topic-specific maps, but for general-use maps I think there are some parts that should be decluttered.


Works surprisingly well! One note for walking directions: It doesn't seem to take elevation gain into account in choosing between alternate routes. Probably a not an easy thing to implement.


What language is it in?

In Italy, most of the cities have the Italian name, some of them have an English name (Florence, Naples), some have a German name (Triest). The behaviour is different than OSM, at least for me.


- Triest is mapped with several names (https://www.openstreetmap.org/relation/179180), ‘Trieste’ being the standard one.

- Naples is mapped with many names (https://www.openstreetmap.org/relation/40767), ‘Napoli’ being the standard one.

- Florence (https://www.openstreetmap.org/relation/42602) has ‘Firenze’ as standard name.

A renderer can choose which name to put as label on the map, usually it’s `name` or `name:locallanguage-abbreviated`.


Is there any support for non-earth maps ? Such as for a DnD campaign or a story set in Middle Earth ? Or even a war game on Earth but with different place names and countries borders names, etc ?


It is all based on OSM data. And I haven't checked in a few months but I believe Maplibre, which is doing the rendering, require the tiles all be in Web Mercator, which is an earth based coordinate system. So you would probably need to hack together something to change the underlying data into what you want. The easiest one would be changing place names and borders on Earth.


What


Awesome work!

What are your thoughts on the feasibility/benefits of using a p2p system like IPFS to share OSM map/tile data between users, rather than serving from one centralized server?


It's... complicated. The discussion for it is here, I'd be curious if you had any thoughts in particular about this. Feel free to comment or join the headway Matrix room if you want to chat more in-depth. In principle I'm open to this but the privacy implications are a little weird. https://github.com/headwaymaps/headway/issues/50


Wow, it is fast! I could see it had my test-route drawn and ready before the scrolling animation to the start had even finished.


Is the data shared between vector tiles, geocoding and directions? Or is each feature pulling its data from a different index?


During data preprocessing it all comes from the same OSM extract, except for Pelias which also pulls data in from Who's On First, and Planetiler which also pulls data in from Natural Earth (and a few other sources).

At runtime tiles are served over the wire but everything else happens server-side. So it's a bit of everything :D


I don't think they share the index as all three parts are different projects.


How much resources does something like this use? (full planet, few (<10) users)? Disk, RAM, cpu cores?


I cover that in the readme, but the short answer is that the server running maps.earth has 64GB of RAM and 6 physical cores. It's using about 600GB of disk space, but the disk usage might change because I suspect my Pelias index is missing a lot of information.

https://github.com/headwaymaps/headway#system-requirements


Love the idea of freeing up map data


I have found an error in a street name. Where could I report it?


On https://www.openstreetmap.org

See the tutorial there or https://learnosm.org/en/beginner/start-osm/


This looks so clean.


I like the fact it uses Earthly for building. Nice pun.


I just asked for directions from my home to the railway station and the navigator told me to enter a roundabout in the opposite direction :/


The direction of that roundabout might be mis-tagged on OpenStreetMap. Would you be comfortable sending me the location, so I can see about fixing it?


Forget that, I erroneously asked for walking directions my bad. Shame I can't delete the comment lol


Is it a web app? A web service?


You can use it as a web service by visiting https://maps.earth/


Can you tell us what this even is before telling us how sorry you are that it’s not perfect?


It's a self-hostable web maps stack trying to be an open-source alternative to Google Maps.


Didn't have my address in NJ.


You should be able to add it at https://openstreetmap.org :-)


But don't expect it to show up on maps.earth right away... it looks like it's at least a month behind OSM at this point.


That's mostly because I decided to use maps from the daylight maps distribution. The current tiles were actually built last night. I think it probably makes sense to track planet.osm.pbf directly though because I want to create a virtuous cycle where users feel that contributing to OSM has a real tangible payoff.

The main issue with updating regularly is going to be that long-term I can't really afford a beefy enough machine that has a fast uplink to Cloudflare where my artifacts are hosted. Getting all of the planet-sized artifacts uploaded even just this one time was a huge pain. I ended up having to generate them on my desktop, ferry the data to a friend's house on a laptop via motorcycle and upload them from there. This is one of the things that I want to help remedy if I get enough support via Liberapay or GH Sponsors.


It doesn’t pull the OSM data (from their servers)? Or you did that once and pre-processed everything now?

I presume you can do something similar for the daily and minutely updates, but I imagine merging that with the existing stuff is more work.


Have you explored whether it would be possible to upload only the changed parts?


[deleted]


A usable example would make this much more effective brochureware.


Like https://maps.earth ?


Huh - rendered a totally different page on my desktop




Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: