The backend at this point is just Graphhopper so I think it would make more sense to use it directly and customize it based on your needs: https://github.com/graphhopper/graphhopper
Thank you! I didn’t add a way for people from other regions to sign up, like a newsletter since I figured most people dislike it here on HN. You can check back on this thread in 2–3 weeks, and there should be an update comment from me announcing a new version of the planner with the US region included.
Oh wow, nice! I wasn't aware cycle.travel had this. I thought I had done something new but I see I was mistaken. By the way, good job on cycle.travel - the route finding is super smooth!
You can change the profile to 'road cycling' and it will avoid paths with bad surfaces.
Not sure about the UK but I've cycled quite a lot around here in Europe (12 countries so far) and from my experience, if you don't follow the official bike trails you're missing a lot. They are usually nice scenic and quiet roads with good (asphalt) surfaces. I'm talking about trails like the Alpe Adria Cycle path. There are tons of them in mainland Europe.
But you're right - if the surface is bad, then it doesn't make sense to follow the trail. I will add a feature that will highlight the selected surface on the line (similar to what Komoot already does) so you can better investigate the route and make sure the surface is alright.
I'm also planning to add a feature where users could rate the track so you could filter out the bad tracks based on other people's reviews.
Avoiding the official bike trails in Europe has advantages. For example, when you plot your own routes away from the touristic ones, local people might be more willing to chat or offer you a place to stay for the night because they have never met a foreign bicycle traveler before. Also, bikepacking is big these days, so a lot of cyclists want routes that don't have good asphalt surfaces.
Thank you, appreciate it!
Travelmap looks awesome—I love the idea. I will definitely try it out on my next trip.
This morning, I picked up an affordable m920q with 64GB of RAM and I’m planning to use it for hosting GH and maybe vector tiles too. If I run into any issues I can’t figure out quickly I’ll reach out but I don’t want to bug you with basic questions.
I have a stack of about 10 older dual xeon based supermicro 1u machines with 512gb of ram....if you are in the oregon area you can have any number of them for free! infini-ram makes life easier when dealing with OSM data.
Oh man, that's sweet! Thank you. I would be at your door by now but I'm located 8000 km away from Oregon :D I just received a tiny M920Q, and 64 GB of RAM should arrive tomorrow. I'll have to make do with it for now :D
Nowadays the cheapest way to self-host planet vector tiles is probably to use protomaps.com and throw the .pmtiles file on a S3 bucket (AWS, Cloudflare, etc.).
So far, I've only generated the graph for Europe(using Graphhopper). Even just Europe required 128GB of RAM and around 10 hours of computation time (the entire planet would likely need 384GB of RAM). I plan to add North America on a separate Docker container soon though. I started with Europe because I’m familiar with some of the bike trails here, which makes it easier for me to check if the routing makes sense.
> the entire planet would likely need 384GB of RAM
Unlikely. Even with turn costs enabled 256GB (or less) are sufficient. You could also try to disable CH as for bike often no long routes are required (you could disable them). Here we have written down a few more details: https://www.graphhopper.com/blog/2022/06/27/host-your-own-wo...
Hey karussell, I really appreciate all the hard work you’ve put into Graphhopper. I wouldn't be able to create this project without GH. I have a question about memory usage during the import stage (specifically in the OSM Reader's preprocessRelations function). I'm using a HashMap<Long, List<Long>> to map way IDs to OSM bike route relation IDs, which means allocating lots of arrays. Could this be causing me to run out of heap memory faster or am I off base here?
I thought I would be able to compute the graph with 64GB of ram but it kept crushing before CH and LM stage. After switching to a 128GB instance, it finally worked, hitting around 90GB at peak memory usage. For context, I was using 3 profiles - one with CH and two with LM, plus elevation data and used all of the tips from deploy.md
Maybe you already considered, but there are a number of collection libraries out there that are optimized for holding Java primitives and/or for very large sets of data, which could help you save significant memory. Eclipse Collections [0] and Fastutil [1] come to mind first, but there are many out there [2]
Thank you! I'm a total Java noob - actually, this is the first project where I've written any Java code (had to slightly modify the Graphhopper source code to suit my needs). Those libraries look very interesting. I'm saving this post for another battle with processing maany GBs of OSM data :D
We already use carrotsearch internally so you could replace the java util classes like HashMap and HashList with it to reduce memory usage a bit. But it won't help much. E.g. all data structures (in any standard library btw) do usually double their size at some point when their size increases and then copy from the old internal array to the new internal array, which means that you need roughly 3x the current size and if that happens roughly at the end of the import process you have a problem. For that reason we developed DataAccess (inmemory or MMAP possible) which is basically a large List but 1. increases only segment by segment and 2. allows more than 2 billion items (signed int).
Another trick for planet size data structure could be to use a List instead of the Map and the OSM ID as index. Because the memory overhead of a Map compared to a List is huge (and you could use DataAccess) and the OSM IDs for planet are nearly adjacent or at least have not that many gaps (as those gaps are refilled I think).
All these tricks (there are more!) are rather tricky&low level but necessary for memory efficiency. A simpler way for your use case could be to just use a database for that, like MapDB or sqlite. But this might be (a lot) slower compared to in-memory stuff.
Oh wow, that's amazing - thank you! I really appreciate your offer and I love RWGPS. I just read your comment about about all the hardware you're using at RWGPS and it's clear you're operating on a completely different scale than I am :D Right now, I'm just running Graphhopper on a €40/month Hetzner server and free tiers on Vercel and Maptiler. Though I already hit the free-tier limits for Maptiler so I'm quickly setting up a Martin server to server Europe mbtiles on some cheap server instead. Once the traffic drops after this HN post I think a low-cost server should do the trick for now.
As for Graphhopper, I ran into some challenges during the Europe-wide import stage. It turns out 64GB of RAM wasn’t enough, so I ended up spinning up a 128GB instance on AWS. After tweaking some config settings and following the deployment guide, I finally got it working. I also had to change the source code a bit to link each "official bike route" edge to its corresponding OSM relation info but I managed to get it working in the end(using KVStorage and KValues).
For now, my planner is pretty basic, so I don’t want to bombard you with beginner-level questions. But if something more complex comes up down the road, I’ll definitely take you up on your offer to reach out. Thanks again—I really appreciate your help and generosity!
Thanks man! Really glad to hear that. Actually, I had bikepacking and bike touring in mind while working on it as I love doing long cross-country bike trips every summer myself.
I definitely want to get the mobile version up and running soon but it’s a lot of work to set everything up for this project especially since Im working solo. For a fully functional route planner, it would probably make more sense to build a React Native app instead of just a website.