
Ask HN: How hard is it to build a platform like Netlify? - pyzhianov
I&#x27;m fascinated by Netlify, by its speed and neat developer experience. I want to learn more about the inner workings of such platforms and thought that the best way to do it is to build something like Netlify myself. Nothing stellar, just a simple proof of concept clone.
Any suggestions on where to start and where to learn about the architecture of platforms like Netlify? I&#x27;m a moderately experienced frontend developer, I have some backend and DevOps skills, but it&#x27;s mostly &quot;street knowledge,&quot; and I don&#x27;t have a CS degree.
======
symkat
If I were to do this, I think I would start somewhere along the lines of:

1 - $5 linode/digital ocean machines at different locations (CA, TX, NJ,
London, wherever you want the things)

These machines would run probably Nginx serving static up static content.
They'd get this data pushed to them over something like rsync/scp from a
central place.

It would answer for the same domains, with the same content at each location.

2 - DNS Route53 / GeoDNS / Round-Robin

Servers in all those locations are cool, but we want to get users close to
them. I think Route53 does GeoIP DNS - routing users to the geographically
closer machine.

3 - $10/$20 Linode at a central location for build/deploy.

I'd probably look at Drone.io as the thing to take gitrepos and then build the
website. This could handle JAM stack stuff pretty easily. Once it has the
artifacts created, you ship them off to the linodes in step one to be served.

This is kind of the broad view of where I would start for technologies to help
with that kind of thing. Then comes a lot of glue and programs to make it
easier to use.

The form submission stuff they made seems pretty cool. You could make
something that scans the artifacts looking for forms with a given data
attribute, and when you find them build a data model to match and stuff in a
POST address. Making that would be really fun.

I hope this helps, I also dig this kind of stuff =)

------
abetlen
I guess it depends on how far down the rabbit hole you want to go and what you
want to learn?

For example, we recently built something to the same effect to deploy our
single page applications to AWS. It creates a new Cloudformation Stacks for
each site, sets up CodePipeline for continous deployment, build artifacts are
placed in a public S3 bucket, and then it's served through CloudFront as the
CDN. Beyond that you just need to set up Route53 for your domain and use the
Certificate Manger to handle SSL.

I know that's pretty specific to AWS but hopefully it gives you an idea of
where you'd start.

Basically to build it from scratch you'd need: \- A service that can respond
to GitHub webhooks, build projects in an isolated environment based on a
config, and push build artifacts out.

\- A content distribution network that you can deploy those artifacts to.

\- A DNS server to route traffic.

\- An SSL service like Let's Encrypt CertBot to issue certificates
periodically.

------
tasssko
Please do it. It is reasonably simple to build something like that, I've done
it many times for over a decade now - its so much fun and truly rewarding.
From the beginning you need to focus on the developer experience that means
you need to make sure your execution of all the components is done in a way
that allows a developer to develop and not have to click buttons or fill in
any forms.

To me the interface is the IDE and important changes in source code happen
when I merge branches to master. From there the ingredients are a well
developed cicd pipeline plus some integrated infrastructure as code (i.e
cloudformation, teraform, ansible). The infrastructure as code can get a
little complex and you need to know the the underlying components quite well
or it might cost you more than you think. You could achieve this with a vps
too if you had one. One of the reasons I use Jekyll is I can host in on s3
which allows us to focus our operational capabilities on our main business,
Servana does devops and cloud services.

I use Jenkins with a declarative pipeline plus cloudformation which configures
all the infra like s3 bucket and cloudfront distribution etc. I kept it simple
but still pro with the main goal that a developer does not have to log into
anything to trigger updates. The pipelines creates a release and tags master
merges.

When I build these workflows the developer experience is very important so I
always make the process completely automated with no interrupts if it can be
helped. My focus is on one thing, the code and the development experience.

------
sneak
In my view, it‘a a rather lot of work. The hard parts are probably less
visible: combatting abuse of the free build system, sysadmin/scaling stuff, et
c. Getting build scripts running and static pages on a webserver, their main
function, is relatively straightforward.

If you want to build one just for yourself, that is probably a weekend
project. Doing multitenant safely is likely where it gets a lot more complex.

------
jamil7
I built something similar with the same basic functionality although lacking
all the polish. Basically I had a cheap vps with docker installed and a small
control panel where a user could log into github and pick a repo and add
domain names. There was then a number of glue scripts to pull and build the
sites and deploy them as Caddy server containers, this handled all the
automatic TLS and static file serving. I had all these load balanced with
nginx listening to the docker deamon and restarting. In hindsight the whole
thing could have been a good opportunity to learn and use k8s, maybe would
have been simpler to piece together but I learned a lot.

------
amingilani
It depends on how much of it you want to recreate. Like, do you want to create
the servers, or are you happy letting a CDN do the heavy lifting? If it's the
latter, you can deploy to GitHub pages, and then use CloudFlare's CDN. I don't
think I need to post it here since it's HN, but I documented this technique in
a comprehensive article once[0].

On the other end, though, you can buy small storage units and put Raspberry Pi
4s all around the world, and to mimic the CDN-ness of it. You'd need a central
build server, which doesn't matter where it physically is, but it'd push JAM
builds to your CDN network, and then a DNS server that routes requests to the
right CDN node depending on the Geolookup of the IP address.

[0]: [https://www.toptal.com/github/unlimited-scale-web-hosting-
gi...](https://www.toptal.com/github/unlimited-scale-web-hosting-github-pages-
cloudflare#contract-just-respected-software-architects)

------
sansnomme
Have fun :) [https://staticdeploy.io/](https://staticdeploy.io/)

------
o-__-o
I use an execution system to run scripts on remote hosts. I create dns via api
calls, configure a web server to accept that domain, which points to a parked
landing page. When ready I run another script that pulls the latest tag from
git and drops it onto the web server . These pages are served until I make a
change to git and push deploy. I would like to make a ui to this but my JS
skills suck donkey butt.

------
svedova
Basically I did exactly this :) I created a tool that helps you deploy your
static (and hybrid!) app, with a few more features like creating environments,
specifying remote configuration and staged rollouts. It took me a year part
time, all by myself. Check it out here:
[https://www.stormkit.io](https://www.stormkit.io). Basically, you gotta have
a nice understanding of the cloud providers, some network knowledge, a lot of
backend knowledge and a descent frontend knowledge if you want to compete with
their awesome UX :) - Obviously, Netlify has a team behind it, but despite
that I believe I did a good job.

------
herohamp
Recently I've discovered commonshost and it seems like a great way to host a
content. It's a free global CDN with < 30ms for most of the regions I've
tested. If you want to startup a node in your region it only costs around 200$
for the equipment to run a node

------
z0mbie42
I can't help but I would also love to learn more about it's inner working!

Especially the certificate fetching and distribution across the edge servers
part.

------
sbolt
I recently used Netlify to deploy a Gatsby site, the experience of deployment
is so easy: just link a GitHub repo or drop in the project directory. The
Netlify community forums are really helpful too. If you’ve any questions
please ask!

~~~
quickthrower2
Wow savage downvoting on that. I wonder why? Was the comment edited?

~~~
sbolt
I’m surprised that a pretty vanilla comment got so much hate, it’s hard to
please everyone!

~~~
detaro
It has absolutely nothing to do with the question, why wouldn't it be
downvoted?

~~~
quickthrower2
The shade of grey was unreadable that implies a lot of downvotes. This is
usually reserved for pure trolling style comments.

