Ask HN: If I wanted to make a web browser, where would I start? - avindroth
======
jbb555
I'd think about it like this :-

1) Write socket code to retrieve the page from a server

2) Get it to draw the text in a window.

3) Write code to look for tags and start diving up the retrieved file into a
proper DOM tree. Perhaps start by creating an object for each paragraph. Then
maybe look for <b> tags, then <div> tags

4) Change the rendering code to draw the screen for the data structure you
created above.

5) Iterate a lot. Add support for parsing more and more tags into the data
model. Start adding annotations for various attributes.

6) Start improving the layout code. Draw things in <b> tags in bold. Draw
things in <div> tags under each other and <span> tags separately.

7) Add limited CSS support. Allow just width and height and border sections.
make the code read the css file and attach the attributes to the correct
sections of the DOM document you are creating.

8) Improve the layout code to look at the width and height tags on each
element, and where tags are nested to propogate the width and height
information up and down the tree as needed. Draw borders if the css tells you
to. Look at font weight etc.

9) Iterate. Add one feature at a time. Repeat for a year until you have an
browser that can render basic pages.

10) Iterate some more. Read specs. Rewrite your document to DOM tree parse a
number of times perhaps using a more formal grammar. You'll probably be on
about year 8 by now.

11) Add javascript support... :P

Perhaps my point is though that this seems something where its very easy to
start small, just render the text of a page, and incrementally add features
and improve the layout. It will take a long time though as there are so many
features to add.

~~~
d33
Is it still even doable for a single person to write a new rendering engine
from scratch? I'm getting the impression that right now it's all about Webkit,
Gecko and IE and given how complex things got, I imagine that it would cost an
enormous sum of money to write something that, say, can display top 10 Alexa
web pages. Things just got too complex.

Perhaps it would be better to have some effort in documenting Webkit code so
that more hackers could actually read it and hack on it?

~~~
zeta0134
It shouldn't be too much for a single person to take on if they're truly
dedicated to the task. A small team is still probably more ideal, but the
browser isn't overly complex, it simply has a lot of capability that it can
pull off with its feature set. All of a browsers moving parts are designed to
be quite generic, which allows quite complex interactions to arise out of a
small handful of simpler components.

Things used to be a lot harder 5 years ago, but the renewed focus on standards
means that many smaller browsers can focus on just that, and ignore a lot of
the weird quirks found in major browsers safely. (The death of Internet
Explorer is finally bringing about the necessary changes in websites that have
clung to its quirks mode so blindly for so many years.)

I think starting with a manually written parser is a necessary first step if
you're writing a web browser. It's tempting to want to pull in Webkit or V8 or
Gecko or Chakra, but if you're writing a web browser, you need to understand
what these tools are actually doing for you before you use them. Writing your
own versions, even if that's time consuming, will teach you why these other
frameworks have become so popular, and educate you about their shortcomings so
you can hopefully code around those too. If you do it write, and your codebase
is appropriately modular, it should be straightforward to modify the project
down the road to use a different parser, or even an entirely different DOM
tree implementation.

------
wslh
You have mainly two options: do everything from scratch or reuse. Starting
from scratch is almost impossible to do alone, just Chrome was developed by
different teams with different expertises, some of them acquired from other
companies.

The second option is reusing a web component or browser and modify it for your
purposes. Building on top of Chrome or Firefox (or their variants) is a good
choice. I think Chrome is better engineered than Firefox and the code is
clear. Obviously this is just an opinion.

Now the question is what do you want to add to such browser? Is this something
you can add via an extension or is something that requires a new direction?

~~~
sglane
>Chrome was developed by different teams with different expertises, some of
them acquired from other companies.

To expand on this, Chrome and Safari use webkit as their rendering/layout
engine which is a fork of KDE's KHTML. IE came from NCSA Mosaic-> Spyglass ->
Microsoft IE.

To be clear on my suggestions below, let's clarify what a browser actually is.
The browser manages things like the navigation bar, bookmarks, download/mime
handling, etc. This is known as the "chrome" or GUI. The layout engine handles
HTML parsing, rendering, possibly is coupled with a Javascript engine like V8.

If you are trying to write a browser, the answer is to use webkit2 in whatever
language you like. It's fast, light, and portable. If you are trying to write
a layout engine (rendering engine), do as another poster suggested and write
one to target HTML 3.2.

~~~
timvdalen
Note that Chrome is actually using Blink now, which itself is a fork of webkit

------
ruraljuror
This series of blog posts might be a good place to start:
[https://limpet.net/mbrubeck/2014/08/08/toy-layout-
engine-1.h...](https://limpet.net/mbrubeck/2014/08/08/toy-layout-
engine-1.html)

~~~
magic_beans
This series was really well-written and fun to work through! I second this
recommendation.

------
pmontra
As somebody already wrote "Starting from scratch is almost impossible to do
alone". However you could set a first goal of doing some basic rendering of
plain HTML, even text based. That should make you familiar with the hurdles of
a rendering engine. Then apply some CSS. Even writing a browser for HTML 3.2
from scratch and no JavaScript should be an exercise to be proud of
[https://www.w3.org/TR/REC-html32](https://www.w3.org/TR/REC-html32) (1997).
Then start looking at modern browsers using the links others provided. You
should have the experience to understand all the moving parts by then.

This is similar to writing the very first web application without using any
framework, maybe even interfacing directly with the web server. I remember
decoding CGI arguments in C in 1994 before discovering Perl's CGI.pm. Then you
understand the magic inside frameworks and don't get burned by surprises.

------
retro64
I did this once back in 2003. It was text only rendered, with mouse support so
you could click on the links instead of tabbing around like Links.

I built it off of the IE object model (Internet Explorer Browser Extensions?
Can’t recall). I think I just used the object model for connectivity settings.
If you could get to the net using IE, my browser would work as well.

I think I coded to HTML 1.1 specs, and had it working after 2 months (with a
few minor rendering issues). It did not support scripting, CSS or really much
of anything (although it could POST), but it was surprising how much of the
web was accessible. And it was really fast. Not because of my skillz, but when
you cut out the garbage, it’s amazing how fast a page will load.

Anyway, I would not recommend this approach, but I would offer you
encouragement. HTML parsing isn’t really that big of a deal, and you could get
something very minimal (text rendering) working in short order. It would at
least give you a feeling of what you are up against.

------
knabacks
My 2 Cents:

Start out with a solid base: (for example)

\- [https://cefbuilds.com/](https://cefbuilds.com/)

\- [https://crosswalk-project.org/](https://crosswalk-project.org/)

\- [http://electron.atom.io/](http://electron.atom.io/)

\- [http://nwjs.io/](http://nwjs.io/)

or you try to help with existing projects: (for example)

\- [https://github.com/breach/thrust](https://github.com/breach/thrust)
(started out with
[https://github.com/breach/breach_core](https://github.com/breach/breach_core)
a "javascript"-Browser (core is not js))

[https://github.com/minbrowser/min](https://github.com/minbrowser/min)

~~~
michaelmior
Breach seems like a poor choice since the project has been inactive for ~2
years.

~~~
knabacks
Maybe you're right, a project like servo
([https://github.com/servo/servo](https://github.com/servo/servo)) looks like
a better fit.

easy bugs to hunt:
[https://github.com/servo/servo/labels/E-easy](https://github.com/servo/servo/labels/E-easy)

------
max_
Browser Engine [https://limpet.net/mbrubeck/2014/08/08/toy-layout-
engine-1.h...](https://limpet.net/mbrubeck/2014/08/08/toy-layout-
engine-1.html)

How to Create your Own Internet Browser in 5 Minutes! -
[https://www.youtube.com/watch?v=Bn22JBRC1UI](https://www.youtube.com/watch?v=Bn22JBRC1UI)

Chrome's Engine [http://www.chromium.org/blink](http://www.chromium.org/blink)

------
stellar678
"If you wish to make a web browser from scratch, you must first invent the
universe."

------
marai2
There is a "Building a Web Browser" course on udacity[1]. Maybe someone who
has taken the course comment on it.

[1] [https://www.udacity.com/course/programming-languages--
cs262](https://www.udacity.com/course/programming-languages--cs262)

------
p333347
If you are seeking tutorials, or UI libraries etc, sorry, I can't help you
with that as I am unaware of those things. However, if you are asking from
which aspect to start building one from scratch, I would say start with
layouting. Try implementing a tiny subset of div related things like position
and display and see if this is what you want to spend your time working on.
Even if you can crack this thing easily, remember, this is just the tip of the
iceberg. I am not discouraging you, but know that you would have to digest the
entire DOM and HTML spec, not to mention have a JavaScript engine and
implement all the security related intricacies, at the least, for your
software to become a web browser.

~~~
anatoly
I'm curious: is there a ready-to-use set of tests for a layout engine that
checks conformance with the specs? Something that you could, without much
effort, run against a toy layout engine you'd write?

I know Webkit/Blink have humongous test suites, but these probably aren't easy
to adapt to a toy layout engine.

~~~
chiefofgxbxl
There's something like Acid3, but it doesn't give you information as to what
failed when you get a less-than-perfect score. (Test:
[http://acid3.acidtests.org/](http://acid3.acidtests.org/) | Information:
[http://www.webstandards.org/action/acid3/](http://www.webstandards.org/action/acid3/))

There are also all those other browser benchmarks (e.g.
[http://peacekeeper.futuremark.com/](http://peacekeeper.futuremark.com/)), but
those include things other than the layout engine, such as javascript
performance, webGL, etc. You could try poking around a few of them and only
looking at the CSS and DOM sections though
([http://web.basemark.com/](http://web.basemark.com/)).

------
chvid
Back when I did CS 101 this was our final project; the teacher made the HTML-
parser, and we students did the renderer and UI.

The HTML-parser is probably the part it makes sense to do first.

What language and platform do you plan to use? I think a modern OO-language
will be helpful.

------
_RPM
Just a note: The browser as we use it today is a result of 20+ years of human
work and innovation. It's what you would call a killer application.

~~~
iamben
Equally - the car we drive today is a result of 130 years of human work and
innovation. But after all that time we're seeing a real shake with the
electric car. Things change when people try. I say good luck to the girl/guy -
perhaps they'll create something incredible :-)

~~~
richardwhiuk
Even the electric car didn't start from scratch - it'll use the same component
manufacturers as the rest of the car industry - e.g. tyres, brakes,
suspension, interior, safety systems.

~~~
reitoei
You and parent poster both make good points.

------
txutxu
Well, this is what I could do in that (or almost any) case:

1) Setting features, key points, points to avoid, and goals of the project.

2) Choosing the toolchain to use, for resolving the points at 1)

3) Coding the features with that toolchain

Step 2, may involve: to read a lot (doc, rfc's, specifications, using search
engines with keywords, search for "problems", etc) to look at other people's
source code of similar programs, to try other people's tools and libraries and
evaluate and compare them, to make integration tests, and to make Proofs Of
Concept prototypes.

The toolchain (programing language, user GUI, etc) comes with an ecosystem (I
hope you don't aim at rewriting also your $lang libraries for validate XML or
interact with devices), so you have to choose looking at both.

To separate steps 2 and 3, use iterations, while you have resources.

If you find an opensource product that full-fills all your definitions of
point 1), except a few of them, you can evaluate to start by a patch and a
pull-request to them, or a fork :P

------
rer
If you wanted to choose how to build the UI, one starting point is here
[https://news.ycombinator.com/item?id=12376242](https://news.ycombinator.com/item?id=12376242)

------
WayneBro
Do you want to build a web browser or a web browser engine?

If you just want to build a web browser, here's my request for a new browser
for iOS since web browsing on iOS is a second or third rate experience:

\- Built-in ad-blocking.

\- A way to disable JavaScript immediately for any website.

\- A desktop mode that actually works, so that YouTube, Reddit, Imgur and
other sites don't redirect me to their crappy mobile site without asking.

\- Launches a completely separate app for private browsing.

Right now I use Dolphin browser on iOS and despite being in desktop mode, I
still have to refresh every YouTube page that I land on in order to get the
desktop style native HTML5 video player that allows me to go full-screen. On
Imgur, gifv files don't play - I have to change the extension to gif and then
wait 60 seconds as it downloads. Lots of other sites have similar types of
issues. Meanwhile, gifv files work fine in Brendan Eich's "Brave" browser. It
could be as easy to fix as just changing the user-agent.

Dolphin has good ad-blocking, but there is no way to quickly turn off
JavaScript for a given page or domain (ala Quick JavaScript Switcher for
Chrome).

The thing that really annoys me though about Dolphin is that Private Browsing
does not work at all. That's probably because they want you to buy Dolphin
Zero for $2.99.

~~~
speg
I checked out Brave yesterday for the first time and I think it does most of
those things.

~~~
WayneBro
Brave has a desktop mode that mostly works and a built-in adblocker.

It does not have a `quick JS disable` button.

It also doesn't have a speed dial that I can manually add items to, which is
one very important item that I forgot to mention earlier.

What I really want to do is make an open source browser that I can compile and
put on my iOS devices myself too, without going to the app store. Then I could
add all sorts of features and extensions without having to ask Apple.

------
hugozap
I'd say, if you have the opportunity and want to go wild skip the DOM and
create an equivalent model using OpenGL. All the other Web platform features
like the distribution model, resource streaming etc can still be used. It's
just that the DOM is the great bottleneck and most of the hours devoted to
improve the Web are being wasted on sorting quirks related to the DOM. Maybe
yours will be the starting point towards a DOM-free Web! :)

------
mixedCase
Well, what are your goals?

------
IncRnd
You can accept entered URLs and fetch & display web pages quite easily
nowadays. You should, depending on the vertical, be very concerned about
security.

What are your goals (to a greater degree than "web browswer")?

------
bigato
1\. Write a webassembly interpreter/vm; (main step)

2\. Using webassembly, write an html interpreter/renderer;

3\. Using webassembly, write an css, js ... interpreter/renderer;

4\. Keep doing this for all w3c standards for the rest of your life.

Chances are that by the time you are somewhere in the middle of step 2, people
will have written the other interpreters in webassembly and the result of step
1 will be a functional browser already.

------
beachstartup
[https://curl.haxx.se/dev/source.html](https://curl.haxx.se/dev/source.html)

------
OldSchoolJohnny
Step one: Build AI or machine learning or genetic code to build a browser by
replicating the results of a real world browser.

Step two: run it!

------
matthewhall
I'm working on one right now. I'm using the electron shell because im not
worried about the backend source (chromium) I'm just working on a new ui and a
reader feature. Email at matthew349hall@hotmail.com if you want to get a beta

~~~
philippnagel
I sent you an email.

------
c-smile
Well, I did exactly that.

Sciter ( [http://sciter.com](http://sciter.com) ) is an embeddable
HTML/CSS/script engine that has all components of a browser - html, css, svg,
script and networking (http,ajax,websockets,etc).

Work started in 2002 when I decided to create WYSIWYG HTML editor for my wife.
So [http://blocknote.net/](http://blocknote.net/) was born. That was pure HTML
(no CSS) rendering and editing engine. 9 months for one developer - myself.

After its publishing joined initial EverNote team to create WYSIWYG editor for
EverNote. Three of us (developers), + Stepan Pachikov that generated the idea
of the EverNote, created first published version of the EverNote (v 2.0). That
was also done in 9 months (I am pretty sure that 9 months is not a coincidence
but rather a law for new projects - optimal period when developers are excited
of what they are doing).

While working on it, and being a UI-in-all-its-incarnations
developer/architect, I've discovered that HTML and CSS are quite convenient
format for defining UI layouts and elements. So next step was wrapping it as a
reusable component and so HTMLayout engine was born (
[http://htmlayout.com](http://htmlayout.com) ).

Almost immediately discovered that plain C++ is not quite convenient for
handling complex UI cases - "on click here hide that and expand those". UI
objects ownership graph is quite complex and contains loops. Something with GC
is definitely required. So decided to add scripting to it. Considered quite
many existing languages including JS for the "language behind UI". At the end
decided to roll my own tiscript (
[http://www.codeproject.com/Articles/33662/TIScript-
Language-...](http://www.codeproject.com/Articles/33662/TIScript-Language-A-
Gentle-Extension-of-JavaScript) ) that essentially is JavaScript++. So 7 years
ago I had what we will have in ES6 in next couple of years.

At the mean time I participated in HTML5 spec. development at W3C as an
invited expert. And worked as an architect / team lead in several Web
Application projects that required deep understanding of the platform -
knowing Web UI stack from inside-out helped that a lot.

As Sciter works on all major desktop and mobile platforms I've created core
engine wrappers for Windows API, MacOS CoreFoundation, Linux/XWindows/GTK and
some others. Sciter core is written in C++ with Objective C fragments in
Apple's wrappers. GFX technologies: GDI, GDI+, AGG, Direct2D, CoreGraphics,
GTK, OpenGL, DirectX and others.

Script in Sciter is not quite a JS so I've created ports of jQuery (zepto
feature set) and AngularJS alike data-binding mechanism based on
Object.observe() that Sciter has.

More of Sciter's history: [http://sciter.com/10-years-road-to-
sciter/](http://sciter.com/10-years-road-to-sciter/)

At the moment Sciter code works on around 350 mln PCs and Mac's worldwide as
part of various products.

Advice: all this is really possible if you have _real_ motivation to do all
this. Estimate that first. Split the task on 6-9 months periods. Start from
creating basic DOM and CSS rendering. You can do that without parsers
initially. Parser are boring - postpone them for the moment when you will have
something breathing. Just a small trick for self-motivation.

Other than that it is doable, yes. For the simple reason: as smaller dev team
as higher its productivity. Solo developer has highest productivity possible.
Some conditions apply of course.

------
pif
By giving up? ;-)

------
vmorgulis
Instrument an existing browser and synthetize a new layout engine with machine
learning and superoptimisation.

------
VLM
Sounds like a pretty good interview question. Better than asking you to invert
binary trees or similar WTF. They're testing your judgment and asking you to
assess your own skills and possibly they're interested in the actual
assessment results assuming you're any good at assessment.

Most people would generally break the task chronologically or kinda data flow
oriented. Probably. Now should you start at the part you're familiar with or
less familiar or unfamiliar with? My philosophy is its a bathtub shaped curve
and if you know the topic dead cold or have absolutely no idea how to do it,
start there because those will be your constraints, either you are an industry
expert on a topic and can define the constraints with the best of them in
which case you should definitely start there unless you suspect a trick
question (LOL you thought we meant HTML browser, actually we're passing you
DB2 ISAM files and COBOL PIC statements), or you have absolutely no idea what
you're doing in which case you should start there to begin to discover your
constraints and APIs and interfaces and data structures. Never start at a
point where you have intermediate level of knowledge, the impact when you
smack into the unknown is going to be painful so start in the unknown to begin
with.

Basically, it boils down to try not to paint yourself into a corner, so either
start painting in the most obvious safe area to start or the most mysterious
area on the assumption escape will be the easiest. But never, ever start to
paint a floor by just dumping the can out in the middle of the floor.

If you have a flat level of skill across the entire project (yeah not likely)
then if its a "simple" project like a viewer/browser where you've seen all the
parts before, start in the middle and work both directions at once if you can
handle the inevitable mystifying weirdness at both ends simultaneously, and if
its a "complex" project like replace the entire air traffic control system
then start at one edge and work to the other to minimize simultaneous
confusion. Yeah yeah I know I just said never start painting in the middle of
the floor, well if you think there's a perfectly flat skill level you probably
haven't analyzed the problem enough so the real answer is not to start at all,
and instead to think about it a lot more.

Of course half the fun is defining the starting point. Do they give you a
working example to clone as a goal, access to all RFCs, no docs and cleanroom
reimplement using a protocol analyzers traces, can you use a language you
like, can you use language libraries that implement some fraction of the
problem, does it have to be a 1995 level of browser or 2015 or 2035 level of
browser ...

To some extent its a meta interview question where they want to see if you
start talking like you're a system architect or systems analyst type like
above or if you're a security guy you'd probably start with "review all the
latest exploits and avoid all of them simultaneously (good luck lol)" or if
you're a code monkey maybe you'd start with a rant about your favorite font to
use in Eclipse and how nice your mechanical keyboard is or maybe you're not
even a programmer in which case the response will sound like the dude from the
office space movie in the scene arguing about his radio "well I was informed
this was a programming test and I got permission to list the integers from 1
to 100 and print fizz when they're a multiple of ..." and so on.

------
newscracker
[Disclaimer: I'm not an expert on developing browsers] It depends on what
exactly you'd want to implement, since a (modern) browser is really a huge
piece of software with several aspects and components in it. Looking at it
from one angle, you could split it into chrome (not Google Chrome, but the UI,
like tab bar, bookmarks, menus, extensions management, etc.) and content (the
rendering engine).

You'd most likely get bogged down if you start writing your own rendering
engine (processing HTML, CSS and JS, and processing image formats, video and
sound formats, even not including any plugins), unless it's a very limited
scope toy browser that can handle some simple stuff. In this case, start with
base HTML support, including handling incomplete tags and quirks like that
have been carried over for decades (the amount of cruft in rendering engines
for backward compatibility and handling undocumented but accepted quirks would
itself take a long time to develop). Then try a little bit of CSS support. I'd
assume JS support could be quite challenging since that moves out of the
already challenging "paint the screen" part into an even more challenging part
on interpreting code, running it, optimizing it, etc., which is a lot more
complex when you consider there would be several JS files loaded on most
websites.

Something a little more easier (only comparatively) would be developing the
chrome around an existing rendering engine and seeing how to handle tabs,
background loading, foreground loading, bookmarks, private/incognito browsing,
etc., and if you feel adventurous enough, adopt a particular process or
threading model for a multi-tab browser.

Another alternative here is to create a WebView application and see which
parts you could improve. For example, Chrome on iOS and Firefox on iOS are
just the chrome for the built-in rendering engine in iOS, but address many
things like tabs, networking, incognito mode, bookmarks, sync and other
features. You could try creating a similar one on iOS, latching on to the work
done by FoxBrowser [1] or Firefox for iOS [2]. On Android, you could try
building on top of Firefox for Android [3].

If you'd like to work on a web browser rendering engine that's a bit more
closer to "from scratch", I'd recommend working with the Servo [4] team and
seeing how you can help. It's a long drawn project that has already made huge
strides, but you'd get an inside view of the complexity of developing a
browser engine. What's more, you could also learn Rust while doing this. [5]

[1]:
[https://github.com/graetzer/Foxbrowser](https://github.com/graetzer/Foxbrowser)

[2]: [https://github.com/mozilla/firefox-
ios](https://github.com/mozilla/firefox-ios)

[3]: [https://developer.mozilla.org/en-
US/docs/Mozilla/Developer_g...](https://developer.mozilla.org/en-
US/docs/Mozilla/Developer_guide/Build_Instructions/Simple_Firefox_for_Android_build)

[4]: [https://servo.org/](https://servo.org/)

[5]: [https://www.rust-lang.org/](https://www.rust-lang.org/)

------
ayyn0n0n0
TempleOS

