
What is a Source Map - aeontech
https://www.schneems.com/2017/11/14/wtf-is-a-source-map/
======
jschorr
Some interesting history of source maps: They actually are older than the
article suggests, having been original developed and used internally at Google
starting in early 2009. At the time, Google had some of the largest
applications written in JavaScript running inside of the browser, optimized
and minified using Closure Compiler [1]. One focus of Closure Compiler was/is
minimizing bytes sent over wire, which resulted in trying to debug issues that
occurred on Line 1, Character XXXXX of the minified file. As an engineer
working 20% on a Gmail feature, I grew incredibly frustrated with having to
add `console.log` or `alert` calls everywhere, and so threw together
extensions to both Closure Compiler and Firebug to produce the very first
iteration of source maps. While only used by a few members of the Gmail and
apps teams at first, the source maps and “LavaBug” (our extension to Firebug
to consume source maps) spread internally, so much so that we decided to
release them as part of Closure Compiler’s first release (“LavaBug” being
released as “Closure Inspector”) [2].

The reason the spec lives in a Google Doc is that it is not a standard; it is
a specification that we wrote at Google and saw teams both internally and
externally pick up as it gained steam, eventually being supported in all major
browsers and most compile-to-JavaScript tools; shows how having a solution to
a problem can cause it to grow into a “standard”.

Source: I invented Source Maps

[1] [https://github.com/google/closure-
compiler](https://github.com/google/closure-compiler) [2]
[https://googlecode.blogspot.com/2009/11/introducing-
closure-...](https://googlecode.blogspot.com/2009/11/introducing-closure-
tools.html)

~~~
itwy
I choose to believe the article writer.

~~~
ricardobeat
If you click the fist link in the article, going to the google doc, you'll
find this:

 _The original source map format (v1) was created by Joseph Schorr for use by
Closure Inspector_

Username checks out. Snark or not, please avoid posting this type of comment
here.

------
iamleppert
Anyone who has worked in real web development knows that source maps are
barely functional. When they do work, they are slow and there's a non-trivial
delay, during which time the minified source or transpiled source or whatever
is shown.

There's a 50/50 shot they won't even work at all, and of that 50% of the time
they do work, they often don't have the correct line and column.

Again, things like the interactive debugger and "pause on caught exceptions"
often casually break with source maps.

In short, it's a mess.

~~~
foota
Maybe it wasn't "real web development" but I've worked in projects with
O(thousands of lines) of transpiled and minimized typescript where they worked
fine.

~~~
coldtea
Thousands of lines is barely enough to get the full pain, given that webapps
can get to multiple tens of thousands to hundreds of thousands of lines.

Not to mention several Chrome bugs about sourcemaps over the years.

~~~
AlphaWeaver
I've had no complaints about source maps in Firefox.

------
sillysaurus3
A source map is the ability to say, for every character in a minified JS file,
"Which source file did this character from? Which line and column?"

Note that there are a few types of source maps that vary in quality, so
sometimes you don't get accurate column info.
[http://webcache.googleusercontent.com/search?q=cache:qYxqOqD...](http://webcache.googleusercontent.com/search?q=cache:qYxqOqD-x3oJ:cheng.logdown.com/posts/2016/03/25/679045+&cd=1&hl=en&ct=clnk&gl=us)

The cool thing is, it's really easy to throw together a toy language with
first-class debugging and tooling, thanks to sourcemaps. See coffeescript and
lightscript for two ecosystems to emulate.

[https://github.com/jashkenas/coffeescript/blob/master/src/so...](https://github.com/jashkenas/coffeescript/blob/master/src/sourcemap.litcoffee)

[http://www.lightscript.org/](http://www.lightscript.org/)

~~~
sametmax
A source map is the reason your browser is so slow after you succeeded to
debug your 4Mo webpack generated dev bunddle you spent 3 hours to configure
right. And "your browser" is not plural because one config won't work with
another browser.

------
ridiculous_fish
The source map format is tricky. One flaw is that the VLQ encoding is strange
and hard to get right. 0 may be encoded as either A or B (?). Even the author
missed an edge case: -2^31 is wrongly encoded as B (i.e. 0) while the correct
encoding 'jgggggE' is wrongly decoded as -1. Try entering them in
[http://www.murzwin.com/base64vlq.html](http://www.murzwin.com/base64vlq.html)
.

More seriously, the format is unsuitable for random access. It must be fully
parsed and held in memory to be used.

------
childintime
This reminds me of what I think is the curious omission of source maps inside
editors or IDEs.

The opportunity is most easily explained by pointing at the issue of
significant whitespace: it should not be a language feature, but an editor
option. Those that want it should get it, no need to fuss at the language
level. It is not semantics. Source maps could do the trick. One could edit a
source mapped version, and have the edits piped back to the source.

I'm sure there are lots more applications that make sense for an editor.
Imagine having to edit a big table with lots of columns. If one could supply
the editor with a filter that pairs the text down to the interesting tidbits,
with a source map, edits could be mapped back to the source. That would be a
big win.

Another trivial example are coding styles. Everybody has their own, the editor
should interface that style to the code-base. I believe source map support in
editors could enable this.

The biggest win would be an ecosystem of plugins for such services.

------
Timothycquinn
For security and possibly performance purposes, I would like to forego sending
source maps to the client in Production and instead store the source maps,
using versioning on the server and just send serialized client side stack
traces to the server for error reporting. Has anyone done this before? I tried
digging into this several times in the past but came up empty.

~~~
prawks
I often hear people point to security as a reason to avoid shipping sourcemaps
in Production, but it seems like such a non-issue given that anyone can
unminify the code shipped out to their browsers. What kinds of secrets are
able to be hidden via obfuscation? The answer traditionally is "none" so I'm
pretty consistently baffled. We do strip comments explicitly so that devs
don't need to be as concerned with exposing anything that way, but aside from
that I don't really understand this angle.

~~~
paulddraper
> What kinds of secrets are able to be hidden via obfuscation?

Source code is very effectively hidden by obfuscation.

If that were not true, GNU and open-source and GPL would not exist.

\---

Regardless, another important thing is not to download source maps onto
client's machines, as that defeats the whole point of minification.

Decoding stack traces server-side dedups work and doesn't impose an
unnecessary performance burden on users.

~~~
WorldMaker
> Regardless, another important thing is not to download source maps onto
> client's machines, as that defeats the whole point of minification.

Browsers don't download source map files unless the developer tools are
opened. If your client is using your app with the dev tools open you may have
other problems that have nothing to do with performance.

------
GuiA
Very naive question as someone who last wrote production JavaScript in 2008:
so basically web developers are reinventing debug symbols?

~~~
the_mitsuhiko
And badly. I rrally wish JavaScript would just use DWARF. Sourcemaps are quite
broken, do not contain scope infos and cannot be used to map function names
wirhout doing stupid things.

~~~
cromwellian
JavaScript doesn't use DWARF. DevTools and production server systems do. If
you think DWARF is better, write up a patch to Babel or Closure Compiler, and
a devtools extension. I think you'll problem find that DWARF doesn't really
match the source-to-source profile case well, and in the end, you'll add so
many extensions that what you end up with isn't DWARF at all.

SourceMaps can be used to global names, but not locals. However, Closure
Compiler has the ability to generate more elaborate maps that do allow reverse
mapping. This is how Google services deobfuscate logs and stack traces sent
back by heavily optimized clients.

~~~
the_mitsuhiko
> However, Closure Compiler has the ability to generate more elaborate maps
> that do allow reverse mapping

Is that an extension to source maps that Google came up with? Because from the
current specification I do not see how this can be done without hacks. The way
we're doing it is inverse token search from a starting position over the
minified tokens until we find our minified function name followed by the
keyword 'function'.

And that approach is slow and not entirely correct.

~~~
cromwellian
SourceMaps as specified are an interop format for devtools on the client side.

On the server side, you can store a lot more information since it doesn't need
to be transmitted to the client. Closure Compiler stores maps for all
variables, all properties, all functions, all renamed strings, all
idGenerators, etc. You can store these if you want.

Google's servers store these maps and when user feedback or exceptions are
logged, they are used to deobfuscate them. SourceMaps + functionMaps +
propertyMaps + the others I mentioned as used to deobfuscate.

This doesn't solve the problem of deobfuscating locals or heap objects. That
needs an extension.

~~~
the_mitsuhiko
Sure. But what you describe is not possible with the documented sourcemap
format. You can’t even map function names at the moment. It’s purely a token
and token location mapping format.

------
holman
Something I’ve been wondering lately and have seen a few different — and
rather surprising — takes: do you include source maps in production?

I’ve seen a somewhat surprising amount of people say that they absolutely
wouldn’t these days, which seems fairly security-through-obscurity. Would love
to hear some different opinions on this, though.

~~~
nallerooth
I wouldn't, or to be more precise - I wouldn't by default. I prefer to enable
things like source maps via an environment variable or a special build - when
I know that I need to debug something. By always including source maps, I just
increase the asset size for the site's visitors, with "no benefit" (most end
users won't have a clue about how to debug something).

Regarding the security aspect - don't put anything you know to be
sensitive/insecure on the Internet in the first place.

~~~
holman
Source maps aren’t actually loaded by the browser unless specifically
requested by opening web inspector, fwiw.

------
gumby
How is this different from debugging info? They should just use DWARF instead
of reinventing the wheel.

~~~
openasocket
It's very similar, but they wouldn't be able to "just use DWARF". Half the
DWARF tables are devoted to byte sizes and where variables are stored in stack
frame offsets or registers. It's not like you could fork libdwarf or whatever
and magically get JS debugging, the effort required to change to the different
scenario is enough that it's not worth the effort.

~~~
laichzeit0
Honest question, will any of this matter once we've gone full WASM? Will DWARF
be a good option then?

As I understand in the not too distant future, everyone that absolutely
despises Javascript and the whole ecosystem it re-invented will be able to
merrily go about their way using the old familiar tools and languages from
yore when they can compile to wasm.

~~~
johncolanduoni
There's some significant complications, like the fact that most languages that
compile to WebAssembly will need to use two stacks at certain times (including
C/C++). In this respect WASM is in a similar situation to .NET and the JVM,
which use their own debugging information formats for good reason.

Considering this wouldn't make much of a dent in the work needed to get tools
like gdb/lldb to work with WebAssembly engines I don't think it's likely to be
the best path forward for WASM.

------
ivanhoe
Unfortunately, at least in my experience, JS source maps are pretty useless
for any real life debugging because browser almost never manages to set a
breakpoint on the line that you've marked, so it gives you some value only if
you don't already have access to the original unminified sources.

However CSS source maps are of a great help for less/sass.

------
Dowwie
check out the Rust implementation that Armin made for Sentry:
[https://blog.sentry.io/2016/10/19/fixing-python-
performance-...](https://blog.sentry.io/2016/10/19/fixing-python-performance-
with-rust.html)

------
Spearchucker
He says in his article that minification is best practice. And I imagine I'm a
dinosaur and (or) completely out of touch, because I disagree.

Minifcation becomes a thing when the page is too big. And many pages* contain
bloat that do nothing to improve my lot. I write my CSS by hand, and keep it
as sparse as I can. When I do use JavaScript it does a specific thing, like
filter a dataset or move a slider. And it's easy to debug because it's pithy,
and uses sensible naming.

* Some pages - specifically real-time apps - need lots of JS bloat. I get and accept that. Happily I don't need to make any of those beasts.

~~~
joekrill
In pretty much every case minification is going to reduce the payload, which
is going to reduce the amount of data a client has to download. Whether you're
minifying 1KB down to 100 bytes, or 50 MB down to 3 MB -- you're still
benefiting.

I can't really see a case where that is a bad thing, particularly when you
have source maps to get you back to the original source code, anyway. Your
stance just seems very naive.

~~~
Spearchucker
Source maps have never worked out for me, and my pages are tiny (I see no
benefit in sacrificing readability for a reduction from 1K to 100 bytes).

~~~
rimliu
That's an interesting claim about readability. You may as well argue that
compilation of C program reduces its readability. Minification should be a
step of making a production build. It does not affect source code and thus it
does not affect readability.

~~~
lmm
> You may as well argue that compilation of C program reduces its readability.
> Minification should be a step of making a production build.

You can't run C without building it, so you have to do some kind of
compilation. I've certainly known people to argue for e.g. including debug
symbols even in the build you ship to customers - the savings from stripping
them aren't worth the added complexity and difficulty of debugging.

------
paulirish
Sokra (of webpack) and I have hacked on a sourcemap visualizer which really
illuminates exactly how the mappings work:
[https://paulirish.github.io/source-map-
visualization/#raven](https://paulirish.github.io/source-map-
visualization/#raven)

A few examples are available, but you can also drag in your own files to view
those.

------
davidcamel
When I look at downloaded minified JS files in my browser's dev tools, there's
a button to de-minify/pretty-print the JS.

What is the advantage of the source map?

------
gowthamgts12
Thanks sir.

------
jankotek
This is why Javascript is 20 years behind...

~~~
jwandborg
Could you tell me the language which is 20 years ahead of JavaScript?

~~~
nevir
Pretty much all the native ones. When you're debugging c/c++/etc, you're
relying on source maps (via dwarf/elf/etc)

~~~
madeofpalk
I guess for certain definitions of ‘behind’.

