
Tackling Webdev as a Bioinformatician: why is it so hard? - LifeIsBio
https://jessimekirk.com/blog/web_for_biofx/
======
mattlondon
Tackling bioinformatics as a webdev: why is it so hard?

"I am a professional in Field X but I am doing work in Field Y despite having
minimal skills or experience and I ran into loads of problems and issues that
I don't know how to fix! Field Y is total bullshit and people need to make it
easier so people with no skills or experience can do it! Wah wah!"

... or to be less facetious, modern web/frontend development is a genuine
skill set that benefits from experience and knowledge.

Should we expect anyone to just jump in cold with minimal experience and not
find it difficult? E.g. in the same way would we expect people to start
bioinformatics cold and be productive immediately?

I'd doubt it.

On the past web dev has been looked down upon as somehow "lesser" than "real"
engineering, with sneering criticism from people more used to traditional
development approaches (particularly criticism of JavaScript). It is just some
web pages right? HTML and CSS? How hard can it be?

Well guess what: it is a genuine skillset and people should not expect to just
be able to jump into something without knowledge training or experience and be
proficient at it, just like we'd not for brain surgery, car mechanics, concert
piano, structural engineering, or indeed bioinformatics.

</rant>

~~~
emsy
This shouldn’t be the top comment. Yes, switching fields and expecting to just
work professionally is naive and insulting towards the other profession. Has
web dev a lot of legacy cruft and suffers from over-engineering? Absolutely.
Even for seasoned programmers starting web dev is harder than it needs to be.
Front end development has JS frameworks, CSS or compile-to-CSS languages,
templating, web packing, JS transpilers, tree pruning, multiple package
managers and what not. But in the end it’s just a bunch of text, images and
buttons.

~~~
stupidcar
You can reduce any field to "it's a just bunch of...": All coding is just a
bunch of binary numbers. All chemistry and biology is just a bunch of
molecules. All construction is just a bunch of concrete, bricks and steel.
It's nonsense, and is exactly the kind of naive and insulting statement you
talked about.

Complexity is emergent. Look below any complex system and you'll find a
simpler one, but that doesn't mean the higher-level complexity is unnecessary.
All the things you complained about have good reasons to exist, invented by
people who actually work as professional web developers and need to solve
problems.

The real problem is "seasoned programmers" who don't work in web development,
but who expect to be able to jump in an master it in a day, based on the fact
that they put together a GeoCities homepage twenty years ago, and are enraged
and disgusted to find that things aren't so simple anymore. It's absurd.
Developers wouldn't expect to be able to jump into kernel development, or
database development, and find it to be a breeze.

~~~
rsynnott
So, the thing is... I'm not a webdev, though I've done web development in the
distant past (early noughties). I'm not a desktop dev, though again done it in
the late nineties and early noughties. I messed around with modern
desktop/mobile dev recently (specifically Cocoa) and it's basically fine; in
many ways simpler and better abstracted than what I was used to (mostly GTK+
and some win32 stuff back in the day). I looked at modern webdev stuff... and
I'm sorry, it's insane. Vast increase in complexity for, for most purposes, no
real gains.

I can't think of anywhere else this is the case. If you want to access an SQL
database, that's easier than it used to be, and better abstractions on top are
available if you want them. If you want to write a web API, that's easier than
it used to be, and again nice abstractions are available. If you want to
process tens of terabytes of data, that's so much easier it's not even funny.
But if you want to write a webapp... unless you totally ignore the modern
stuff it's vastly, vastly more complex than it used to be.

I suppose this may be my grumpy old man side coming out, but I really don't
find modern single page apps to be a better experience than mid-noughties
webapps in the vast majority of cases. They're usually worse; in particular
they're usually slower, and mess with UI expectations more. And yet they are
_vastly_ more difficult to produce. The whole thing seems like it took a wrong
turn somewhere.

~~~
unclebucknasty
This is not talked about enough. We frequently ask whether it's too complex,
but that assumes the objective is even sound.

Anyone who has done significant work on a full scale SPA using any framework
(or no framework) would have to weigh whether it was worth it. And that's on
two levels: 1) Was the additional effort and complexity worth it for the
_perceived_ improvements in user experience that were sought? And 2) were
there actually any such improvements or, even, is the user experience actually
worse as a result?

Because I have to say that I'm not convinced. There does seem to be a period
when AJAX first became a thing wherein the added responsiveness gained from
judicious use of dynamic operations with partial page updates improved the
experience over full page reloads exclusively.

Then, someone said, "hey, if that's cool, then why don't we make the whole
thing dynamic, fighting Web semantics (and standard browser behavior like back
buttons and history) every step of the way?

Not so sure that's a good premise or has led to a good result.

------
saagarjha
All programming has a barrier, but I think web development is a unique
situation where there’s a lot of complexity because it’s so open to abuse. If
you don’t do HTTPS right bad things can happen, so setting that up is hard. If
you process payments bad things can happen, so setting that up is hard. If you
send people email bad things can happen, so setting that up is hard. With
things like game programming the worst that can happen is your game is slow or
buggy or laggy but usually you’re not exposing it to the huge, often malicious
audience. With web development you make one mistake and someone is stealing
thousands of dollars from you and running cryptomining on your VPS while
defacing your website.

~~~
Cthulhu_
But things like HTTPS, payment processing, emails / account management and
even game programming, all of these have already been made easier - nobody has
to build any of that on their own anymore, at best they have to configure it.

To make a safe website, stop wanting to do everything yourself. Use a webhost
that doesn't give you SSH access, or a service like Netlify.

~~~
AdrianB1
And linking all these heterogeneous pieces together and configure them to work
is like cat herding: each component is so different and the developers are so
proud of their "innovative" way of doing things that one cannot master it. A
few years ago I started to believe that modern web developers are more about
showing off and hipstering than having a genuine desire to make things better.

~~~
searchableguy
Weirdly, I think the same. I noticed an uptick of twitter programmers.

------
ertian
I feel like interacting with there real world, and with users in particular,
just inevitably leads to orders of magnitude more complexity.

You can see that even in simple cases. Writing a basic single-propose script
is trivial; it may end up being just a line or two. As soon as you decide to
make it general and reusable, though, complexity explodes: positional
arguments are confusing, better name them. Users are gonna expect a familiar
format. Need a library for that. Reasonable and understandable names are
tricky! What arguments are required versus optional? What if _this_ option is
specified without _that_ one? Gotta add proper errors for every bad state!
What if the user starts using globbing? Or wants to specify multiple inputs
and one output, or vice versa?

Your three-line script is now 100+ lines. And you're not even worried about
hostile or abusive users yet!

Writing code for the web is all that--and much more besides.

------
teekert
In a related article: "Tackling bioinformatics as a Webdev: Why is it so
hard?"

Maybe Webdev is just complex? But tbh, as a bioinformatician (in Next
Generation Sequencing mainly) I actually find Webdev easier. I mean, we face
Bam files, Sam files, Cram files and annotate them with Bed files (with many
different poorly documented standards) or GTF files depending on your Ensembl
or UCSC references persuasion... Some file types are 0-based inclusive, some
are 1 based exclusive, some call chromosome just ints (1,2, and then error
when X, Y if you really assume they are ints) some call them chr1, chr2, chrX
etc. We have fastq files with different quality encodings depending on their
age, I recently had some strange errors, turns out the R2 of a fastq file is
reverse complementary from the standard (why?!)... It's a mess and many people
that spend years in the field still don't know more than just "choose one side
and stick with it"... We also have a huge amount of frameworks for data
analysis pipelines, Snakemake, Bpipe, Broad's WDL, etc, etc, etc. Not to
mention the insance amounts of read mappers (BWA, RSEM, Tophat 1 and 2,
Kallisto etc). Need I go on?

~~~
LifeIsBio
That's super interesting. If someone asked me if bioinformatics was complex,
I'd say "Yes, massively so." But then, I would take it in a totally different
direction. To me, the complexity of bioinformatics come from the biology. How
do you think about weird biological edge cases? What do your distributions
look like and how do you choose thresholds? How do you even form a good
question?

Thinks like [B|S|Cr]am files are complex in their own way, but mostly they're
just kinda annoying to have to figure out. You can dig in as deep as you need
pretty easily.

On the other hand, maybe I'm over-simplifying because I feel more comfortable
in this domain, which I think is your point.

~~~
teekert
Well, I come from Biology, always had Webdev as a hobby and then moved into
bioinformatics which now turns towards software development. So Biology feels
most familiar, but it is indeed also always complex.

Btw, the files are ok and with effort can be understood and maybe some could
even be considered elegant. It's just the shear amount of "standards" or
rather the lack thereof.

------
MattGaiser
Documentation is frighteningly bad across the industry and nobody has a good
solution yet other than hoping that someone else will do something about it.

And if you think Django is murky, try Spring :). Django is one of the easiest
frameworks to work with simply because it includes so many functions you would
otherwise need to somehow integrate. And even then there are still these
issues.

Good post.

~~~
bayesian_horse
I think Django's documentation is fantastic. 90% of the time I have to look up
something, google redirects me there anyway.

~~~
MattGaiser
Django's documentation is great for developers who know their way around
documentation like that as it is very comprehensive. It drills down nicely and
is quick and easy to understand if you otherwise know what else is going on.

What it misses is mapping that documentation to the specific problem a person
wants to solve or the feature they want to build. When I was learning Django a
couple years ago I found the examples and best practices sparse.

~~~
jldugger
A long long time ago, there was a really great website called djangobook.com.
It had like 20 chapters and projects explaining how things are done in Django,
and each paragraph had a comments section where people could ask questions
specific to that paragraph. That site probably did a lot for Django's success
in the community.

Unfortunately, giving away a free resource like that doesn't balance the books
and years of neglect later the site no longer represented modern Django. It's
entirely possible the new Django book site is just as good, but the paywall
discourages novices and experts alike. My friends who stayed webdevs think Two
Scoops of Django is close in scope, and comes with free updates. It's not
quite the everbook that I had hoped the original site would be, but is highly
focused on best practices.

~~~
VBprogrammer
The Two Scoops book is good but I found it difficult to read due to its
extremely contrived examples. Maybe that is just me though.

~~~
porker
Not just you. Also annoyed at their pricing model; I bought the original book,
3 months later it was out of date and "buy our updated copy" was their answer.
I got no use out of it.

------
Hasu
It's funny, when I was first learning programming 15 years ago, the thing I
heard everywhere was "webdev is easy and for noobs", and sure, it was 15 years
ago and webdev has come a long way... but when I interview the fresh-out-of-
college candidates for junior positions, there's still the elitism of "If you
aren't managing memory and flipping bits you aren't a REAL programmer."

No, most web developers aren't implementing the most complex and intricate
algorithms or 'close to the metal' or whatever other gatekeeping people want
to claim, but that doesn't mean it's easy work. I've never had a job where I
didn't see some green engineer laugh off a feature request with "Oh, that's
easy, it's just calling an API and putting the information into some HTML,
what's so hard about that?" and then getting stuck when their first
implementation fails QA, or the product manager explains that the customer
needs it to be ever so slightly different, and the complexity of those edge
cases turn out to not be so easy to deal with as the world of perfect computer
science might suggest.

Is it hard in the same way? No, I will fully admit that coming up with a novel
algorithmic solution requires a different mindset and brilliance that very
few, if any, really good web developers I've met really have (although I'm
certain there are people who excel at both, they're rare enough that I haven't
been lucky enough to encounter them). But being really good at building a
solid, secure, modern website isn't easy either, and it's _also_ valuable.

This post focused on the complexity of the tooling, which yes, is definitely a
thing, but if you don't know web development, and you _know_ that you don't
know web development, why are you attempting to build something using complex
high level tools designed for and used by advanced web developers? Then again,
if you don't know web development and you don't know what you don't know, and
you've just heard 'Django is good' how would you know to maybe pick Flask or
something simpler to understand?

I don't have an answer or a larger point other than this: writing software is
difficult and complex and none of us should look down on anyone else because
what they do seems easy on the outside.

~~~
guggle
I wouldn't count Flask as simpler than Django. These 5 lines hello world
examples are incredibly misleading.

~~~
padthai
Exactly.

You end up cobbling together half a dozen plugins (auth, forms, orm, admin)
and adding dependencies that, albeit useful, are complex themselves: Celery,
SQLAlchemy, etc.

~~~
guggle
Having a single source of documentation and components perfectly integrated
out of the box make a huge difference.

------
gexla
You start with HTML. Write your HTML in an editor. You now have what we
started with and I won't complain about your un-styled site. Right now I'm
working on a simple personal thing with unstyled pages and a static site
generator (Hugo.)

The first Google search for hosting should be fine. There should be a simple
tutorial for uploading your HTML file.

That's all there is to it. Beyond that, maybe you're just trying to bite off
more than you can chew because you think you need things. It would be like
going to ACME engineering supply and grabbing a bunch of stuff off the shelf
because it sounds good. Then you get home and you have no idea what these
things do.

A lot of this stuff looks deceptively like it all goes under the category of
"web development." Sending an email isn't web development. That's dealing with
a different protocol and different server software. You don't need to go far
down that path before you start to wonder how any of this mess can work at
all. I'm not even talking about web development, I feel this way all the way
down to the metal. Yesterday I spent hours trying to troubleshoot an issue
with Ryzen 3rd generation chips apparently having a problem when you load up
all the slots with RAM (I threw in the towel because I can get by with half
the RAM.)

That has always been the case. Software hasn't gotten worse, it has always
been bad. But functional. Maybe it has become more complex.

~~~
Cu3PO42
Anecdotally I have a Ryzen 3600 on a B450 board with all RAM slots populated
and have yet to notice an issue that isn't there when using only half the RAM.

~~~
gexla
Probably the wrong place for the discussion. Do a search and you'll find all
sorts of people with this issue. I had the problem after I first built the
computer. Then I was able to boot by lowering the speed of the memory in the
BIOS, now I have the problem again after a reboot. I don't know what changed.
There's hundreds of threads which cover a lot of ground on this issue. I'm
just going to make this a long term project to understand the issue until I
finally cave and buy something which has already been built.

Edit: Also 3600 with B450 board. Kingston DDR4-2666 Ram (8x4) which is all the
computer store here in smallish city in the Philippines had in stock.

------
KaiserPro
I 100% feel your pain.

I used to work for a very large financial news company. They were all
javascript. I though to my self: I learnt JS in 2008, this should be easy (it
was 2014). Apparently very hard.

HTML isn't HTML anymore JS is forced everywhere, and everything has a
different name.

Anyway, I disappeared back into HPC/VFX/machinelearning, to re-appear into
webdev in 2020.

I was forced to do some front end work as part of a bootcamp. It took the best
part of a week to make a bit of text a link.

The template library was so far removed from anything googleable, there was no
documentation, worse still no style guide or example style guide.

I am glad I don't do web GUIs, they are hard and nasty.

~~~
onion2k
_It took the best part of a week to make a bit of text a link._

This highlights the problem with web dev quite nicely. Not doubt you weren't
_really_ just making some text in to a link. I know the sort of thing
bootcamps do, so you were probably installing and learning React, displaying a
set of components, implementing React Router, and figuring out how to update
app state based on the browser's History API. All with a template library that
was a terrible choice for learning (because it didn't have docs). You _could_
summarize that as "making a bit of text a link", but that's glossing over the
details.

What you were _really_ doing was learning component orchestration, reactive
state management, and a browser API. Why _wouldn 't_ that take a week?

So the problem with web dev is actually that it's really, _really_ easy to
trivialize. You can make hard things sound like they should be very
straightforward because they look a lot like something simple from the past.
And when people find that they're not really doing the simple thing any more
they complain because the new, harder way of doing something (which is
actually doing something _completely_ different eg reactive state management
and in-browser routing vs asking the browser to do a GET) is hard.

~~~
dynamite-ready
And there's the problem...

"...component orchestration, reactive state management..."

Webdev tool makers are sadists, and many of the 'thought leaders' in the field
have Stockholm syndrome.

Some aspects of web development certainly are hard (security especially).

Frontend development however, does not need to be as hard as it currently is.
At the very least, I think there's an opportunity to simplify that layer of
the stack, and give the control back to design professionals.

Dreamweaver to my mind, represented a more productive direction than React
currently does.

~~~
onion2k
On the contrary, front end isn't good enough yet, and getting to the point
where it's ideal will require _more_ complexity. Users want things to work
fast. If you're including a roundtrip to a server your app will _never_ be
good enough for a user on a rural 1Mbps satellite connection with 500ms
latency. That's the goal - your app has to work well for that guy. You _can
't_ get it to be fast if you're using the network a lot. We need concurrent
rendering, async background processing, and as much work shifted on to worker
threads as we can manage. All the things that native apps do to crank up the
performance need to happen on the web too.

Also, don't forget that the web is fantastically backwards compatible. You can
write web apps without the complexity if you want to. You can use Dreamweaver!
You can server-side render every page. It works. If that's what you want to
serve your users then you should do that. Just don't complain when they use
the competitor's app that's faster, more responsive, and does fun things like
working offline.

~~~
dynamite-ready
More than any framework, what most market leading applications need to provide
a competitive service, is greater access to creator resource (developers /
designers / content creators, whatever) and / or innovation in pre-production
and design.

The problem with current development trends is in how the complexity found in
the tools, makes the pool of available creators (at a want for a better term)
a tiny, exclusive club. This has a detrimental effect on efforts to innovate
at design / pre-production stage, because the designers are near enough held
hostage by us developers.

It also has an overall effect on project velocity, because work that could
possibly be covered by techically competent designers (namely HTML/CSS
development), can only be addressed by expensive and exclusive 'framework
specialists'.

I strongly believe this can change by rethinking where the complexity lies.

The culture built around over-complicated tooling is possibly analogous to
placing the responsibility for the production of a feature film in the hands
of the VFX and lighting teams, or perhaps only deeming a mechanic as having
the aptitude to drive a car.

I'm not rigidly stuck to this idea, I'm just musing here.

~~~
onion2k
_perhaps only deeming a mechanic as having the aptitude to drive a car_

This analogy is a really good one, but you've got it slightly wrong. The
website user is the person driving the car. The developer is the equivalent of
the person building the car. Saying the development process is "too
complicated" is like saying developers should only ever build compact family
cars like a Ford, but some of us want to be building BMWs and Teslas and
Porsches and F1 cars and monster trucks and semi-trucks, etc. Sure, it's more
complex and difficult, but the end result suits the user better, fulfils their
needs better, and often just makes them a lot happier.

------
valera_rozuvan
I start getting my friends up to speed by giving them just enough JavaScript +
HTML to write a Tetris clone. Of course - I guide them along the way, answer
any questions they have, and push them towards the right direction if they get
stuck. If they can get through it - then they can make it in the world of
WebDev, and software engineering in general.

The main thing to remember is that you can only show the way - the person in
question must actually step forwards and be motivated enough to go through the
learning process. Otherwise - you need to think of another career. Programming
is not for everyone IMHO.

~~~
LifeIsBio
A little tangential, but learning to code through video games has always
seemed like one of the best ways to do it, particularly with friends.

------
bpizzi
And it looks like you've just scratched the surface, you're not finished
encountering walls yet.

Hang in there, there's a relative important chance that the fog will clear for
you in some months/years.

The hard truth is, software engineering today is made of layers upon layers of
abstractions. You have to dig threw those layers, understanding sufficiently
the inherent and different complexity of each one before jumping to the next.
And then finally it'll make sense, and new layers (there's a new layer every
year btw) will be easier to grok.

If it helps, we've all had to go threw this learning process. All of us.
However I don't think everyone is made for enjoying this process, that's
another hard truth.

The real difficulty is sticking to the learning process while everything and
everyone is screaming at you "Go! Build something! Ship Something! Show
something you lazy b __ __".

------
travolter
Http is just clearly not made for the current state of the internet. It was
obviously made for much simpler uses cases but has organically grown together
with the internet. This has led to numerous hacks and workaroudns which often
became the defacto standard to do things on the web. It's not necessarily an
easy problem to solve, client server communication is hard. But http protocol
seems like the least effective way to do it, but almost impossible to replace
now.

~~~
TheOtherHobbes
Applies even more so to CSS and JS.

All the above are quick hacks that never really did the job properly in the
first place. So they're propped up by a creaking pile of frameworks and other
support systems that try to mitigate their awfulness, but mostly just add
extra awfulness of their own.

Meanwhile backend systems - core server and db infrastructure, user
authentication, computing resource allocation, payments, media and content
management, privacy, and project build systems - are even more byzantine and
non-standardised and more or less have to be retooled from scratch for every
project - a huge job which duplicates an incredible amount of work across the
entire industry.

------
maelito
> or the “right” framework to choose, changes from year to year

I've been using React in all of my projets since it came out in 2013. That's 7
years. React's (+ node for backend if needed) been one of the recommended for
at least 4 years.

And lots of things have changed ! Using modules and building complex apps now
with Webpack & co is so much easier than before. Not needing to support IE is
also a blessing.

The piece doesn't say how mobile dev would be easier. Or how embedded dev
would be easier.

It just says : dev is hard. Yes it is, it's a a full job.

Hopefully nocde solutions will reduce the gap for some use cases.

~~~
simongray
React - as a library, but especially as a UI paradigm - has clearly won the
frontend war for now, taking the mantle from JQuery. I don't even use React
directly, just its DOM renderer through a ClojureScript wrapper (reagent) and
the high-level concepts are still basically the same.

Still, when I glance at the JS frontend world people are _still_ looking for
the next hot frontend framework all the time, even if many are doing something
similar to React. Going by github stars, Vue is the biggest one now. There's
also Angular, Svelte, Ember, and Aurelia.

~~~
maelito
You're right. React and Vue are hard to compare though. The fact that React
has no router, while react-router has 40k stars makes it a very imprecise
metric.

------
thrower123
Yes, web development is hard and stupidly complicated and slapdash and we all
hate it. It sucks, and the whole thing should be burned to the ground, the
ashes sown with salt, and pissed on by donkeys.

Unfortunately we're stuck with this Rube Goldberg collection of excrement, and
it pays the bills. So there's nothing much you can do except lean in and
embrace the suck. Maybe hit yourself in the forehead a few times with a
clawhammer or start carrying a hipflask to keep yourself at happy-drunk.

------
cpach
Very thoughtful article. There is so much accidental complexity in software.

~~~
LifeIsBio
It's kind of ironic. While I've been thinking about the complexity of
software/webdev for a while, I was feeling a little unsure about posting this
particular link because I wrote it in a hasty, semi-embittered fit of
annoyance. Regardless, thanks for the read. :)

~~~
tejtm
hopefully this observation won't be taken too harshly...

I have also worked in programming for the sciences my entire career, multiple
decades, would not want to do anything else. There is an awful lot of actual
complexity out there especially in the life sciences which is inherently messy
... a system of exceptions ... Sometimes looking at the stuff that has to
happen now, just to give information away for free it is staggering. The only
conclusion I come to is complexity envy, Because I can't believe all this
inevitablely became necessary. Yes there are bad actors and good security is
very hard, and much more so if you are transacting money, but in other cases
you can't really steal what is being given away but now there is the
expectation that all this extra stuff is par for the course it you are putting
on the web.

------
sradman
I love these kinds of retrospectives as they give insight into the learning
process. We tend to focus on the acquisition of specialized knowledge but
there is often a body of shared knowledge, the context you might call it, that
we pick up indirectly.

You have to read between the lines but what we have in this post is a Data
Scientist skilled in the Python programming language struggling to acquire the
shared knowledge required to build and deploy Web Apps. One toolchain
(Python), one framework (Django), one backend architecture (12-Factor Apps)
but the narrative surrounding URLs, HTTP, DNS, TLS, REST APIs, JSON, CORS, and
www-forms is missing.

We seem to be missing a simple way to learn the shared knowledge needed to
build apps in a world reshaped by Web, Mobile, and Cloud technologies.

~~~
LifeIsBio
I’m not sure exactly what it is about this comment, but I really like it.
“Context” is a great word for this idea. There are a bunch of small/medium
complexity things that need to play well together to get things to work, and
it can feel trickier than necessary.

------
cmrdporcupine
Honestly I've always found UI development or anything with some aspect of UI
in general has the potential to devolve into a tangled mess. I've avoided web
facing development for the last 10 years of my career, and have transitioned
to embedded work. Every time I traipse through job postings and look at the
tech stack companies are using ("wanted: full stack developer") I just shake
my head. I am sure most of the choices are justified, but it all seems so
brutally overwrought from the outside looking in. But beyond a fetish for
novelty in our industry, I think that's just endemic to user facing
development.

------
bayesian_horse
Trying to make the opposite transition, my quibble with web development is
that much of the work seems so pointless.

I hope that after this crisis we will care less about making people click ads
than about learning how stuff works in biology.

~~~
Animats
_Trying to make the opposite transition, my quibble with web development is
that much of the work seems so pointless._

That's the problem with web development today. Something that not only should
be straightforward, but used to be straightforward, is now very complex.
Partly because it involves connecting a large number of complex and constantly
changing packages.

The last time I wanted to do something with an API, I coded up the API in Go
and ran it from FCGI to get multiple copies. Easier than all those
"frameworks". Probably faster, too, since Go is hard-compiled.

~~~
realharo
_> used to be straightforward_

As someone who remembers having to do weird CSS tricks for really basic
layouts (negative margins? hidden extra elements that don't make any sense,
but cause other elements to move?), manually modifying the UI made up of
jQuery plugins that require a very specific prescribed HTML structure to even
work, or doing significant adjustments for 3 or 4 different browsers, I
definitely dispute that claim.

Web development right now (at least the frontend for sure) feels like a breeze
compared to about 10-15 years ago.

Steps 1 and 2 of the whole process may have gotten more difficult, but steps 5
through 100 are a lot easier these days.

------
IshKebab
Backwards compatibility.

It's possible to design coherent easy to use UI systems. Flutter and Qt are
great and both much easier to develop for than the web IMO. But they don't
have to worry about running in web browsers.

------
rgoulter
I don't think the author is saying "why front end application development
hard?" so much as "why is setting up a website hard?".

On one end of the spectrum, simply putting content on a website isn't so hard,
right? On the other end, making a web application is understandably difficult.

Features like users, payment forms, and email confirmation (so e.g. the
website can be used for a course) sounds like they ought to be as simple as
the former, but are closer to the latter.

Is the value proposition from the website building websites like Wix that they
take care of the hard stuff?

------
6510
People who understand the mess are much more influential and they are moving
the show forwards. Making things easy for newcomers is just not very high on
the agenda (anymore)

Perhaps things have to get complicated enough for it to get priority, perhaps
cool things will happen all of a sudden. I was highly amused by the
spreadsheet programming tool a while back. It seems a giant leap in the other
direction.

------
not_a_moth
This is common among academics, who expect full transparency and understanding
in what they're doing, instead of a "get it working and go" attitude.

The problem is the OP didn't test anything: He launched and users couldn't log
in.

Ideally I'd like to gain 100% visibility into everything related to my tasks
in webdev, but it's impractical and counter productive.

------
dade_
It is hard to get the hang of Django is because the polls tutorial isn't
helpful to actually understand Django and no one wrote this book yet, but I
think Nigel has done a great job: [https://djangobook.com/mastering-
django-2-book/](https://djangobook.com/mastering-django-2-book/)

------
juskrey
Welcome to real world, academic Neo.

~~~
LifeIsBio
As my PI used to say, "If it were easy, it wouldn't be fun. And we wouldn't
have jobs."

------
edoceo
It's awesome to see an article get to the five why's. This is the Way.

~~~
LifeIsBio
This has been my first experience with it and it seems like a great tool to
ensure structured thinking. It's easy to give up a level or two deep and just
say, "Oh, it's probably because..."

------
walleeee
Related: institutions and labs need to recognize that 'research software
engineer' is an increasingly necessary role and attract/fund more of them.

------
heisnotanalien
Because instead of fixing genuine problems and making it easier people invent
new languages and frameworks to do things in their own way.

------
namelosw
It is no brainer for professionals, but to be frank, it's too hard for
everyone. It's time for a new PHP.

------
CameronNemo
looks like it got hugged

[https://web.archive.org/web/20200507060013/https://jessimeki...](https://web.archive.org/web/20200507060013/https://jessimekirk.com/blog/web_for_biofx/)

~~~
LifeIsBio
Oh no! It's github pages and totally static, so it should recover pretty
easily, I hope. Thanks for the archive though!

~~~
CameronNemo
Must have been a blip on my end then.

~~~
hutattedonmyarm
It definitely got hugged, I couldn't access it either for a bit. Recovered
quickly enough though

------
tannhaeuser
Is the author asking why putting up a paywall for Django hosted on Heroku with
Stripe integration on what should be static content is difficult (for stories
related to overcoming idiosyncrasies in using Python for bioinformatics, or
all things)? I'd like to give a rebuttal, but the site (the Minimal Errors
theme for Jekyll?) does click-jacking (blocks copy/paste) so I won't bother. I
guess the answer is right there: webdev (and bioinformatics) is full of
incidental complexity and zero-sum "solutions" advertised to you when the
original purpose of the web is just publishing structured documents via
HTML/SGML (even though I'd agree Python is great for learning, and Heroku has
at least a reputation for being the easiest cloud hosting option out there).

~~~
LifeIsBio
The site that's giving me problems isn't the site you were on, but another
site I'm building:

[https://mycodestories.com/](https://mycodestories.com/)

It's not static; it's a platform for helping people (life scientists) learn to
code.

In general, Heroku has been a great platform. The fact that you can just `git
push`, and your dyno rebuilds and restarts and your changes are live is so
cool.

------
ml_basics
A great read!

------
jldugger
> Why’s there so much data in the url? A common refrain I’ll use for the
> remainder of the post is, “I don’t know”. One of my biggest gripes with web
> programming is the number of rabbit holes to fall down. There’s just not
> enough time.

I mean, bioinformatics is _notorious_ for spending just enough time working on
software to publish a result and no more. If we want to talk about rabbitholes
I nominate BioPerl.

> here are a list of terms that you’d never come across as a bioinformatician:
> ["X-Forwarded-Proto", "proxying", "requests", "TLS connection", "HTTP",
> "loopback/localhost", "whitelisted proxy servers", "gunicorn", "nginx",
> "Apache/httpd", "HAProxy", "Django", "SECURE_SSL_REDIRECT",
> "SecurityMiddleware", "adapter", "URI"]

This list is more than a little disingenuous:

Django: This is like, the framework OP built their software on top of. I don't
know how one gets the point of filing this bug without knowing the term Django

requests: literally, a user's web request.

HTTP/URI: this is the web developer equivalent of a biologist not knowing the
term DNA. Or DOI for that matter.

whitelist: the opposite of a blacklist

Much of rest comes down to the simple fact that job of web developers is to
build a network service. This is primarily done with layers; each layer only
needs to know how to interface with the adjoining layers above and below, like
a lego brick. This is actually a _solution_ to the complexity, allowing us to
constrain the design of each layer to interact only with those above and below
it. This layer abstraction lets us reason about each layer locally,
independently of the system as a whole.

nginx, Apache/httpd, HAProxy: in this context, mentioned as possible proxies
forming the outermost layer. in this context, accepts HTTP/HTTPS connections
and makes HTTP connections to gunicorn

gunicorn: the thing that runs the Django app. Accepts HTTP connections, makes
WSGI calls to the Django app

Django: Accepts WSGI function calls, routes them through middleware, then to
the app as you define in urlpatterns

SecurityMiddleware: a class that potentially modifies inbound requests and
outbound responses

Finally, OP is in the unhappy situation of bridging two network protocols:
HTTP and email. The normal trick of using a protocol relative URI will not
work here because you've gone outside the browser request-response loop. This
puts them in the unhappy state of the lowest layer needing to know the details
of the highest layer, hence the discussion of absolute uris. This is a direct
violation of the local reasoning constraint, and what makes all of this hard.

X-Forwarded-Proto: An HTTP header, added to incoming HTTP requests designed to
indicate to layers below which URI scheme was originally used, HTTP or HTTPS.

SECURE_SSL_REDIRECT: a setting for SecurityMiddleware that will stop
processing the request and send a response to the browser telling it to resend
the query over https (an HTTP 302 redirect), if the incoming connection is
seen as coming over not http.

tl;dr: email is the thing that sucks.

------
goatinaboat
Because it's artificially made hard by people who enjoy complexity for its own
sake. The article is about implementing a shopping cart, as far as I can make
out. Well people used to write those kinds of websites, that perfectly well
enabled users to browse products, fill a basket, place the order and make a
payment using nothing more than notepad.exe.

Some software is legitimately hard to write, even some websites. But this
isn't an example of that. The author has been sent on a wild goose chase.

~~~
LifeIsBio
I'm curious where you got the idea that the site is a shopping cart?

CodeStories is a site to help life scientists learn to program by giving them
problems, then letting them submit scripts and then auto-graded their scripts
for correctness.

~~~
goatinaboat
_CodeStories is a site to help life scientists learn to program by giving them
problems, then letting them submit scripts and then auto-graded their scripts
for correctness._

Because it’s a pretty generic e-commerce application that could have been
built on the web 25 years ago. It should not have been complicated to do.

