Hacker News new | past | comments | ask | show | jobs | submit login
Ask HN: Why web frameworks are becoming so complex?
33 points by mr_nobody on Aug 23, 2015 | hide | past | favorite | 42 comments
A little about me: I am web-developer with almost 10 years of experience. I've spent lots of time in my life learning about design patterns, frameworks, and going through hundreds of programming blogs. I work mostly with quite big business apps (crm's, intranets) but also big frontend websites.

Since I started working as developer I can see that every year majority of leading frameworks (at least in PHP, Javascript, Ruby...) for web applications are becoming more and more complex. Instead of writing code in programing language we move into xml/yaml direction and hundreds of weird config files. Each frameworks use different syntax for config, often different templating languages, few layers of abstraction with a lot of magic behind the scene, Dependency Injections, ORM'S, CRUD generators with supposed to make our life easier actually are painful to use...

Even small website often requires hundreds of megabytes of dependencies. Sometimes one application can have 2 different build systems e.g. (ant, grunt) and few dependency managers...

Micro/nano services trend looks optimistic, but I am not sure if it's not double-edged sword, and instead of using complex frameworks developer will be forced to use complex communication system to orchestrate communication between them.

In few years instead of writing code are we going to write configs files?

The answer is simple: Web Applications are becoming Applications in the traditional sense of the word -- and Applications have always been complex. Now we're just moving that complexity on to the web.

But that said, the Webpack + React + Flux model is the simplest kind of programming I've done in a very long while.

Webpack + React + Flux solves so many problems so elegantly.

They have done a phenomenal job of abstracting away complexity behind stable apis.

>Instead of writing code in programing language we move into xml/yaml direction and hundreds of weird config files.

Blame this partially on the weird phenomenon of programmers refusing to touch languages they consider aesthetically ugly. Javascript? CSS? Ick. Much better to pile on a build system to compile those languages from another language that looks kind of Ruby. HTML? Ew - no significant whitespace. Let's generate HTML from something that looks and acts like Python.

But, snark aside, I think it's a good thing if a framework can be agnostic about the kind of configuration syntax it uses, although that in and of itself implies some minimal complexity. Whatever choices the developers make about this, of course, amount to personal preference. Heck, I prefer INI files for configuration.

>Even small website often requires hundreds of megabytes of dependencies

That depends on the language and framework you're using. And how small you consider a "small" website.

Unfortunately (in my opinion, with which I know others will disagree) frameworks tend to want to do too much, and as such tend to bloat into vast, overly complex attempts at reinventing the wheel, the chassis, the transmission, the bumpers and deciding what kind of air freshener you should want hanging from the mirror. I would prefer a framework not have a templating library or ORM or code generators baked in at all. Those should be implementation specific decisions. Rather, a framework should manage dependencies, routing, global config, responses, have some hooks to allow extensibility - the basics. If I want more, I can pick libraries for that.

The problem with JS is not that is it aesthetically ugly, but the lack of a decent type system. The fact you can add whatever data you like to any object, and the object itself could be the data or a promise to return the data, and you cannot verify that the caller and callee are compatiable at compile time is asking for so much trouble. Let alone the nasty implicit type casting, two types of null value, truthy values etc. It is a real mess for the programmer who likes to reason about their code.

It is really difficult to explain the benefits of static analysis and type safety to people who only work in dynamically typed languages.

The JS community is just now being introduced to compilers, and there is significant push back and lots of people crying for "simpler age". I think it will be a while (if ever) until languages like closurescript, scalajs, & typescript start seeing widespread adoption outside of their fiefdoms (google, fb, microsoft).

If you are interested in a typesafe universal js ( or isomorphic js, whatever we are calling it now ) proof of concept you can check out this project using scalajs and scala-js react.


It depends what you are doing. If you have 10 lines of JS then probably no need for an abstraction. However if you have 1000s of lines of client code, then using a purescript or similar can be very beneficial to maintainability and provability of the code.

But there is a learning curve and unlike Haskell where you just know have to know the language, you have to know both the JS quirks and the compiler's quirks too. It is a leaky abstraction. This is the problem I have with a lot of the compile-to-android frameworks vs. just putting with with Java.

Innovation always leads to change. The fact that framework complexity has been increasing is only incidental.

In fact, I think we're starting to see a trend in the other direction. Angular 1 was a massive mess of complexity, but Angular 2, React-based frameworks, and various other frameworks coming out these days tend to be much simpler.

Believe it or not, the web has been the platform where most of the innovation in UI development paradigms has been taking place in the past few years. And I don't just mean in terms of the sheer number of new things coming out.

The web community pioneered UI development paradigms involving functional programming techniques and immutable data, and have brought its benefits to other platforms (through React Native). I personally would never want to go back to the old way of building UIs. And I suspect I'll feel the same way about client-centric data fetching when I get to try out Relay/GraphQL, Falcor, Om Next, etc in production.

If constant change is the price we have to pay for innovation at such a relentless pace, I personally welcome it with open arms.

    > The web community pioneered UI development paradigms involving
    > functional programming techniques and immutable data
Ugh, No. https://raw.githubusercontent.com/papers-we-love/papers-we-l... and https://www.youtube.com/watch?v=Agu6jipKfYw

I'm honestly not aware of any UI development platform that makes heavy use of functional techniques and immutable data outside of the web. Sure, papers have been written about the idea, but where else has it been used in practice?

RE: the video. I'm almost certain you know this, but Elm is a part of the web community.

Notable examples on the desktop: GitHub for Mac (ReactiveCocca) and GitHub for Windows (ReactiveUI)

- https://www.youtube.com/watch?v=1XNATGjqM6U and https://www.youtube.com/watch?v=HPyKHxy7X0w and http://web.archive.org/web/20120714082000/http://www.reactiv...

(True Elm is part of the web-community, just saying React was not the first mover but I agree it will certainly be most likely the one that makes FRP cross the chasm)

I appreciate your resources, namely the video by the creator of Elm, https://www.youtube.com/watch?v=Agu6jipKfYw.

However, I agree with above poster. I remember developing my first windows app by doing a main event loop. It's been that way for a long time. Only recently are we doing more asychronous callback / bound type mechanisms of reaction. And still not as strong as the FLP stuff mentioned in the video. Not as much functional stuff, just functional paradigms.

I personally really like the perfection of static input/output transformation graphs. But yeah..it's not been that way for a while and I agree with OP, the web being so event/input/signal based, has spurred the movement for better ways of reactive programming.

> web applications are becoming more and more complex

The way I see it, it's becoming complex the more you learn. For instance, the first time one learns a programming language, one learns the language and steers clear of frameworks. The deeper you go, you learn tricks and stuff "which this library does" or "this framework does". Eventually you get exposed to the stuff that does more stuff in less work, away from the programming and more on configuring.

> Even small website often requires hundreds of megabytes of dependencies.

You wouldn't want to write those from scratch.

> Sometimes one application can have 2 different build systems

One tends to use the build system in the language nearest to the platform. I still have yet to see a person write Grunt on a Java project or Maven on a JS project.

> Micro/nano services trend looks optimistic, but I am not sure if it's not double-edged sword

It is double edged. More control but you need to set configurations, set conventions. Do it wrong, and it's easily spaghetti.

> In few years instead of writing code are we going to write configs files?

For all that configuration, your are configuring something. That something is written by someone. So while you're configuring here, someone is programming somewhere.

We've come full circle and have just pulled our stack back from a complex framework (Durandal, akin to Angular) to a simple architecture centred around knockout.js, and everyone is happier for it. The code is cleaner and debugging is a world apart.

Backend wise I've taken a liking to Express because it hides nothing. I prefer it over Rails or ASP.NET MVC, where there is so much magic. I would switch if I could but we have too much legacy code.

The theory is that a DSL should be better than whatever generalised programming language you're using. SQL is a good example - ORMs tend to fuck up querying databases (ie pulling back unnecessary data) almost as much as they fix. SQL, a language designed for querying relational data, is (unsurprisingly) pretty good at doing what you need efficiently.

In practice, a lot of frameworks are made by people (who are just people, and don't 100% know what they're doing at the outset). I'd say that if you're spending too much time in configuration, you're not using the right frameworks.

No, in a few years those frameworks will be forgotten. From the terms that you've used, it sounds as though you're coming from the Java world, which has an inexplicable love of config files - if that's the case, it might be a bit longer than a few years, but they'll eventually figure out that config files aren't a good representation for options.

Though the layers of magic and abstraction will continue to increase and grow, at least until simple LOB usecases can be achieved in a couple of lines at most.

It's annoyingly bad at this point. You know it's gotten bad whenever new frameworks are coming out where being "minimal" is a feature.

Page loads are very large on some sites now, where they don't have to be. Though, advertising can be blamed for a lot of that. There's just a lot of extra cruft out there now, without much added to the user experience, in fact, it often detracts from it.

What happened to using only what is necessary? Why throw so much unused stuff behind a site, and especially, why load anything you're not using? Why complicate things?

Maybe I'm nostalgic for static pages or something, but I miss the days where things were simple in terms of page loads and efficient design, and I think a technologically-advanced but focused (avoiding excessiveness) approach is best going forward.

This philosophical approach is one reason I like the Hacker News site itself. It accomplishes its task well, and is almost solely text and minimal css. It's running on some stuff built underneath (as dynamic pages require), but it seems pretty minimal in general. It needs nothing extra.

Websites are becoming more complex. Deal with it.

I'm sure people said the same thing about the gutenburg press vs locking a bunch of monks in a room for 20 years.

That being said, if you don't understand the tech, or the tooling that implements it, don't use it. Keep doing things the hard way, I really don't care.

In a few years we will be writing config files for machine learning algorithms to stitch together a ui based on requirements doc, or an api specification. You can already see this within AWS API gateway and lambda functions, and a lot of the R&D being done around procedurally generated ui's.

The developers that fail to adapt will be left building sites using legacy tooling for shit clients who don't know better. Or move into management where they can impose their archaic views on their subordinates who do know better.

Either way, no one is forcing you to write code you don't understand, that is 100% your fault.

I look forward to the day that I can automate away the need for sub par developers polluting codebases with their versions of "simplicity". I am currently re-writing a 100k line js app for a fortune 500 company, the initial contractors were trying to keep things "simple" by using 10 year old tech. They all got fired 2 days after my proof of concept demo.

I was with you initially, but this seems hostile in a way that's honestly kind of confusing. The argument, as I understand it, isn't something like "websites are too complex now," it's that "web-app programming is becoming more complex without corresponding returns on functionality or UX."

Your example, though it's buried under a bunch of stuff calling people dumbasses, makes an interesting point: You're right, there isn't anything simple about 100,000 lines of code in ANY language, even if it's 100,000 necessary lines. What were they trying to do that was so easy to replace? What kind of JS was knocking around in 2005 -- is it all display stuff and wonky jQuery?

If I seemed hostile, it is because I am dealing with this exact topic on almost a per contract basis, and it is becoming exhausting. However I don't recall calling anyone a dumbass.

Here are some ( sanitized ) examples from my current project that exemplify the danger of allowing this line of thinking among your contractors:

1. Completely unmanaged client dependencies on a per page basis ( negligible code, infinite heartache )

2. Template files that do not reuse code due to laziness or differences in markup style ( 20k+ extra lines of code )

3. CSS files crafted on a per page basis, again little code reuse due to developer style and competence ( 20k+ extra lines of code, huge download size, difficult cacheing requirements )

4. Confusing and undocumented deployment procedures ( I had to completely rebuild the infrastructure from the ground up )

5. Each and every page had a its own js file, with its own global exports, dependencies, and code style. Most were over 1000 lines of untested jQuery Stackoverflow answers. 65 unique pages, sharing almost no code.

6 My favorite, a grunt task that searched the code for 'some_var + other_var' and replaced it with a string literal "https://my-dynamically-generated-cdn-fingerprint.cdn.com/res...

7. Superflous build tasks and bugs due to inconsistent project structure. Some people like /dist others /builds etc.

The list goes on, I'm finding more stuff every day.

This is a well funded project, 1 year into development.

Just implementing react, gulp, and sass allowed me to remove 60k lines of code over 2 months. The app still works in a request / response method, haven't gotten to that yet. The backend is still a tangle of coffeescript.

This is the risk you run by not imposing structure on your developers via solid frameworks and tools. If you can't pick the best tool for the job, hire someone who can. Otherwise you'll end up hiring him anyways to fix your broken 20 million dollar "unicorn".

This is really interesting, thanks for responding. It seems, though, like the vast majority of these problems aren't coming from the necessary evils brought about by outdated technologies -- it just seems like inexperienced (or just...bad) coders trying to shove stuff out the door on the cheap. If the people who made the original project had been "using" React, I'd imagine you'd just be dealing with the busted CSS and mangled JSX everywhere.

This raises an interest point that I think I was missing before -- "imposing structure on your developers via solid frameworks and tools." The idea that even if you don't need the functionality of a particular framework, a framework's prescriptive way of doing things might reduce enough of the variance in styles to make a big project more coherent.

That's just the thing.

Having a framework in place minimizes the impact of those "bad programmers" to just a single ui component, or api route, or database migration.

Not having a framework in place means that that intern or contractor from india can now introduce bugs that impact the entire system and are nearly impossible to track down.

I feel like it shouldn't be such a hard sell to get software engineers to admit that structure and consistency are good things.

"imposing structure on your developers via solid frameworks and tools." is exactly why i use frameworks.

We initially may have seemed to have different ideas of "simplicity", because I certainly agree that if using a framework works for a site, and allows you to re-use code to avoid extra lines (like the extra CSS, jQuery, etc. you mention) then that is the best way to go.

When I said simple, maybe I meant "efficient" instead. I just see a lot of misuse of frameworks where all of the code is thrown in and then barely customized to make it work, and all the extra unused cruft is left in. Basically pages are loading a bunch of stuff that is never actually used...

So I think maybe we were speaking past each other a bit, because I generally agree with you.

The initial post at first seemed a bit hostile, but some of that I think we're inferring- because we're reading our own tone into the text. Also, as you said, it's something you constantly deal with, so I'm sure it's a big point of frustration for you.

When frameworks work to allow you to clean up a codebase or prevent it from becoming the mess you describe, then they definitely make sense. When they cause bloat for no good reason- that's when I have a problem with them.

I guess it all comes down to developer use/misuse of them, not the frameworks themselves, in most cases.

I get where you're coming from - when the initial project for the framework you're working with has so much stuff that you don't understand or need yet, it feels like a huge burden to do the simplest thing in it.

You might want to give Ruby Sinatra a whirl. I just helped a friend build a starter site in it. The Hello World for it is 1 file with 4 lines of Ruby, no config files at all. Save that 1 file, run it, and you have a live website.

What I think really makes it cool is that there is a decent amount of sophistication to it, but you only see it when you need it. It doesn't force any particular way of doing things onto you. It's also kind of a cousin of Rails, so if your needs get to that level, you should be able to switch over pretty easily and make use of the enormous ecosystem of gems.

I think express.js on Node has kind of the same idea, might be worth a look also.

"It doesn't force any particular way of doing things onto you."

Might as well be

"Your app is spaghetti 1 year after you leave the company."

There's some truth to that. I think it's all about knowing the right tools for the job. If you spend several dev-years on a Sinatra app, then you probably will have a unmaintainable mess unless you have some good architects enforcing a sensible structure on it.

Sticking with Ruby for examples, if you want to put up a quickie site with 4 routes and no DB, then it seems kinda silly to pull in Rails and deal with all of the complexity and expectations. Throw up a Sinatra site and be done with it.

On the other hand, if you're getting more demand for it and now you want more logic and analytics and a DB after all and some user accounts etc, then you should know when it's time to move over to Rails.

I've found that it's all about the team.

If it is a small team of close knit developers who communicate well, and expect to work together for a long time. Roll your own setup, loads easier and much more flexible.

If you are a corporation with reasonable dev turnover and good management, you might be able to pull off your own solution too.

If you are using contractors, have high dev turnover, a distributed team, and especially all of the above. Use a framework, any framework, otherwise you will wind up with a disaster within 6 months.

There was a really great talk a couple years ago about Twitter's defunct FlightJS framework. The framework was meh, but something in the presentation stuck with me, "If the shit is in a box, no one can smell it.".

I took that to mean that if your application is structured and organized ( like what a framework imposes ), suboptimal parts of the application will have a smaller impact on the app as a whole, and refactoring those parts is often a matter of just "throwing out the box".

On the front-end React has done a fantastic job of providing lots of boxes for contractors to shit in.

I went to a simple website developed probably in the 90's with a simplistic comment board that let anyone submit a comment and specify their name. It was one of the best web experiences I had this week. Loaded up quickly and didn't have any useless animation. Did its job of letting users communicate with each other. No complex web framework needed.

I think complex web frameworks are developed by expert coders to make their lives easier, supposedly. With little concern for other less-expert developers. But it just turns out to be so complex because they are trying to DRY the shit out of everything.

When I realized that exposing configuration options is not DRY I was able to break away from most frameworks. Within the scope of your own project it's faster to change code, and once you do that you can start to customize it further.

That's the point, these days it's even to start without IDE supporting code completion because of tons of vendors.

This sort of "complexity" (DI, ORM, extensive configuration) has existed for a long time, particularly in the Java webapp space.

You don't need a license to release a framework, and a framework doesn't have to be sensical to be adopted.

If you learn the fundamentals of the things you're working with, particularly HTTP, web browsers, the web in general, and your chosen programming languages, things will become much more clear.

It's not that these things are not clear for me, I build many successful applications using these technologies. Also in the past I was really happy using them, but lately I started to realise that it's usually just increase the time we need to spend building applications. I can't see what really I get in return. Especially debugging this kind of application is also hard, as we can not just set breakpoints in yml file.

> I started to realise that it's usually just increase the time we need to spend building applications.

This might be because I'm not building anything that's going to blow the doors off of the internet, but I've found having at least a casual familiarity with a few frameworks (Django is the first that pops into my head, though you may be talking about .JS stuff) has made prototyping, at least, far simpler. Building a basic Python CRUD app from scratch, depending on what you're trying to do, could take, I'd guess, two weeks to get something* up and running. With Django, you could absolutely have your basic front-end and admin interface done in a day.

Once you get into the tricky bits (or just bumping against the edges of the Django database abstraction layer), yeah, I can see how it's annoying -- is that more what you're talking about with the "we can not just set breakpoints in yml file" bit? That setting up models by hand is more predictable than letting the framework do it?

I think an important part of this question is who is making you use such frameworks in the first place?

Usually when I join company there is some big framework there, and bunch of people who jerk off to the new hype framework in given language. Companies believes it's best solution, because when they send people to conferences they see bunch of other people jerking to the same frameworks. I don't like on small projects with 1-2 developers and these days its the only solution to have normal environment.

Writing config files or filling some template classes instead of actual programming is pretty much the purpose of a framework. As a senior developer it may be your responsability to filter the hype from the real stuff as you can influence which technologies should be used in your projects. Fortunately there is a parallel universe of people developing wonderful libraries that you can combine and use as you wish.

Arguably it's because websites (or more to the point web applications) are becoming more complex

In my experience, the widely-used ones tend to be fairly sane for the most part, and can often be used without having to delve too much into the complexity unless you need to - which I think is the important bit

Complicated frameworks are pain in the ass, I fully understand but there is a value in following some "market standard" frameworks as they make the team speaks the same language and will make it easier to bring developers in at any time without long learning curve.

Pick frameworks that solve problems you actually have and understand.

Your gripe isn't about the tooling, but developers ability to learn.

Because every new framework is trying to outdo all the previous frameworks, and they figure that by adding one, two or more feature is the key. You don't have to consume all the new stuff. You can always choose to stay lean.

I feel like most of the conversations around this topic boil down to:

"I don't want to have to learn new stuff."

GTFO of this industry then.

Yep, especially when they put no effort into understanding the design decisions behind different frameworks and glibly dismiss new ones as "hype" and "jerking off".

"Building blogs with wordpress is easy, so building facebook should be easy too."

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact