
We Should Ditch Nginx - jesalas
http://www.hipyoungstartup.com/2013/11/we-should-ditch-nginx/
======
dknecht
CloudFlare generates 50gb/s of logs globally and have handled collecting this
volume in two ways. Historically the logs are sent to a local syslog-ng
through the use of a PIPE and the forwarded to central logger. This can be
done with nginx with no patches by just treating the PIPE as file. Just make
sure you do a little buffering inside nginx.

access_log /dev/nginx_access log_format_name buffer=64k flush=10s;

Since this is a pipe there is still some blocking IO, but no worse off then
writing to local file.

The way we are migrating will be non-blocking IO through the use of Lua Resty
log module ( [https://github.com/cloudflare/lua-resty-logger-
socket](https://github.com/cloudflare/lua-resty-logger-socket) ).

We will write up a blog post at some point but hopefully this is useful to
you. In the future we are going to be going one step further and have NGINX
emit protobuff files. Feel free to email dane AT cloudflare.com if you have
any questions.

~~~
chrissnell
Apologies everyone for the somewhat o/t question here but, what do you guys do
with your 50Gb of logs every second? Where do they go after they leave nginx?

~~~
sbierwagen
To the NSA, presumably: [http://exiledonline.com/read-yasha-levines-
introduction-to-t...](http://exiledonline.com/read-yasha-levines-introduction-
to-the-crypto-spy-service-cloudflare-isucker-big-brother-internet-culture/)

~~~
eastdakota
To be 100% clear: we have never been asked or ordered to share log data with
the NSA. We've not participated in any program like PRISM and would fight
vigorously if we ever were. We do receive law enforcement requests on
occasion, typically to determine who owns or hosts a site behind our network.
When we've received law enforcement requests for customer data (e.g., account
information like the email address of an account) which we determine are
abusive or do not follow the principles of Due Process we have and will
continue to go to court to fight for the rights of our users. Whenever
possible, even if the legal request meets our standards, we also notify
customers of legal requests and allow them to challenge the requests
themselves before turning over any data. We take this extremely seriously and
spend significant technical, legal, and public policy resources to ensure law
enforcement's job is neither easier nor harder by the mere existence of
CloudFlare.

~~~
sbierwagen
Sure, but why should I have any reason to believe you?

If you had been served with a national security letter, you would be obligated
to lie about it.

Plus, you have a pretty significant financial incentive to lie about how great
Cloudflare is, with no downside, since you're not under oath on HN. And even
if you were, officials who have lied about the extent of surveillance programs
while under oath haven't been prosecuted.

Representatives of Facebook lied. Ditto with Google. Etc, etc, etc.

~~~
true_religion
No. National security letters can act as gag orders, obligating you not to
talk about their existence but you're never required to _lie_ and say you have
never been served a letter.

In that case, you can be cagy (as Google has I believe) and say something on
the lines of "I can neither confirm nor deny receiving a NS letter" which of
course is double speak for "I have one, and I can't talk about it directly".

------
jacques_chester
This blog post demonstrates why the syslog feature is ideal for segmenting
Nginx's market.

Almost nobody cares about it, except the sort of place doing > tens of
millions of requests per day. The sort of place that might have some money and
the scale to realise that a few thousand bucks is cheaper than a bunch of
engineer time.

It's a good case study in smart pricing.

~~~
venus
"NGINX Plus Standard" costs $1,350 annual subscription per server. It sounds
like this company has a pretty strange setup, running "hundreds of VMs",
presumably all running nginx, and it's the combined impact of those VM logging
writes that are so burdening the NetApp devices.

In this case, if they kept the same setup, I can see that it would be
prohibitively expensive to pay for nginx plus on every single one of those
VMs. I'm not sure of the nginx's license terms but if they count VMs as a
separate server - probably a no go.

A better solution might be to turn all those VMs into application servers and
put a big meaty nginx reverse proxy in front of them; that sounds like it
would reduce the nginx bill but could be an awful lot of work. Then again, I
presume they have automation, you'd hope so with hundreds of VMs...

My point is that it's likely not quite as simple as a grand a year, which
would likely be a no-brainer if it was a site license.

~~~
jacques_chester
I am not disagreeing that in this _specific_ case that the licence structure
might lead to unhappy arithmetic. I do however stand by my original point that
syslog support is an excellent cleavage point to segment the customer base.

~~~
venus
Yeah, I do agree with that.

I'm slightly less happy about the HTTP live streaming being in the Plus
package, though. I've been playing around a lot with a media app using HLS and
I can't even trial using nginx for it - not that I'd be too happy paying $1300
a year since I don't have a single user yet. I can see how they are trying to
get money out of the big streaming companies, but it's not too friendly to the
little guy.

~~~
jacques_chester
Segmentation is a lossy function, unfortunately. If you try too hard to fit
the price/feature curve by creating too many segments, you wind up with
nonsense like Windows Home Professional Premium Plus Ultimate Edition.

------
secstate
Sigh. FOSS and NGINX as an example of FOSS have given us so much. If direct to
syslog logging is what you're after, why not use Apache? As many people have
pointed out before, Apache can certainly be hold a candle to NGINX if you put
the time into tuning it. If you're expecting Black Friday traffic of this
magnitude, and you haven't spent the last two months planning and load testing
for it, you are either lazy or underpaid. If the first, okay, I get why you
don't want to roll your own RPM. If the second, that's just what this business
can expect out of their setup.

No one is going to dump NGINX over this. If it is this mission critical to
your business, pony up and pay for NGINX.

> We’re using php-fpm anyway; the performance difference between nginx and
> httpd in this scenario is negligible.

So why are we even having this discussion? Honestly if you're experiencing
that much traffic and PHP performance is hindering you, you should consider
some sort of Varnish full-page caching anyhow. Disk writes, as you note, are
not your primary issue, and certainly is no reason to dump NGINX.

~~~
ars_technician
>No one is going to dump NGINX over this.

His post seems to contradict that. People will dump products for very trivial
reasons if marketed correctly.

~~~
Semiapies
Reading this post, I'd assume that this time next year, if the company is
still there and this guy still has this job, they'll still be using nginx.
Someone dropping this sort of frustrated blog post about how his company
_should_ dump it probably doesn't have the authority to make that call or the
motivation and time to make it happen if he does.

The whole tone of the blog post is passive-aggressive, whether directed at the
makers of nginx (for wanting to get paid), or at the end, towards some
hypothetical people who take it upon themselves to build a fork that suits all
his needs (despite his not actually needing such a fork). Such a tone suggests
that he feels powerless.

------
danenania
Why is this getting upvoted? We should ditch Nginx because they are trying to
build a sustainable business on top of their amazing OSS contribution?

When quality OSS projects like Nginx turn into profitable businesses, that's
very good for OSS as a whole. We should be cheering them and _gasp_ paying
them if we need their high end features, not abandoning them.

~~~
GhotiFish
sensible right up until they start rejecting patches for needed features
because they conflict with commercial goals.

There's a legitimate conflict of interest here.

~~~
fsniper
No you can always fork. That's a business they have the right to accept or
reject patches however they like.

~~~
makomk
Yeah, and other businesses have the right to conclude that they'd rather use a
full-featured open source web server than one that's crippled to create a
market for the expensive proprietary version.

Of course, some people in this discussion seem to disagree - I've seen a lot
of people acting as though it's somehow unfair to nginx for him to point this
out to other potential nginx users or to switch to a different server rather
thay paying up. It's like people here think the nginx developers have some
right to bait people in with the open source version and then charge vast sums
for basic features, and that anyone who isn't onboard with this scheme is
greedy.

~~~
fsniper
I am not against someone deciding to ditch some software in any means they
believe fit. But thinking any company should go on development according to
other party's demands and otherwise asking third parties to ditch the software
is nonsense.

~~~
Shamanmuni
As the article states, a patch exists which implements the functionality
required by the author, so the only 'demand' is to apply that patch, hardly a
monumental task.

The problem here is that Nginx developers refuse to implement a free patch
which already exists for a feature easily found in the competition in order to
protect their business model.

Sure, what Nginx devs do is a legitimate practice, as legitimate as the author
complaining about it and proposing a change of software or a fork. I don't
understand why many get so upset about it.

------
rdtsc
Ok, fight fire with fire (inflammatory post with inflammatory response) here
it goes:

Scumbag Hipster Young Startup -- with a 3 floor building (mentioned CEO coming
down all the way from the 3rd floor), aggregating 50M events related to sales
every day, taking advantage of hundreds of thousands of volunteered man hours
put into the FOSS stack they are using, are throwing a hissy fit about having
to pay the creators of the software they are using for an enterprise feature.

~~~
ars_technician
Is this money going to the external contributors as well? Volunteers that
submitted patches, tests, and documentation?

~~~
lsaferite
Arguably you are paying for a license that covers support and the ADDITIONAL
functionality. Everyone still has free access to the core code that people
contributed towards.

If they were to take contributed code and lock it behind a commercial license,
THEN you could get all uppity.

~~~
redblacktree
Others are pointing out that they routinely reject these features in the FOSS
offering, citing a conflict with the enterprise edition. That's the worst part
of this, imo.

~~~
lsaferite
I don't disagree, but my point is that the code people freely contributed
prior to this change is still freely available to the public.

If they are rejecting patches for features that impinge on their enterprise
offerings and you have an issue with how the project is run now, you are
welcome to not contribute, fork, or use another project.

~~~
ars_technician
>If they are rejecting patches for features that impinge on their enterprise
offerings and you have an issue with how the project is run now, you are
welcome to not contribute, fork, or use another project.

Which is precisely what is being argued.

~~~
lsaferite
No, you were insinuating they had an obligation to provide monetary
compensation to the people who contributed code to the open source product
when they sold licenses to the closed source product.

------
davidu
I'm curious what business model the author of the post thinks would be
possible for the author of nginx and not result in this scenario for feature
x/y/z?

Also, as an aside, it seems like the author of the post pays for NetApp and
the very, very expensive support contracts that come with it, nginx contracts
are a walk in the park, financially, in comparison.

~~~
rodgerd
> I'm curious what business model the author of the post thinks would be
> possible for the author of nginx and not result in this scenario for feature
> x/y/z?

The author presumably belongs to the same cohort who download films and music
without paying and sneers "how you feed yourself isn't my problem, go sell
some t-shirts or something". I doubt the author gives a shit, they just want
stuff for free, so they can continue to make money with it.

------
yeukhon
> Who will pick up the mantle and actively develop the next game-changing FOSS
> web server?

We are writing and waiting. I don't mean to be harsh - but if you were to
suggest people to ditch nginx, take the initiative and fork it and start doing
the support. It's one of those "rants" we hear and nobody does anything.

> Getting these NGINX log events into a remote server is easy with rsyslog.
> But preventing the log events from writing to disk in the first place?

And what is the problem with writing to disk? I guess RAM disk is fine? I am
not really sure what's the issue with writing to disk. Most of the time it's
the application and securing disk more important.

I still don't see any convincing argument why we should ditch Nginx. To me all
the points so far is about Nginx going commercial.

~~~
ars_technician
>I don't mean to be harsh - but if you were to suggest people to ditch nginx,
take the initiative and fork it and start doing the support.

Perhaps he doesn't have the hours/skills free to do so. People are free to
complain about things or do a 'call-to-arms' about things they can't fix
themselves.

>And what is the problem with writing to disk?

Performance

>I guess RAM disk is fine?

It works, but it's a hack and major technical debt to hold.

> I am not really sure what's the issue with writing to disk.

Performance. Specifically the thundering herd problem. All of his servers log
to virtual disks all stored on the same SAN, which is stressing under the
load.

>I still don't see any convincing argument why we should ditch Nginx.

It's a bit of a slippery slope fallacy, but the point is that a basic feature
is being held hostage behind a pay-wall. It would be like MySQL requiring you
to pay to any users other than the 'admin' SQL account.

~~~
grey-area
_Perhaps he doesn 't have the hours/skills free to do so. People are free to
complain about things or do a 'call-to-arms' about things they can't fix
themselves._

I think in the case of open source we have far too many people who feel
entitled to the work of others without any recompense already. Working on a
large OS project like this is mostly a thankless task - people will complain
about the problems while taking for granted all the features that just work,
and they're unlikely to be raking in lots of money because of attitudes like
this.

Given the volume of sales, the 3 floor building and the use of NetApp, I would
have thought this company has money to pay nginx for what is a core piece of
software, even if they have to negotiate a special license or change their
setup. If they don't want to or can't, they can use apache, find workarounds,
or even patch nginx themselves.

Issuing a call to arms over this is pretty obnoxious behaviour, because it
implies the nginx people have done something wrong or antisocial in wanting to
be paid for _some_ of their work. How outrageous!

 _the point is that a basic feature is being held hostage behind a pay-wall.
It would be like MySQL requiring you to pay to any users other than the
'admin' SQL account._

Yes, it would be like that (though this feature is more trivial and could be
worked around, and is not an existing feature, so not exactly the same).
That's the way companies make money when they segment their market and have an
OS offering and a commercial one. Nothing wrong with that. If you don't like
the rules set by the creators, don't use it; write your own software or use
other software. If they had retrospectively retired features and put them into
the paid version, perhaps he'd have a point, but as far as I'm aware they
haven't.

He's not entitled to have the nginx guys work for free forever on his terms on
their software. Ending with the exhortation 'Fork it' sums up his position
perfectly - _someone else_ should fork this software, add the features I need,
and then continue to work for free for me. He's asking 'Who will pick up the
mantle', because clearly it won't be him; he's not a sucker after all.

~~~
ars_technician
>Yes, it would be like that (though this feature is more trivial and could be
worked around, and is not an existing feature, so not exactly the same).
That's the way companies make money when they segment their market and have an
OS offering and a commercial one. Nothing wrong with that. If you don't like
the rules set by the creators, don't use it; write your own software or use
other software. If they had retrospectively retired features and put them into
the paid version, perhaps he'd have a point, but as far as I'm aware they
haven't.

But what they have done is made a pretty basic marketing mistake. They decided
to charge for something that has been 'solved' by apache and hundreds of other
pieces of software many years ago. People don't want to pay for something
that's not new in the field.

>Issuing a call to arms over this is pretty obnoxious behaviour, because it
implies the nginx people have done something wrong or antisocial in wanting to
be paid for some of their work. How outrageous!

You say that like all approaches to making money are the same and it's
disingenuous. How about some document editing software that works for free for
30 days and then encrypts all of your files until you pay up? The developers
just want to get paid, right? How outrageous!

>If you don't like the rules set by the creators, don't use it; write your own
software or use other software.

That's what he's advocating...

>Ending with the exhortation 'Fork it' sums up his position perfectly -
someone else should fork this software, add the features I need, and then
continue to work for free for me. He's asking 'Who will pick up the mantle',
because clearly it won't be him; he's not a sucker after all.

That's exactly what good open source foundations do. I take it you are
unfamiliar with the apache web server project? What about OpenStack? Maybe
Linux or Firefox? None of these projects turn to their users to squeeze money
out of them to activate basic features.

~~~
nemothekid
>What about OpenStack? Maybe Linux or Firefox? None of these projects turn to
their users to squeeze money out of them to activate basic features.

Thats disingenuous. Linux squeezes money out of users via RHEL and SUSE.
Firefox straight up sells your data to Google (who in turn provides over 90%
of Mozilla revenue).

Even Apache, a "good" open source foundation, houses Cassandra, which is
largely contributed to by DataStax, which sells its own version of Cassandra
which has "pay-only" features such as Hadoop integration without the need of
HDFS.

Long story short, there are large number of OSS that function the way nginx is
running things right now. You could probably throw a pebble in the
OSS/Enterprise ocean and land on a project that has "pay-only" features.
Unless you are willing write the solution yourself, no one in obliged to take
up the mantle and fix your problems for you.

~~~
ars_technician
> Linux squeezes money out of users via RHEL and SUSE.

Linux is not RHEL and SUSE. You can use Linux without using RHEL and SUSE and
still get all of the Linux kernel features, which is the entire point. That's
exactly what makes it a good foundation. They aren't arbitrarily screwing
their users out of features. There is no such thing as an 'enterprise-plus
Linux Kernel'.

> Firefox straight up sells your data to Google (who in turn provides over 90%
> of Mozilla revenue).

This is a blatant lie that shows nothing more than your ability to troll. If
you can't handle a specific search provider getting your data when you search,
change your search provider.

>Even Apache, a "good" open source foundation, houses Cassandra, which is
largely contributed to by DataStax, which sells its own version of Cassandra

It doesn't matter, you seem to be failing with basic logic. Cassandra may be
contributed to by DataStax, but DataStax doesn't control the gate so they
can't stop contributions that overlap with their product's functionality.
nginx can and does stop patches that duplicate 'enterprise functionality'. See
the problem?

~~~
nemothekid
>This is a blatant lie that shows nothing more than your ability to troll.

Source: [http://thenextweb.com/insider/2013/11/21/mozillas-
reliance-g...](http://thenextweb.com/insider/2013/11/21/mozillas-reliance-
google-increasing-90-2012-revenue-came-one-source/')

Maybe I should have provided a source, but calling something that is easily
google-able and was on the front page on HN a "blatant lie" is worrying. Why
would I lie on the internet?

>Cassandra may be contributed to by DataStax, but DataStax doesn't control the
gate so they can't stop contributions that overlap with their product's
functionality.

Again, you should research your claims before assuming I'm lying. The project
chair for Apache Cassandra is also the Co founder of DataStax - so yes,
DataStax does control the gate.

------
Mikeb85
Then ditch it and move on. If everyone made a post every time they chose not
to buy X product but instead to use Y product then HN would be unreadable.

You have the source, patch/fork it. If you're too lazy/not skilled enough,
switch to Apache and don't tell us about it.

~~~
twelve40
Although I share your sentiment, noc and devops people are legitimately
paranoid about stability, so no, he's neither lazy nor unskilled - this part I
can easily believe. What I can't understand is, if they have tons of traffic
and he admittedly uses a product he likes, why doesn't he just pay for it.
FOSS does not enslave its authors to perpetually provide free infinite
scalability for every use case.

~~~
Mikeb85
Very true. Forgot to mention that part. Why not pay for a great product if you
find it useful. I've paid for FOSS stuff before, and supported distros I
use...

~~~
ars_technician
Price restriction in this case. 100 vms * $1300 a VM = a big pile of cash that
wasn't in the budget.

------
Rantenki
It's $1350/server per year to pay for the feature. You mention in your post:
"Our noble HTTP server". Wait. Not plural? I know that a that is a fair bit of
money for a bootstrapped startup, but if you are anticipating enough traffic
that the syslog feature is an issue, AND you have at least three floors of
offices (as mentioned in your blog post), then maybe your OPEX prioritization
is a bit off, and you should just pony up.

Nginx is a great piece of software for $no dollars, maybe it would be good
karma to pay for the extras you need?

~~~
ars_technician
Yes, $1350/server per year * 100 VMs which just broker requests from clients
to php-fpm. Totally reasonable. /s

For that price, they could pay someone who's sole job is to maintain a private
fork with the feature. It's not like they can submit it back since nginx would
reject an enterprise competing patch, but at least they would have the
functionality and the benefit of another dev on hand.

~~~
rhino444
So is that about the pricing or the closed source? Would it be different if
the pricing was something else? Just curious. Also, like quite a few people
pointed out, why not request "the best pricing" given certain deployment
scenario?

------
thejosh
I hate when people whinge about free software...... You didn't pay anything
for it, you use it to make money yet you whinge.

~~~
skj
This position is without merit. People have every right to say that one
particular free option is not what they're looking for, and explain why it's
not what they're looking for.

~~~
secstate
While I agree with your statement on it's face, in the context of the OP, this
is just whinging. The author is not just offering constructive criticism on a
possible fork of NGINX, he's calling the NGINX business out for "withholding"
features. The tone is distinctly one of being "owed" a feature common to
another popular FOSS project, instead of being grateful for everything NGINX
provides (which is a lot, and probably why switching to Apache is suggested
with some reluctance in the OP).

~~~
pjmlp
This part of FOSS culture I really dislike as a developer.

Many people seem to consider an offence that other developers need to make a
living, instead of being grateful of being able to sell stuff using software
they haven't paid a dime for.

Then become outraged when the developers come to the conclusion that the baker
won't sell bread for pull requests.

This is what moved me into the direction of suggesting dual license for open
source projects, every time someone consults me on it.

~~~
ars_technician
What you are suggesting is the part of a greedy FOSS project. One that enjoys
the external open source contributions, yet still charges for features in the
software. It's disgusting and an insult to your contributors, unless you pay
them all money as well.

There is nothing wrong with making money, but don't to it with sleazy tactics
like this that put you in a conflict of interest. Charge for support or a
hosted version, but don't artificially cripple the software. If you do want to
go that way, don't be open source at all because you are just a leech on the
community.

~~~
rhino444
> What you are suggesting is the part of a greedy FOSS project. One that
> enjoys the external open source contributions, yet still charges for
> features in the software. It's disgusting and an insult to your
> contributors, unless you pay them all money as well.

Just for the records:
[http://www.ohloh.net/p/nginx](http://www.ohloh.net/p/nginx)
[http://www.ohloh.net/p/nginx/contributors/summary](http://www.ohloh.net/p/nginx/contributors/summary)

Indeed there are useful patches/bugfixes from the community, however nginx has
always been almost a "one-man operation".

To the best of my knowledge, syslog code in nginx plus isn't based on any 3rd
party work too.

------
DigitalSea
I can't tell if this post is serious or not (the domain name throws me off a
little). I use NGINX myself, but I am also aware that Apache is playing
catchup quite quickly as well and Apache 5.4 is a decent version of the
popular open source server.

Use whatever fits your needs, just because it didn't fit your needs doesn't
mean you should go forcing your opinion upon others. Many people including
myself have had nothing but great experiences using NGINX and I don't actually
see any real argument here to stop using NGINX.

Based on the tone of the article I get the kind of vibe that the author has
the "I liked this band before they became mainstream" mentality about
software. NGINX deserves all of the money they can get, they have a great
product that is worth paying for and it's quite usuable without paying a cent,
so I don't understand what there is to complain about.

~~~
davidw
Apache httpd is at version 2.4.7:

[http://httpd.apache.org/](http://httpd.apache.org/)

~~~
DigitalSea
I had MySQL on the mind obviously. I meant 2.4, thanks for point it out
though!

------
Gigablah
> Fork it.

Well, there's a fork of nginx called Tengine, made by the Taobao folks:

[http://tengine.taobao.org/document/http_log.html](http://tengine.taobao.org/document/http_log.html)

~~~
voltagex_
From a (very) brief glance, it looks like you could almost pull this code back
into the open source version on nginx.
[https://github.com/alibaba/tengine/blob/master/src/http/modu...](https://github.com/alibaba/tengine/blob/master/src/http/modules/ngx_http_log_module.c)

------
gkoberger
Given the name of the blog (and how it's written), I assume this is meant to
be satire?

~~~
trimbo
It is not in terms of the content:

"Logging to syslog is available as part of our commercial subscription only."

[http://nginx.org/en/docs/ngx_core_module.html#error_log](http://nginx.org/en/docs/ngx_core_module.html#error_log)

~~~
gkoberger
"Satire" doesn't mean "false" or "fake".

------
cbsmith
Really, it isn't too hard to patch the logging in nginx and/or have a named
pipe that writes to syslog. My team made a patch to have it log to UDP
multicast, and it wasn't too bad. If you're really pissed at the nginx guys,
contribute the patch back.

Sure, I quibble with their approach to a commercial model, but I really think
this is a case of "me thinks she doth protest too much" in both the literal
and implied meaning.

~~~
donavanm
> Really, it isn't too hard to patch the logging in nginx and/or have a named
> pipe that writes to syslog.

Be careful with high performance expectations on named pipes. The fifo is in
memory, but the inode is on disk. It _really_ sucks to block on stat() and
friends when you dont expect it. Oh! And those FIFOs act as a 64kb buffer.
Need more buffering? Put another pipe on it!

~~~
cbsmith
syslog is also a syscall, and has limited buffering. A named pipe can actually
potentially amortize overhead more effectively. (And just because an inode is
involved does not mean that a call to stat() involves a trip to disk.)

If you are really worried about buffering, you can always have a consumer of
the named pipe with its own buffering before writing to syslog (yes, that can
get ugly).

In a perfect world, I'd actually prefer to write to a ring buffer in shared
memory (one per worker to avoid too much contention) and then have another
process that empties it as fast as it can.

Either way, it's actually not that hard to hack in to nginx. If someone were
to contribute a patch that did logging the "right" way and host it on say...
github, I think you'd find the nginx _commercial_ folks would have a response
that works out pretty well for everyone. The catch is, it requires someone to
do something.

------
mythz
Sigh, another example of the worst thing about OSS - privileged, self-entitled
users thinking the world owes them something.

~~~
ars_technician
They aren't self-entitled if the project has any sense of community and has
accepted external contributions of code/documentation/etc. Lots of external
hours go into projects like this and moves like these are myopic and generally
a big 'screw-you' to active external contributors.

~~~
mythz
External contributors are usually not the problem, since they're intimately
familiar with the project and know just how much time and effort the core team
invests in it to keep it running. Contributors that have had a positive impact
are usually given free commercial licenses, thanking them for their efforts.

The 'screw you contributors' is a common argument used by self-entitled users
to try and shame core maintainers into continuing to devote all their future
efforts, giving away all their future time and IP away for free - just so they
can continue their unfettered use of everything they make, and for them to
continue to provide support, so that end-user commercial products can continue
to benefit from all their future efforts, uncompensated.

------
jesalas
Hi everyone, I certainly didn't expect this strong of a response! I wrote the
blog post as more of a slice-of-tech-life brain dump or request for
comments/ideas. I certainly won't suggest that everyone should ditch NGINX,
but it has possibly run its course for us. As I mentioned, I think that NGINX
is an amazing piece of software. It's actually powering the very site where I
complained about it. I just can't justify recommending licenses for it at work
when we don't do anything with it that Apache httpd couldn't do for free.

You guys are right, The Site is poorly designed and suffers from a lot of
architectural flaws. That's why I have a job in the first place! It's been a
long journey getting to where we are and there's a lot more to do.

Thanks to everyone for the opinions and ideas!

------
adrianlmm
You get so much from NGINX and yout don't want to pay?

The world has gone crazy.

------
dreamdu5t
You can compile nginx with syslog support with this module
[https://github.com/yaoweibin/nginx_syslog_patch](https://github.com/yaoweibin/nginx_syslog_patch)
and avoid writing to disk.

syslog is still blocking though.

------
vertis
I do find it a little amusing that the author doesn't want to pay and doesn't
want to do it himself.

FOSS is two sided, this is the perfect opportunity to contribute.

FWIW, It's really not that hard to build your own RPM/DEB

------
krenoten
If you don't want it to hit disk, why don't you just not log to disk? Are you
running a unix variant? There are several ways to do this. Why not log to
/dev/shm or a pipe, instead of changing nginx?

Rsyslog is also probably not the best choice for high volume log aggregation.
It performs an rpc per message. That sucks. Use something that batches, like
scribe.

~~~
kul_
I agree, If you are aggregating logs so small that they can fit into memory,
most likely that you do not care where it goes. Otoh if the logs are enormous
you must have robust periodic log aggregation jobs like map reduce to do that
and it is must to have logs on disk.

Moreover 50m/day is ~ 578/sec , which isn't huge for a cluster of servers.

------
pnt
Nginx can log to stdout (access_log /dev/stdout) and be piped to a program
that reads from stdin and forwards to rsyslog.

Technical workarounds aside, the blog author should get approval from his
manager/ceo to purchase commercial nginx. I don't see what's unreasonable
about a profitable business paying for a useful commercial feature.

------
oijaf888
Given it seems to be a product of
[http://www.simulantproductions.com/](http://www.simulantproductions.com/) is
this just satire or link bait?

------
ridruejo
Apache 2.4 with event MPM should give you comparable performance to nginx

~~~
DigitalSea
Performance is pretty good in Apache 2.4, but compare RAM usage alongside
NGINX and you'll notice that NGINX's event based threads approach takes a big
dump all over Apache's separate process lets consume every bit of available
RAM the server has until it starts throwing errors and filling log files.
Apache is getting better, but it's not really on par with NGINX (yet, but it
will be).

~~~
gnaritas
You didn't hear what he said, he's specifically not talking about the memory
hogging process bases mpm. You seem to think Apache only works that way, but
the worker modules are pluggable.

~~~
DigitalSea
He's talking about Apache MPM Worker which while offering better performance
over the traditional Apache MPM Prefork because it's threaded based like NGINX
is, it still consumes more RAM than NGINX does, so my point remains...

~~~
nisa
He's talking about MPM Event¹. It's different from worker and probably not as
light as nginx but for most stuff it should compare well.

1:
[https://httpd.apache.org/docs/current/mod/event.html](https://httpd.apache.org/docs/current/mod/event.html)

------
kolev
Nginx Pro is a bit expensive. I haven't looked into their licensing, but
usually with paid software, the biggest issue is activating licenses and
stuff, which can't work nicely with autoscaling. One example is when wanted to
buy the commercial version of s3fs back in the day - it required running some
tool on your server and emailing the guy the hardware signature so that he can
email you a key for that server. I'm not saying this is the case with Nginx,
but in general, the more strict you make your licensing, the less revenue
you're gonna collect. FOSS that charges you for extra features is pretty much
doomed - nobody wants to use the less capable version and subconsciously or
consciously starts looking for alternatives. Charging for support I truly
accept - that's the way to go and it keeps the attitude positive.

------
cassianoaquino
why not pointing access_log/error_log to fifos and use rsyslog/syslog-ng to
read direct from it?

------
wereHamster
Why not log into a pipe where syslog-ng listens on the other end?

------
nso
It's never "impossible" to avoid disk writes. I'm assuming it's running on
Linux, and in that case you could simply log to a shm mount. Since Linux is
file-based, it really doesn't care what medium it stores the file on.

Ofc. you'll have to handle a couple of potential pitfalls, like running out of
shm, but I think this should be much easier than ditching nginx altogether.
Linux was made for hacking, so hack away.

------
ajclark
I wonder if:

A) The author is using a load balancer that can do centralised logging in a
sane fashion perhaps they can turn off end logging on nginx all together.

B) The author has looked in to the Lua nginx scripting capabilities for direct
logging without touching disk.

C) Place a greater emphasis on SaaS logging with javascript on the client
side.

It sounds as if their architecture needs a massive re-think.

------
kh_hk
Maybe this would be steering out of the point, but doesn't something like
collectd suit the problem?

------
neya
Not relating to the Author's opinion, I would just like to know if there's a
legitimate, competent competitor to Nginx apart from Apache? Just curious as
I've already had a bad experience with monopolies in the past.

~~~
jaytaylor
HAProxy[0] has fit the bill for me in many instances.

[0] [http://haproxy.1wt.eu/](http://haproxy.1wt.eu/)

~~~
orthecreedence
Just want to mention that HAProxy isn't a web server like nginx, but a proxy.
So while it competes at some things nginx is commonly used for, its not a full
competitor (you wouldn't replace nginx+php-fpm w/ HAProxy+php-fpm).

That said, if you're looking for a proxy/load balancer, HAProxy is most likely
the absolute best you're going to find anywhere, ever...and the price is right
;)

------
mrmincent
So these guys have one of the biggest days of their year coming up, seem to be
running a large operation, and are only just realising this now. You get what
you pay for, and from the sounds of it they haven't paid much.

------
youngtaff
As an interesting aside…

If someone contributes code for a feature that ends up in the commercial
product, are they stuck with the options of paying the annual per license fee,
or compiling their own code to get the feature?

~~~
voretaq7
Basically, yes.

------
embro
It's open source. If your company lives from the web traffic, hire a coder to
patch it or just pay for it.

There is also other solutions like Cherokee web server and Lighthttpd.

------
happywolf
Given the magnitude of the traffic, I assume the revenue will be proportional.
What to bitch about of paying for something that adds value to one's business?

------
vacri
... so write to a named pipe instead of the disk?

