
My blog can’t keep up: 500 errors all over - nkurz
https://lemire.me/blog/2019/01/31/my-blog-cant-keep-up-500-errors-all-over/
======
thaumaturgy
30 to 50k uniques per month really isn't that much. At a rough guesstimate,
I'd say I've got at least one hosting customer that does that much with
Wordpress, gracefully, on a site that's not optimized even a little. I charge
them $10/mo.

Static site generators and everything else are lovely and all, but it's hard
to beat Wordpress when it comes to being able to just plug in a new feature.
Which, of course, also gives some folks the rope they need to hang themselves.
As other folks have pointed out, there could be a plugin issue.

But my guess is it's the hosting provider. They can have professional-sounding
support but still be running an architecture that hasn't been updated very
much in the last 5+ years. mpm-event and php-fpm alone really changed the
performance characteristics for bog-standard LAMP hosting.

The error logs should give a number if memory's exhausted. I bet it's super
low, because a lot of shared hosting providers still try to cram as many sites
as they can onto a beleaguered colo'd box. At a glance, that does look like
the csoft.net architecture (they have "Two Dedicated, Redundant Servers (with
Private Gigabit LAN)").

There's certainly a broader point to be made about the gross, inexorable bloat
of web-based applications, but there are also plenty of people running even
more bloated, busy stuff for less money with less trouble.

Oh, and you can install the traditional editor for Wordpress, they've made it
available as a free plugin and it works just fine.

There are plenty of people around who can sort out stuff like this. I'm one of
them, but I'm heading to bed. Email's in my profile, or just accept help from
anybody else that knows what they're doing.

------
xupybd
Consider moving away from a CMS like Wordpress. It's great at what it does but
I think you've out grown it. Static content generators might help here. They
let you edit off line and compile to a static version of the web content you
currently serve. I'd suggest looking at Hugo or one of the many specialised
for blogging. [https://opensource.com/article/18/3/start-blog-30-minutes-
hu...](https://opensource.com/article/18/3/start-blog-30-minutes-hugo)

You can then push the content to something Amazon's s3 and put cloudfront in
front to speed things up.

You'll have to then offload the comments to a service but that is relatively
easy and cheap. A service like discus is only $9 a month. Moving to s3 you'll
save that in hosting. (I run two sites like this and pay $0.07 on average per
month). Not to mention those services are pretty good at filtering spam.

There are some other options around too. [https://darekkay.com/blog/static-
site-comments/](https://darekkay.com/blog/static-site-comments/)

Wordpress is nice but it will never out perform static content.

EDIT: I just read this in discus "Free For small, personal, non-commercial
sites who do not run any ads. This plan includes everything in Plus except
Direct Email Support."

~~~
meritt
> It's great at what it does but I think you've out grown it.

There is absolutely no justification for him to be running into these sort of
issues at such an incredibly low load. Let's pretend he gets 50,000 hits per
_day_ (he says 50k per month but I don't know reqs/sec). Than's less than one
per second. The $5/mo box over at DigitalOcean can handle 100 times that with
the most poorly optimized PHP, Wordpress, and MySQL installation.

The problem is entirely due to a shitty host and/or grossly misconfigured
server. No CMS should run into load issues at the microscopic traffic levels
he's complaining about. He doesn't need Cloudflare, either.

So, yes, a static site would indeed perform better but to imply that he's
actually reaching the limits of a simple CMS is absolutely ridiculous.

~~~
xupybd
>There is absolutely no justification for him to be running into these sort of
issues at such an incredibly low load

Yet he is running into these issues. Is it a buggy plugin or a configuration
issue?probably, yes.

But that's exactly the problem static sites solve. I don't mean to say he's
out grown it as in you can't serve Wordpress at the volume he's currently
getting. I mean you have to have everything setup correctly to do so.One
option would be to learn the workings of Wordpress and maybe the web server
hosting this. If that's what he wants to do with his time great. I suspect he
may have other interests, if that's the case static hosting would be a good
alternative. It might not be the best, but it's the best I know.

------
trothamel
$50 seems quite a bit of money for a wordpress blog, when you can have
wordpress.com host you for $5/mo.

------
usaphp
I am running multiple Wordpress customers on digitalocean $20/month plans, and
some of them get more than 50k uniques per month, never had any issue. Never
did much optimization. I am pretty sure the issue is your hosting. Also check
your Apache error log, maybe you are using some plugin or theme that is
overloading the memory with some crap code like majority of free Wordpress
plugins/themes.

p.s make sure to update to php 7+. The speed difference was very noticeable
when I upgraded

------
gremlinsinc
I'm just going to say one thing: EasyEngine.

If you need help managing/setting it up, I can do it for $15 a month + DO cost
- or AWS/Google/Azure. I'll manage your stack, depending on time to migrate,
there might be like $50-100 for migration.

I'm looking to start a WP management service, and hosting company -- mainly to
get some MRR during my slow months.

I'm currently a Full stack laravel/vue/react freelancer.

For reference, I ran a political blog during 2016 w/ 100k hits an hour for a
few weeks (traffic died down a ton), but my site wasn't functional at all on
shared hosting.

After easy engine on DO's lowest tier, everything just worked. Nginx + redis
cache is amazing.

Just drop me an email : patrick @ zvive.com -- linkedin:
[https://linkedin.com/in/patrickcurl](https://linkedin.com/in/patrickcurl)

~~~
dahx4Eev
How does EasyEngine compare to dokku-wordpress?

~~~
gremlinsinc
Can't say, haven't used dokku-wordpress. I know EasyEngine recently moved to
docker images, which I'm not totally a fan of -- simply because you can only
have X number of images per instance, iirc -- and that limits how many sites
you could potentially host on a single server.

~~~
dahx4Eev
Do you mean it uses more memory? or you are hitting the docker limitation per
host?

~~~
gremlinsinc
[https://easyengine.io/handbook/v3-to-v4-migration](https://easyengine.io/handbook/v3-to-v4-migration)
"As v4 depends on docker, it is affected by Docker’s limitation of 31 network
per machine.

v4 creates one network per site. Then v4 also needs few Docker networks for
global services. So roughly we limit the number of sites to 27."

I'm also wondering what all those docker instances do to performance, seems to
me there might be a better way.

------
lettergram
That’s just a bad hosting service.. I manage somewhere in the neighborhood the
OP gets with a basic Wordpress deployment. It’s extremely stable, and only
~7/month. I use limited plugins and have optimized for speed.

My guess is a plug-in or theme is bugging out here. Easy solution is to remove
all and slowly add until the problem exists again.

In 2015 I had “Crayon Syntax Highlighter was Killing my WordPress” for
instance:

[https://austingwalters.com/crayon-syntax-highlighter-was-
kil...](https://austingwalters.com/crayon-syntax-highlighter-was-killing-my-
wordpress/)

------
nodesocket
Ignore all the comments telling you to switch to static content generation,
that's generally a large time endeavour and for a blog that is getting 50k
uniques a month should be pretty easy to serve all requests with a little
CloudFlare tuning.

First, let CloudFlare do 99% of the work. Don't worry about optimizing the
linux box, PHP, MySQL, PHP Opcode Cache, WP-Cache at first.

First, let's check that CloudFlare is caching and serving all static assets
such as js, css, images, .xml, .ico. I don't see a cache-control header being
returned on js, css, images, etc. Make sure this is setup in CloudFlare with
Caching Level => Standard and Browser Cache Control => 1 month. Also, consider
enabling Auto Minify => Javascript and CSS. Finally enable Brotli and make
sure HTTP/2 is enabled under network.

Now, here is where you can really make things fly. It is possible to have
CloudFlare cache the result of generated html pages, so requests don't even
hit your shared hosting provider. This is done with Page Rules. The trick is
going to be crafting a rule that you can match posts and posts only. With my
site, I leave .html extension on my files, so my rule is simply:

[https://imgur.com/a/4iNpB4V](https://imgur.com/a/4iNpB4V)

Your challenge will be creating a matching rule while excluding admin backend
pages. Also, most certainly you probably going to want a much lower cache time
than 8 hours. My site content does not change often. Perhaps something like 30
or 60 minutes to start.

Good luck, let me know any questions, my contact info is in my profile.

------
mpweiher
Although a lot of the recommendations are great, and to me it also looks like
there are specific hosting problems at play, possibly serious overcommit by
the provider or some plugin problems, I do think there's a deeper issue:

> What am I missing? How can I be in so much trouble in 2019 with such a
> relatively modest task?

Something is not quite right with our web-platform when we have these sorts of
problems, and the solutions also require quite a bit of sophistication.

As an experiment, I recently switched my
[http://objective.st](http://objective.st) site to run on Objective-Smalltalk
itself on a Digital Ocean $5/month droplet (their smallest). Surprisingly made
it to the HN front-page and didn't hiccup, despite an egregious memory leak on
my part (fixed now).

This was reported as "blazing fast"[1], despite being a single binary hosted
in Frankfurt, having no CDN support and no static files, with everything
served from the embedded web-server. Heck, I haven't even hooked up gzip-
compression

I don't log much, so I don't really know how much traffic it's getting. I do
know that hitting it with wrk remotely I got around 1000 requests/s, locally
on the same box it was around 8K. So I think there's plenty of headroom. Oh,
and the logs do show continuous attempts to break in POSTing to what I presume
are php vulnerabilities.

I am emphatically _not_ saying I did something amazing or even particularly
good, everything was somewhere between straightforward and egregiously simple-
minded, and there were other (mostly easy) ways of accomplishing the same
thing. Previously the same site was served using static files running the same
code as a static site generator.

However, it does confirm my suspicion that we've path-depended ourselves into
a corner.

[1]
[https://news.ycombinator.com/item?id=18965054](https://news.ycombinator.com/item?id=18965054)

------
tomschwiha
Which PHP version is being used? If it is still 5.* I would consider upgrading
to 7.*. Just estimating now: 45000 users would be like about 1 user/minute, so
maybe 10 requests per user per minute, could equal on peak max. 1-2 requests
per second.

Some shared hosting shouldnt run into trouble.

------
grok2
For 50$ (Canadian, I guess), will not someone like wordpress.com do better?
Their business package is USD25 per month.

~~~
kuuspa
Their plugin selection isnt great though and you cant install your own.

~~~
hbcondo714
The WordPress.com Business Plan[1] actually allows you to upload your own
plugins and themes.

[1] [https://en.support.wordpress.com/business-
plan/](https://en.support.wordpress.com/business-plan/)

------
cyberferret
I come across colleagues in the same boat as the OP, but the truth of the
matter is that a $5/mo VPS hosting WordPress is really not going to handle a
surge of high traffic. Been there, done that, when I had a $20/mo Digital
Ocean droplet with my blog on it. Anytime I posted a link to it on HackerNews
or Reddit, it used to die under the 'hug of death'.

Since moving my blog to SquareSpace, I've never had that issue. In years.
Note: Not an advert and not an affiliate for SS, just a happy user.

We still have a WP blog on a VPS for our SaaS, but I am keen to move that over
to another platform, because I have enough on my plate to develop code for my
SaaS itself. For my blog, I just want to post my thoughts without having to
wade through a WordPress Admin console and see dozens of flags, warnings and
popups about a myriad of plugins that need attention. Seriously - our WP admin
console causes a lot of stress just logging in, which sucks all the creativity
out of me.

SquareSpace (and even Medium, for all the negativity surrounding it) makes it
so easy to start with a blank slate and just _write_ and let the ideas flow.

~~~
mrighele
To help in case of a surge of traffic you can enable some sort of microcaching
on the web server. For example with a cache TTL of as little of a second you
are sure that Wordpress will never see more than 1 req/sec (per page, but
usually surges are specific to one or a few pages), and at the same time you
will never be serving stale information.

The downside is that your Wordpress installation will not see the requests
served from cache, which means so if you rely on a Wordpress plugin for
statistics and/or analytics the data will not be correct.

------
ashton314
Anyone else use Ghost? I've got Ghost running in a Docker container on a DO
droplet. It's the smallest droplet there is. I don't have a ton of traffic to
my blog (at least, I don't think—I haven't found a privacy-respecting analysis
system that I like (maybe roll my own... just count page request)) but it does
the job.

~~~
johnonolan
We generally serve several of the front page stories on HN every day (some we
host, some we don't) - so very used to dealing with high traffic loads

@OP - If you're interested, drop me an email on john@ghost.org and I'll set
you up with a 2 month free trial on our Pro service where you can deploy a new
instance in a few seconds and kick the tires. No obligation.

------
ddebernardy
If I recollect correctly (haven't used WP in years), the bulk of error
500-related problems on WordPress are due to poorly written plugins -- of
which there are many. One of the better ways to track down a problem is to
disable plugins one by one until the problem goes away.

~~~
mikemotherwell
Further to that, delete the folders. Plugins are often targeted directly if
written poorly, so if you don't need 'em, delete 'em.

~~~
ddebernardy
Disabling them is good enough, actually. The php code in them won't run at all
unless they're enabled.

------
Tepix
50 CAD a month (33€) for such a small site and then it doesn't even work
properly. It boggles the mind. I'm sure you can get a working setup for
5€/month without breaking a sweat.

Having a company that deals with your Wordpress installation is only
beneficial if they get it right.

------
sareiodata
WordPress might be slow, but it's probably just the host or not enough PHP
memory assigned to it. There is nowhere near enough traffic to bring down a WP
site, unless you have a ton of bots pinging your site that never appear in
your actual google analytics (because they are boths) .

* look at an actual server access log. cPanel offers one if your host is offering cPanel. * try and increase your WordPress memory limit: [https://docs.woocommerce.com/document/increasing-the-wordpre...](https://docs.woocommerce.com/document/increasing-the-wordpress-memory-limit/) * check your server's PHP error log and see if there is an error * set WP_DEBUG to true in wp_config.php

------
burritofanatic
Hugo. It's completely static and great out of the box. You can add new posts
in Markdown.

~~~
hbcondo714
I looked into Hugo but stuck with WordPress because the transition of 2M posts
with 1,000+ daily new posts seemed too much for the build process:

[https://discourse.gohugo.io/t/transition-2m-posts-from-
wordp...](https://discourse.gohugo.io/t/transition-2m-posts-from-wordpress-to-
hugo/12704/5)

------
zokier
Oh, good old traditional LAMP shared hosting and WP. Honestly, I doubt that
combo will ever work reliably. WP hosting is bit of a dark art in the best of
times, and operating under shared hosting does not help. That being said,
self-hosting on something like AWS probably would end up much more expensive,
and would take also some effort to set up and maintain.

Personally I would either start looking for fully managed WP service
(wordpress.com), or switching to a different, easier to host, platform

------
tlarkworthy
I use wp-stateless to offload images, and cache-control to serve most of the
content through CDN. (I also put a serverless reverse proxy infront to mixin
other stuff on a common domain). I get great performace off shared hosting.
[https://corepox.net/devlog/simple-extensible-performant-
low-...](https://corepox.net/devlog/simple-extensible-performant-low-cost-web-
serving-using-firebase-hosting)

------
getty
Caching 'anonymous' requests at edge with Cloudflare workers - no need for the
$200 p/m business plan

[https://www.mmaton.com/2018/07/02/cloudflare-cache-
anonymous...](https://www.mmaton.com/2018/07/02/cloudflare-cache-anonymous-
requests/)

------
hayd
You can export to static pages (one for every blog) with a plugin. You'll lose
the comments functionality, but views will work.

An easier/quicker solution is to just stick a cdn in front, that way you can
cache pages and still have to back-end work.

------
warent
No offense intended at all but I can't help but wonder how a
technologist/computer scientist has ended up asking for help resolving 500
errors on his blog rather than hypothesizing what the problem could be and
testing his theories.

A quick Google search on the error he's getting says it's an Apache error
either from invalid file permissions or running out of memory. His provider
refuses to transparently increase memory so the hypothesis seems stronger
here. Now just test it by spinning up a simple load balancer, clone your
website on a different server, split the traffic 50/50, and see if the errors
continue. Repeat this process until you've isolated the issue. Trying to solve
the problem by adding a cache layer isn't going to help before the problem is
understood.

~~~
groundCode
We don’t know that the author didn’t google and try a few things. And it’s
perfectly possible to be a computer scientist and not know how to tune a web
server. There are lots of specialisms and paths. Personally I admire someone
who, despite being an expert, isn’t afraid to ask questions and admit when
they don’t know something.

~~~
warent
That's a great point. My comment above seems to be coming from a rather
limited mindset

------
chemmail
Seems like Wordpress has outlived its usefulness. It's like the Windows XP of
the internet. It was great and simple, until the security problems and feature
creep plagued it to death.

~~~
usaphp
Most of the time the problem is not Wordpress but poorly written
themes/plugins (especially free ones). Also $5/month shared hosting running on
a decade old hardware are not helping either.

------
vfclists
This won't end well for csoft.net

------
amaccuish
Reading this, I feel they should hand this job on to someone else. There comes
a point where if you're not experienced enough to do this, no amount of
Googling and investing more time will help you.

tl;dr I get the feeling that this person has been sold the idea that throwing
money at things will help (why is he using argo for a smallish blog, it's not
worth the money?)

------
stevenicr
I've had similar issues with several wordpress installs.

Some of the less known issues I found that can add to this problem:

Spiders/Indexing Bots Brute Force Login Attempts Hidden Hacked Pages

Helps to look at logs sometimes, if you get hit with all three of the above at
the same time, it's a tricky mess to clean.

You may find relief making an aggressive noindex robots.txt file (there are
tons of bots to block which are worthless and use up your resources. - it may
be best to say * disallow all, then specifically allow google, bing, etc
(would love to find a way to tell archive.org it's okay in that situation))

Even when doing the robots block, I've found some aggressive spiders or bots
that use some of their user agents need to be blocked via ip subnets and cidrs
with htacess for example.

Brute force login has many options for slowing down access. I love a combo of
ipgeo block, sucuri, graphic captcha, shield security

plugin for disable xml-rpc or use the setting for such with the shield
security plugin (this blocks massive login attempts) to block all xml-rpc use.

Hidden Hacked Pages If you find bots spidering a bunch of product pages for
things in other countries in your logs for example, you may have been hit with
a bunch of pages being created that you may not see on your may front end, or
even in the backend admin area of your site.

Removing those pages is a start, and then a plugin like 404 to 301 redirect
may be beneficial - but you will likely find a bunch of bots coming to index
them, and other hackers who have been sold access to your site trying to login
and make more of them.

For me it was more log checks and looking up ips to add to the server block
lists to slow it down. Just make sure you don't add google's or bing's ips to
the list.

With a few sites I had that got some of these issues I decided to pull down
wp-login.php - and in an effort to lighten up the load on the shared server, I
decided to use one of the plugins that make wordpress into a static site
(there are two good ones in the repo now)

Once I generated the static html and css, I created a new wp-login.php that
would log all ips and user agents / date to a separate text file, and display
a message on the screen warning the visitor hacker that I was casting a spell
on those who access the login page.

The amount of bots trying to login, and trying to access the old hacker-
created-pages that were made over a year ago is still crazy. Their bots are
still coming and trying the same login page and trying to access the same
pages that have been removed over a year ago.

If I had left those pages available as php pages, each one of those visits
would of been taxing my sql, even if they were showing the wp 404 page, or
failed login.

I hope your issue is simpler, like a bad plugin. Certainly I've run into that
issue with other problems not related to these as well.

------
angel_j
Pssh, all you need to do is install docker, convert static files and blog post
into lambdas, then route each unique http request to a serverless IPv6 address
located on the nearest continent.

~~~
xupybd
Can you elaborate a little? I'm not sure I understand what you're suggesting
but it sounds interesting.

~~~
reilly3000
Here is the architecture: [https://serverless.com/blog/serverless-wordpress-
hosting-wit...](https://serverless.com/blog/serverless-wordpress-hosting-with-
shifter/)

[https://www.getshifter.io/](https://www.getshifter.io/) seems to be a
relatively turnkey solution for existing Wordpress sites to migrate to
serverless hosting. It spins up a Wordpress docker image when somebody is in
the admin panel, then builds and deploys a new static site to S3 when a change
is made. Each build is immutable and easy to switch between.

There are limitations with static site hosting. Their approach to comments is:
Use Disqus.

------
geralmeller
I can recommend you Publii CMS
([https://getpublii.com](https://getpublii.com)), it's excellent static blog
tool I use from over one year and manage four blogs.

