
Ask HN: My host has me over a barrel.  I've placed myself in a pickle. How to handle? - brandnewlow
I publish a social news site/blog network (URL in my profile).  It's hosted on a "Gold" VPS with Midphase. Details here: http://midphase.com/hosting_vps.php. I'm using Drupal to power it.<p>While we had 5,000+ uniques per day last week, on average we've been getting around 1200-1500 per day.<p>This doesn't seem like a lot of traffic to me.  However for the last 4 days, apache has been failing every 3-4 hours and restarting about 10 minutes later, meaning 10 minutes of downtime for each crash. My host is saying the site is hitting its memory limit all the time.  Needless to say, this is a problem.<p>For the last three days I've been filing tickets with my support guys (I bought a 5-minute response support package for $60/month thinking it would free me up to do more writing/editing) every time there's an outage.  I've now filed 34 tickets.  Their responses say the same thing every time "We are making apache configuration changes and will monitor your performance."  Last night we had about 5 hours without a crash.  I thought things were fixed.<p>They just went down again.<p>I've spoken with a manager at Midphase.  He said that he doesn't see any obvious problems with how their server is configured or how my site is configured.  It's just using up too much memory.  His suggestion (of course) is to move to a dedicated server with Midphase at $200/month.<p>Now, I'm more than willing to concede that my site could be better optimized.  I'm working on that and have some Drupal pro's who've offered to give it a look sometime in the near future.  I'm also aware that I should be planning for future growth with our server setup.<p>I just don't want to get ripped off.<p>It seems like I've got three options.<p>1.  Ask my hosting guys the right questions and find out what's causing the problem and get it fixed.<p>2.  Upgrade to a dedicated Midphase server, potentially encounter the same problems.<p>3.  Go elsewhere, potentially encounter the same problems.<p>Are there some specific questions I should ask my hosting people to help me get a better picture of the situation?
======
midnightmonster
You're paying quite a lot for a very weak VPS--compare at slicehost or linode.
So your first step might should be to spend _less_ for a more powerful server
at one of the other VPS providers. OTOH, I'm not sure linode or slicehost even
offer the kind of support package that you have paid for with midphase--you'd
be on your own. OTOOH, if I read your post right you're paying midphase
$130/month currently. You can get a whole lot of server for a lot less than
that.

For some combination of Drupal's general heavy-weightness, lack of caching,
slow server, your front page performance is sporadic and mostly very poor. I
get times mostly in the 0.6 - 1.4 sec range just for loading the HTML, and
that's with about 0.15 sec as the baseline for just loading a similar-sized
static image from the site. I'm not a Drupal expert even a little, but if you
could fully cache the front page (so that apache serves a static file
directly) and update it, e.g., once every 10 min, you'd be flying,
comparatively.

But about what's going on, I'd lay money it's all of the above: you have a
low-resource server, drupal isn't fast/light by default and you haven't made
it any better, you're not using any caching where it would really help. You
should work on fixing all of those.

~~~
jhancock
Excellent analysis given little info. Although it is possible the VPS host is
not managing its resources well and its possible Apache is configed poorly, it
is just as likely the app itself is a pig and needs lots of cleanup.

If you don't have the skill set be a good sysadmin and you "dont know" if its
your app, you are in for a world of hurt. I highly suggest you find help: a
good sysadmin and maybe an outsider to look at your code and web page
structure. I have not heard enough to say your hosting company is screwing
you. And its maybe wise that you are paying $60/month for better tech support
but you still only get $60/month worth of tech support and that's not much ;(.

------
lethain
If you're willing to handle the stuff yourself, and it isn't that bad,
SliceHost is a charm. I've hosted my django based blog (600-1000 uniques per
day, ~6000 some peak days) on a 512 megabyte vps (Apache2 with mod_python,
nginx for static files and proxying other requests to the Apache2, memcached
instances with maybe 32 megs (not a lot of content to cache), and postgres)
for the past 8 or 9 months and it's been great. On a 256 meg vps it used to
crash, but 512 has been golden with my traffic levels.

Oh, and its $38/month for 516 meg slice. Can't say enough kind words about 'em
(if you're willing to fire up SSH yourself:).

~~~
brandnewlow
That's the problem with Slicehost for me. I did sign up for a Slice, about a
month ago, based on the reviews here. I spent three days trying to learn how
to "make it go" and after some thought and cost/benefit analysis, decided
against migrating over. Perhaps I should bite the bullet and spend a month
learning sysadmin basics...

~~~
0x44
We have a lot of articles at <http://articles.slicehost.com> which will walk
you through a lot of typical server installations. Additionally, even though
our service is unmanaged, we're usually in a campfire chat and will answer
questions if we're able (and direct you toward the answer if we're not).

~~~
plusbryan
Slicehost is amazing. I wish they had pre-built LAMP stack images ready, but
it's not that hard to set up (esp if you follow their tutorials)

------
jbyers
From what you've said, I doubt host swapping is a good idea - yet. First you
need to get a good understanding of what's causing the system to run out of
memory. One way to do this is to watch vmstat and top when the system is
failing, or to install one of many system monitoring packages.

From what you've stated, I'll take a stab at what's going on. Apache processes
are big, from a memory usage standpoint. A single Apache + mod_php could use
25 MB or more. If you have a dozen or more requests that hit at once -- or
processes that are feeding slow clients -- blam, you've run out of memory.

One solution is to switch to lighttpd, nginx, or another webserver with a
smaller footprint. Here a single process (or small set) serves a pool of PHP
processes. But that's a bunch of work for a guess at what's going wrong.

Instead, try running nginx as a proxy in front of Apache, and let it spoonfeed
slow clients. It's likely you will not need to change Drupal at all, and make
only minor changes to Apache to do this. A good description is here:
<http://blog.kovyrin.net/2006/05/18/nginx-as-reverse-proxy/>

------
FiReaNG3L
384 MB of RAM is really low for a site (apache + db i assume). I host my
Drupal site ( <http://esciencenews.com> ) at ServInt on a 90$/month package
with 784 MB of RAM. Dont know how I would do it with 384, looks impossible to
me. Higher traffic daily than your spike.

~~~
brandnewlow
Hey there! I forgot you hang out here (confession: I read your esciencenews
Drupal showcase writeup about once a week for inspiration).

Ok. So 384 looks low. That's helpful, objective advice. Thanks.

~~~
silencio
I'm paying for a dedicated server at the moment with 1GB ram for 39
euros/month, an upgrade to 2gb ram (total) is 10 euros/month extra from
LeaseWeb (<http://leaseweb.com>), which has half decent routing. I know the
exchange rate sucks (being in the US myself), but an option to consider,
perhaps.

I've always been partial to Slicehost, but my main need uses a lot of
bandwidth and disk space and they absolutely can't compare to overseas hosts
for bw/$. slicehost's most generous plan is 1600gb transfer at $280 and our
4000gb for 79 euros from leaseweb is vastly cheaper, although you have to mix
and match some of the other specs.

Honestly, do not go with midphase dedicated servers. If I remember correctly
they're just reselling softlayer(.com), and softlayer is a LOT better in every
aspect compared to midphase. I've had some friends use midphase and have so
much trouble with their customer support when something goes wrong...softlayer
in my experience has been everything great about a company providing these
kinds of services.

------
davidw
1) You need to understand this stuff yourself. You need to learn about Apache
and Drupal and the other technical bits - maybe not to the point of becoming
an expert, but enough to work with them a bit.

2) Go elsewhere. Slicehost (or linode) has better VPS deals, and Layered Tech
used to have some pretty good deals on 'real' servers.

------
durana
I'd ask them to clarify what they mean by a crash. Is it a system crash or is
it Apache going out to lunch or is it something else? A "crash" can mean so
many different things because it is somewhat vague. I've seen plenty of cases
where if you press someone for a root cause of a "crash", they can't give it
to you because they don't really know what went wrong. You've got to
understand the problem before you can solve it or be sure what they are
recommending makes sense. If you want some more input, feel free to e-mail me.

~~~
brandnewlow
After every outage I receive an automated e-mail telling me that httpd has
failed on my VPS. The way this usually works is I notice the site goes down.
It comes back up 8-10 minutes later and I find another e-mail in my inbox
saying that apache failed and has restarted.

------
rcoder
First, you should know that many types of caching will _increase_ your memory
usage, not decrease it. The whole point of a cache is to use fast storage
(RAM) to keep a local copy of data that would be slower to retrieve from its
persistent source (filesystem or DB).

Second, 384 MB just isn't much memory for a medium-traffic Drupal site. I
would recommend at least a gig of memory, and perhaps more if you use plugins
heavily. Drupal is neither particularly lean nor blazingly fast, so you're
going to need a lot of Apache listeners working to handle that load, with each
process probably consuming 30-50 MB of RAM.

Third, you're right to think that you need someone on-call who knows Apache,
Drupal, and database tuning and configuration inside and out. A $60/mo.
support package isn't going to do it, though. If you think about it, that's
really only paying for an hour or two of tech time each month, so expecting to
get 24/7 interactive performance tuning and optimization for free along with
your support contract is pretty unreasonable. Put a performance guru on
retainer, and establish an SLA with them if necessary to maintain your site
uptime.

Finally, opening >30 tickets in less than a week is not the best way to get
quality support. It's like the stereotypical impatient kid on a road trip,
asking "are we there yet?" over and over again.

My suggestion would be for your to find an experienced LAMP admin who is
willing to manage your hosting for you, and pay them a reasonable fee for
doing so.

~~~
silencio
I once had to deal with a hosting company that had a ticket tracking system
that would mark tickets as closed if there was _any_ kind of customer reply,
and they had a fat fingered support person or two that also marked tickets as
closed..and of course, if a ticket was open for too long they'd automatically
be closed, so any ticket I send on a Friday gets closed by Monday morning. I
HAD to open a couple dozen tickets and replies to tickets before someone
finally replied to me and apologized for the whole thing :) Needless to say
that was one reason why I am no longer with them..

But I can sort of commiserate with the OP. Midphase support is not exactly the
best...I've spent hours trying to get them to understand the most trivial
things, like when a friend's website wouldn't work because the mysql server
was awol and the support person didn't understand what was wrong with nothing
showing on the website. Yeaaaahhhhh.

------
brandnewlow
Ok. I'm going to call the folks at ServInt and Linode and talk to them. I am
concerned about migrating from one host to another as I've never done that
before but I can figure it out.

I'm also going to look more into improving my site's performance. I have
enabled Drupal's page caching, but I guess it's not doing much to help. I'll
pursue that further.

------
patrickg-zill
Well the first you can do, is to configure apache to restart an httpd process
after serving some number of clients. Under Apache 2.x this is called
MaxRequestsPerChild and the default is 10000 requests. Since you are not
bumping into CPU problems, try reducing this number to say 500 requests. After
a child process has served 500 requests it will exit, and another process will
be started.

------
rms
Before slicehost I was on Mediatemple's gridserver... the latency was terrible
and there was more downtime than I would have liked, but it was ok. I suspect
their service has improved somewhat in the interim and they provide more
bandwidth than any other shared, managed hosting provider for the price. If
you really don't want to manage your own server, they are probably worth
looking into.

------
bstadil
Host if on Amazon Web services. Will cost you $72 / month and you can scale up
from there at peak times if you need. You get 1.7GB of memory and roughtly a
2.5GHz processor. If you need help setting this up use Scalr.net or we
(cloudincode.com can help you for free) Drop a line if you are interested
bjorn At Stadil.com

------
pistoriusp
Move your static files to a different subdomain / host or vhost. (AWS?)

Turn Keep-Alive Off. (If you're site doesn't have tightly packed multiple
requests). The default is 15 seconds, I saw a 50% speed increase when I
disabled this. Set MaxRequestsPerChild to 500, in case your code is leaking.

~~~
shutter
Better yet, set Keep-Alive to something very low (1 or 2), so that when users
load a page, any initial assets can still reuse the same connection, but
you'll still drop the connection quickly to serve other users.

------
theantidote
I agree with most of the above. Move to a more cost effective host, optimize
your Apache server, and use page caching in Drupal. I think it's as simple as
a checkbox in the settings somewhere, but it's been a few months since I've
worked with Drupal.

<http://drupal.org/getting-started/6/admin/settings> The manual says caching
is under the Performance section and it's disabled by default.

------
alex_c
Their plans seem a bit expensive. Depending on how much bandwidth you use,
slicehost.com might be a better option - you get 1GB of RAM for the same
price.

------
staunch
For another $10-$20 you could have a physical dedicated server with 1GB memory
on ServerBeach. I highly recommend them. You can get $100 off if you use my
referral code 7XYHDMBU8A but even if you don't use my code I _still_ recommend
it!

[http://www.serverbeach.com/catalog/cust_ref_landing_new.php?...](http://www.serverbeach.com/catalog/cust_ref_landing_new.php?REF=7XYHDMBU8A)

------
noodle
tweak your apache configuration to see if that will fix your swapping
problems. there are tons of articles on what you need to do to do this
available online. sounds like you're just spawning a lot of children that are
sticking around to fill up memory and swap space.

i had similar problems with a default config of apache on a 256 slice. if you
tweak things correctly, you'll be fine.

------
lsc
you have 2 options.

1\. buy a much more powerful server/VPS.

for seventy some odd dollars a month you can just leave your amazon ec2 image
up all the time, and that's something like 1.7Gb ram. if you are willing to
use a small provider (like me) you can do even better- I'll give you 2GB ram
for $64/month. Having more ram makes a lot of problems just go away. (using a
Xen based host also means that you can use swap, which means that your out of
memory events would be slowdowns rather than crashes.)

2\. get a skilled SysAdmin (possibly you) to optimize your system from a
systems standpoint.

ultimately, you want both. But often just buying a better server is the
cheapest option.

But yeah. no matter how much you like your host, I recommend that you always
have a hot copy of your website/application at another server, ready to switch
the dns over at the first sign of trouble. you should test this periodically.

------
matthewking
How about EC2? 1.7GB on their lowest virtual server - Works out at about $72~
per month I think?

------
DanielBMarkham
I had the same problem four years ago. It killed my startup. The volume
numbers weren't there for a dedicated box and the VM-type hosting was
continuously (and randomly) rebooting.

------
niels_olson
I know HN discourages superfluousness, but I'd just like share my moment of
zen as I contemplate how this is exactly the kind of post that keeps me coming
to HN.

~~~
brandnewlow
Happy to help! I was honestly worried I'd get booted for posting about hosting
when most here can host their own sites.

------
blender
Couple quick things for Drupal:

Do you have Optimize CSS files (CSS file aggregation) enabled?

Do you have Caching mode (Normal) enabled?

Cheers

~~~
brandnewlow
Yes and yes.

I'm also using the blockcache module for most of my blocks and have js
aggregation switched on (though I have to omit tinymce and a few other js
files in order for them to work).

~~~
mseebach
FastCGI and mod_xcache? Made all the difference on a couple of TYPO3 (PHP CMS)
site I've run.

------
adatta02
check out <http://novanix.com/>

I realize the website looks shady as all hell. But we use them for EVERYTHING
and we've never had a problem. Their support is also top notch. Usually
answers+fixes <30 mins.

------
speek
linode.com asap

~~~
petercooper
Seconded. You could probably host this site quite easily on their $40/mo
package - if not lower.

