
Linode NextGen: The Hardware - rk0567
http://blog.linode.com/2013/03/18/linode-nextgen-the-hardware/
======
grey-area
According to the linode FAQ [1], current linode servers have roughly 20GB of
RAM shared out. Does anyone know the spec of the new servers and what the max
memory would be? The max for the CPU seems to be 750GB [2]. Presumably servers
would be far less than the theoretical max, but fingers crossed they'll be
able to bump the memory at some point soon so that they're a bit more
competitive with others.

I was really hoping that was going to be this announcement, however the fact
they've titled it 'The Hardware' hints that there might be nothing new on RAM
in the next announcement, which would be a shame. Upgrades to CPU are nice,
but it'd be nicer to see a RAM upgrade, as almost everyone is constrained on
RAM rather than CPU or disk, particularly on newer hardware and with their new
bandwidth limits.

I feel rather ungrateful now having said all that. Thanks anyway Linode!

[1] <http://www.linode.com/faq.cfm#how-many-linodes-share-a-host> [2]
[http://ark.intel.com/products/64595/Intel-Xeon-
Processor-E5-...](http://ark.intel.com/products/64595/Intel-Xeon-
Processor-E5-2670-20M-Cache-2_60-GHz-8_00-GTs-Intel-QPI)

~~~
bpatrianakos
Same here but I'm still holding out hope. I recently wrote a blog post
comparing my experiences with both Digital Ocean and Linode and both companies
ended up commenting on it. Linode hinted that there'd be a few changes coming
soon that'd please me. Considering I mentioned RAM and pricing I want to
believe they were hinting that they really were planning on bumping up the RAM
for current customers. They didn't say anything we haven't heard them state
publicly before but even so, I'm still holding out hope. I think more RAM is
probably the number one request and they'd have to be living under a rock not
to know it. So I'm hoping that maybe they're playing it close to the chest for
now until all the other upgrades are complete and then maybe they'll increase
our RAM when it's all over. Who knows. We can always dream.

~~~
grey-area
Just seen this link to a pic on twitter lower down in this thread:

<https://news.ycombinator.com/item?id=5396634>

Not sure where that pic is from or whether it is legit, but if they double the
RAM on all plans as it implies I'll be really pleased... there's some
intriguing text top right too which looks like it might be some sort of stats
monitoring system (longview?).

~~~
mglinski
I can "confirm" that double the ram is coming, they were handing out flyers at
SXSW stating as such. Here is a photo I took in high resolution:
<http://code.mglinski.com/X3ps/30zoh6Yf> Of course it is all still plans, but
after talking with them directly about it I feel very confident they will be
doing your ram junkies sweet justice once they have had the chance to move
everyone over to the new E5 boxes.

They had a demo of Longview on the show floor too, from what I gathered it is
a package install based daemon to monitor processes and the like, something
like the server part of NewRelic but sitting directly in their datacenters.

~~~
grey-area
That's good news! They're just stringing the announcements out for more
publicity. Thanks for the better pic.

------
recuter
> So what about SSDs? There’s no question SSDs are in Linode’s future, however
> enterprise-class SSDs (SLC or eMLC based) are prohibitively expensive. And
> although MLC-based drives are cheaper we just don’t feel right about using
> consumer grade laptop drives to power your Linodes. So we will wait until
> capacities for enterprise SSDs increase.

Well that is disappointing and arguably more important for many people. Not to
be a debbie downer or anything.

~~~
afreak
Enterprise-class SSDs are grotesquely expensive at the moment. A quick gander
around shows that a 200 GB SSD from HP runs you about $1,400. Since Linode
needs ample space, the largest I found was 800 GB which runs you almost
$7,500.

When you're looking at around $8/GB for storage of this type, it's no surprise
that Linode didn't bother.

Surprisingly, HP is offering the drives with 3-year warranties, which is a far
cry from sometime ago where they were something like a few months.

~~~
insaneirish
People often say SSDs are expensive. But they're assuming that the capability
of a hard drive is some scalar value (storage capacity) that makes it easy to
compare SSDs to spinning rust.

The fact is, if you care about IOPS, SSDs are cheaper than spinning disks. In
other words, you need to stripe out across way more spindles (and thus consume
a lot of power) to get the IOPS you get from a decent SSD.

~~~
EwanToo
Exactly, to say an SSD is 8x the cost of a hard drive, or even 2x the cost, is
missing the point.

Even an expensive SSD is _cheaper_ than a HD on an IOPs to IOPs comparison,
and IOPs is what matters to most people.

------
trotsky
I'm not super up on os level virtualization providers, but I'm curious why
exposing 8 cores is an advantage. Based on the ram sizing it seems like they
expect to run 100+ containers per processor - so surely they're restricting
cpu time per process (instance) somehow. If you use a bunch of cpu time won't
you just get starved out when you hit an invisible cpu wall? If you're only
getting 1/10th or less of a cpu core wouldn't it make more sense to pin each
instance to just a couple of cores minimizing context switches?

~~~
asb
You can use as much spare CPU as is available. Upon contention, the CPU is (in
theory) dished out fairly. Other schemes are of course possible (like Amazon's
EC2 where you get a fixed CPU allotment).

~~~
bradleyjg
EC2 doesn't actually reserve physical CPU time per vm, do they? I had assumed
that the lack of burst just means they are keeping the statistical
multiplexing windfall for themselves.

~~~
shrike
Excluding t1.micro instances you are getting a fixed, dedicated amount of CPU
per instance, CPU is not a shared resource on EC2.

~~~
bradleyjg
So just to be clear: If I have a reserved instance I'm not using there is a
CPU sitting somewhere doing running HLT cycles?

Wow.

~~~
toomuchtodo
Vs needing CPU cycles and you having to wait if someone else is using that
shared CPU time? There are always tradeoffs.

~~~
bradleyjg
I didn't mean to be critical, customers certainly get better latancy on a
fixed time slot implementation over an interrupt based QoS system. I'm
surprised from Amazon's perspective, especially because they are competitive
on price.

------
thesmok
Wow, first time in my life i was able to found a link to the company.com at
the header of blog.company.com! Good Linode.

~~~
StavrosK
I like my approach: <http://www.instahero.com/blog/>

There's a link to the site at the top, and a link to the blog below that.

~~~
simplon
yes! i like this approach.. rather than hitting the blog main page when i
click on the header

------
madsushi
Just like a good showman would, Linode will be saving their RAM announcement
for last. Network, CPU, disk again (maybe?), anything else (new
services/options/load balancing/etc), and finally, memory.

~~~
asb
Yes, definitely looks like it:
<https://twitter.com/asbradbury/status/312984696852185088>

~~~
taligent
If that is the case then pretty disappointing. It really needed to be 4x to
compete with DigitalOcean and the like.

<https://www.digitalocean.com/pricing>

~~~
bpatrianakos
Digital Ocean may be cheap but I honestly think both Linode and DO are on even
footing right now. DO has to be cheap to make up for what they lack in a web
UI, IPv6, less CPU cores to work with and just all around lack of established
trust (they're super new and I for one am slightly wary). Oh, and their
network can use some work too. I can get a bit slow sometimes.

Linode is expensive but they give you lots of control in your account panel,
you know they'll be around for a long time, excellent support, IPv6, and on
and on. Not that DO is bad though. They've got SSDs which is awesome, they're
cheap, free snapshots, and so on.

I'm a happy customer of both services and I don't think one is better than the
other. I think both serve different types of customers. For me, personally,
Linode is where I go to host my serious projects - the money making ones.
Digital Ocean is where I go for a quick, cheap server to mess around with. I
have one "main" server on both services but I trust my Linode more. Maybe
that'll change as Digital Ocean matures.

~~~
kijiki
I recently went DO for a VM because they use KVM, not Xen paravirt, so 64bit
VMs have decent performance. I needed to run a software package that was 64bit
only.

------
bluedino
I wonder if they considered allowing 4 cores, but double the CPU allotment?
4x1000MHz CPU instead of 8x500MHz CPU, for example.

8 cores does give you the highest best-case performance, since you can use
100% of each core if the other Linodes on your host are not using them.

They show the 1024 Linode with 8 CPU's as an example. But they also state that
there are on average 20 1024 Linodes on each host machine, making 20x8 = 160
vCPU's being used.

Marketing-wise, more cores sounds impressive. But I wonder if performance per
physical host is reduced splitting them into so many vCPU's

------
hhw
The E5 came out last March, is based on the previous Sandy Bridge
architecture, and is hardly new. It was also practically a year late,
launching the same time as the Ivy Bridge based E3 v2's. The 55xx series was
launched in Q1 2009, and the 56xx series was launched in Q1 2010, so their
existing hardware is 4 years old at this point. Yes, this is a major
improvement, but in reality, they are simply catching up to last year's
hardware, not exactly being innovative. The claim that their average hardware
will be less than a year old when their upgrade is complete comes off as
deceitful PR spin. The E5's are already a year old now, so they must be basing
the age of the hardware on when it was purchased, and not when it was released
to market.

The Ivy Bridge based E5 v2 is also coming out in Q3 this year, and supposedly
with 12 core models, so it seems that this is a somewhat poorly timed upgrade
on Linode's part. They should have either upgraded to the E5 v1's much sooner,
or just wait it out for the E5 v2's.

~~~
ksec
Exactly my thoughts. The Ivy E5 is just so much better in Power/Performance. I
dont understand why they need to update now, not Earlier or later.

------
derengel
I must admit Linode IO is pretty bad,

:~# dd if=/dev/zero of=test bs=64k count=3k oflag=dsync && rm test 3072+0
records in 3072+0 records out 201326592 bytes (201 MB) copied, 41.3478 s, 4.9
MB/s

this a 512MB Linode but still...

I hope they fix this with the new hard drives

~~~
zxer
another datapoint:

Linode 512: 3.96647 s, 50.8 MB/s

DigitalOcean 512: 5.87315 s, 34.3 MB/s

(though the digitalocean one feels much snappier)

~~~
derengel
wow, in what DC is your linode on?

~~~
zxer
London

------
larrys
"We’re investing millions to make your Linodes faster. Crazy faster. "

Skeptical. I'm wondering if this is just advertising hyperbole because Linode
doesn't seem to be large enough (and there is no evidence to indicate any
"funding") to be able to "invest millions".

They operate out of a suite in an office park outside Atlantic City NJ.

(I think it's a great company by the way I just don't think they are investing
millions it doesn't make any sense given what I know about them.)

~~~
showerst
Linode has over 45,000 customers (as of 2011), paying a bare minimum of $20
usd/month. Realistically they're probably bigger than that now, and the avg
$/customer is higher. I'd say they're pretty good sized.

<http://www.prweb.com/releases/2011/8/prweb8739680.htm>

~~~
larrys
"Linode has grown to over 45,000 customers"

I'd like to point out as someone that has made the INC 500 list and knows the
process very well that there is no vetting to the numbers that you give them.
Back when I did it you simply needed a letterhead from your accountant and Inc
went with whatever you said. (Might have changed but that's the way it was).
There is obviously no audit. And there is no question that people fudge to get
on the list in various ways because it's a good marketing tool.

Linode is not a public company releasing information. Once again I'm not
saying the info isn't correct and that they don't have that many customers.
I'm simply pointing out that the fact that they issue a press release saying
they have 45,000 customers (along with the math that you are assuming gives
them $20/customer) isn't necessarily correct.

~~~
agwa
Linode has been around since 2003 and only made the list for the first time in
2011, which happens to be right after several years of massive growth in the
industry. It seems quite plausible that they would legitimately make the Inc
500 list.

Another way to approximate their number of customers would be by IP address
assignments:

[http://bgp.he.net/search?search[search]=linode&commit=Se...](http://bgp.he.net/search?search\[search\]=linode&commit=Search)

I count 176,640 IPv4 addresses, and that's only direct allocations - their
older customers are still using IP addresses owned by Linode's datacenter
vendors (SoftLayer, HE, etc.).

If what you say is true, I have no doubt that some companies fake it to get on
the Inc 500 list, but given the facts about Linode it seems fairly unlikely
that they faked it, and quite possible that they could be spending millions on
infrastructure upgrades.

------
moe
Pfff. Try harder linode.

    
    
       # grep MHz /proc/cpuinfo
       cpu MHz		: 3922519.116
       cpu MHz		: 3922519.116
       cpu MHz		: 3922519.116
       cpu MHz		: 3922519.116
    

That's on a rackspace small instance. 4000 GHz, baby!

~~~
jebblue
That's only 4 cores. I'd rather have the cores.

~~~
rektide
They could flip a switch tonight and tell you you've got 128 cores. That
doesn't mean those cores are going to do you any good.

Linode is the only VPS I've seen that gives a gross number of cores to
everyone. Instead of allocating a cpu count correlated somewhere along side
plan-tiering, they just give everyone a lot of CPUs (8), and then they use
tiers to determine priority, who gets to actually use those cores.

An entry level plan has 1/16th the CPU priority of a top tier user, and pays
1/16th as much and has 1/16th as much ram. If you're not RAM bound, for a
somewhat loaded system you'll see slightly better than linear scaling (since
you aren't paying as many context swaps) as you move up in tiers.

I think it's a really bad policy to give entry level users scheduling on so
many cores: their workloads will never stick on a local cache! Their jobs are
going to be bouncing around the system getting evicted from everywhere and
bouncing to the next place where they can trash it's cache next. This is a
marketing gimmick, a "our number is higher" trick, played on people like you
jedbblue!

Some time in the past: the Pentium IV netburst core was going to hit 4GHz,
then 5GHz. The mhz wars were on, mhz sold, they were what people saw, first
and foremost, and it's more the fact that the insanely-deeply-pipelined core
didn't even scale that scrapped the idea than people wizening up. I'm not sure
if Linode's scheme here is as bad as all that, but it seems unnatural to me,
and I certainly would not consider 8 cores to be a boon in this circumstance.

Edit- this does change things a good bit! A quote from elsewhere in this
thread I was not aware of-

    
    
      We limit the number of Linodes placed on each host 
      machine. We also only place one plan type on each host. In 
      the worst-case scenario, you're splitting CPU time evenly 
      with your fellow Linoders, but are still able to use the 
      full potential of the host if others are idle.
    

This is a little different circumstance than what I'd guessed at above as
you're not going to be outclassed by higher tiers, you just have to play with
peers, and pay to have less of them.

~~~
jebblue
Yup, I've tried many over the years and Linode is the best I've found. :)

------
pavs
Anyone knows whats the clock speed for each virtual cores? Isn't that a
relevant and important information?

For instance if the previous virtual cores were all 1ghz and new cores are all
500mhz, doubling the cores won't do much good if you don't know their clock
speed.

I was critical of last Linode's NextGen post discussion, because I didn't
think upgrade was impressive compare to others, but this is one is a nice
bump, memory is probably what most people will care about more though. So
maybe next refresh is the memory?

~~~
drKarl
Just rebooted and made cat /proc/cpuinfo and the and its 2.27Ghz, so the last
of the 8 CPUs is

processor : 7 vendor_id : GenuineIntel cpu family : 6 model : 26 model name :
Intel(R) Xeon(R) CPU L5520 @ 2.27GHz stepping : 5 microcode : 0x11 cpu MHz :
2266.746 cache size : 8192 KB physical id : 0 siblings : 8 core id : 0 cpu
cores : 1 apicid : 0 initial apicid : 1 fdiv_bug : no hlt_bug : no f00f_bug :
no coma_bug : no fpu : yes fpu_exception : yes cpuid level : 11 wp : yes flags
: fpu de tsc msr pae cx8 cmov pat clflush mmx fxsr sse sse2 ss ht nx
constant_tsc nonstop_tsc pni ssse3 sse4_1 sse4_2 popcnt hypervisor bogomips :
4533.49 clflush size : 64 cache_alignment : 64 address sizes : 40 bits
physical, 48 bits virtual power management:

~~~
timewasted
I just rebooted one of my Linodes, and I got this:

    
    
        processor       : 7
        vendor_id       : GenuineIntel
        cpu family      : 6
        model           : 45
        model name      : Intel(R) Xeon(R) CPU E5-2650L 0 @ 1.80GHz
        stepping        : 7
        microcode       : 0x70a
        cpu MHz         : 1800.059
        cache size      : 20480 KB
        physical id     : 0
        siblings        : 8
        core id         : 0
        cpu cores       : 1
        apicid          : 0
        initial apicid  : 13
        fdiv_bug        : no
        hlt_bug         : no
        f00f_bug        : no
        coma_bug        : no
        fpu             : yes
        fpu_exception   : yes
        cpuid level     : 13
        wp              : yes
        flags           : fpu de tsc msr pae cx8 cmov pat clflush mmx fxsr sse sse2 ss ht nx constant_tsc nonstop_tsc pni pclmulqdq ssse3 sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes hypervisor ida arat epb pln pts dtherm
        bogomips        : 3600.11
        clflush size    : 64
        cache_alignment : 64
        address sizes   : 46 bits physical, 48 bits virtual
        power management:
    

Edit: I should perhaps mention that this is a 512MB Linode, on the off chance
that it makes a difference.

~~~
doppel
I rebooted mine as well and got @ 2.27 GHz, so I guess that they probably
haven't gotten to your server yet.

------
antihero
Hmm. Just rebooted and I'm on 8 cores but an L5520. Does this mean the CPU
upgrade is skipping my Linode?

~~~
radimm
> Linodes will start landing on NextGen hardware in the next week or so.
> Linodes on servers that are being retired will be required to migrate onto
> newer hardware. For those affected, you’ll receive support tickets with the
> details and with plenty of lead time (weeks). You’ll also have the
> opportunity to perform the move early and at your leisure if you prefer.

------
JshWright
I wonder if this means an upgrade to security procedures and host operating
systems?

------
derengel
A problem with linode is they don't charge by the hour.

~~~
josephb
<http://www.linode.com/faq.cfm#how-am-i-billed>

"This will issue a pro-rated credit to your account"

It's not per hour billing as such, but if you spin up Linodes and cancel about
X hours/days you do get the leftover fees prorated against your account.

