Hacker News new | past | comments | ask | show | jobs | submit login
Scaleway cloud adds AMD EPYC instances (scaleway.com)
149 points by nirv 12 days ago | hide | past | web | favorite | 57 comments

I've written extensively about Scaleway on Twitter, and not good stuff.

- Their network ranges are very likely to be blacklisted by essentially everything that employs blacklisting.

- The disks/volumes should be treated as ephemeral. There is no redundancy.

- Their SLA is a joke. I had a mailserver crash in the night and not booting, sent a ticket in the morning, resolved next evening. "SLA does not apply because problem was not a power failure". There is no SLA anywhere in their T&C, it's just mentioned on their marketing site without any detail.

- Turning off systems takes 30+ minutes. They mirror the entire disk to their storage system (very slowly) and free the entire machine (and you're still billed for it). This even applies if you resize the machine or add a volume. Turning on systems is a bit faster, but still in the 20+ minutes area. Oh, and you have a chance of not being able to boot again because although yes, you are billed, they're not reserving a machine for you.

- Can't pick own kernel on some machines. Arbitrary restrictions on number of volumes / addon services that aren't documented anywhere. They tell you when you try to hit save.

- Their network stack is crap. No machine gets an assignable public IP. If you attach an IP you still have the private IP on the NIC, but some firewall NATs traffic to you. It's also just slow in general.

Yann Leger speaking here, VP Scaleway in charge of the Cloud Compute business since 2014.

First of all, thanks a lot for your feedbacks, we greatly appreciate you taking time to provide details about your experience with our platform!

I would make sure to understand the problems you are describing as most of the issue you faced seems to be related to our first generation of platforms.

I will try to provide a factual update of the situation: 1 - Are you speaking of SMTP? We have suffered a lot from fraudster in the past (~2017) but we have now more strict process toward SMTP opening and a complete team dedicated to fighting abuse. 2 - That's totally true for all generations prior to the General Purpose range. We do not guarantee local data and our users are responsible of doing backups. The new General Purpose Range is backed by replicated local storage but this doesn’t mean we can 100% guarantee the storage as simultaneous SSD failure do happen. We now have a backup feature and support hot snapshots, which means backup can be performed with no downtime. 3 - The answer you’ve got to your issue is inappropriate and abnormal as we provide refund toward outages and a power failure is of course an outage. I would like to further investigate if you can share more information in private. 4 - I’m not sure to follow you here. If you archive your server, we will indeed save your server to long term storage. You are only billed for the storage when your server is archived, not for the machine itself. A new server starts in about a minute but servers with large volumes might indeed take longer as we do copy your data locally on the hypervisor to reduce 5 - Our C1 and C2 BareMetal server indeed have a network served kernel. On all new server range, including our START, GPU and General Purpose instances, the kernel is locally installed and is the distribution provided kernel. 6 - There is no firewall NAT but a stateless NAT system which lets us transparently migrate IPs between servers in one API call. We have no offer providing less than 100Mbit/s with one of the largest network backbone capacity in Europe. Our network map is available online (http://netmap.scaleway.com/), we have an open peering policy and over 5Tb/s of network capacity. If you have specific destination with bandwidth issue, I would be happy to let our network team investigate as I’m pretty sure we have no internal congestion.

Source : I'm managing the Compute Scaleway business since 2014, so I'm aware of the issue we faced in the past and which one are now resolved.

I’m available on twitter (@yann_eu) if you’re available to discuss the issue you faced as I truly believe our last platform generations solve the issue you encountered.

I've had similar issues. Network outages, server power cycles taking forever etc. The only good thing Scaleway is for is hosting Tor exits, which they don't seem to mind.

To add to the network stack comment - they assign _single_ IPv6 address to your server. Not a /64 block, just a single /128 address. I have no idea what they were thinking.

We were basically thinking about routing /64 blocks over the /127 interconnection to provide routed /64 ;)

> To add to the network stack comment - they assign _single_ IPv6 address to your server.

Also, that single IPv6 address changes if you shut down then start up the server again. eg it's not a stable address, like the IPv4 one is.

Can be a problem, until you figure out what's going on. ;)

This mirrors my experience as well. The 30 minute shutdown was a particularly nasty thorn in my side. I really wanted to like them and spent over a year trying to use them.

To add to your list, their C1 instances do not support IPv6. And never will because something something routers incompatible. It’s a shame.

Nice. The only thing keeping me away from Scaleway are several bad reviews about Online.net network. Anyone got any experience on this?

Their arm offerings are nice but unfortunately they are always out of stock. When I contacted support to ask if they have plans to mitigate this, their full response was "sometimes we are victims of our own success" (nice, good for them, but some real information would be useful).

Another major problem for anyone considering Scaleway is several reports of not being able to launch an instance from control panel, or completely bricking a working instance by simply restarting it.

I've been a customer of theirs for less than a month. In that time I've had several problems.

For example I rebooted a host, and it didn't come back. For 10 hours it was down. After getting in touch with their support I was told "Oh yeah, there is a problem with the hypervisor, we'll fix it". Meanwhile their status-site showed "zero problems". (The next day it came back.)

Provisioning a stock (Debian) system results in a host with no working serial console, which makes it hard to rescue.

You can't make outgoing SMTP (25/587) access without giving them copies of passports, etc. Though if you enable IPv6 you'll soon discover outgoing SMTP works ;) Only downside there is you can't set reverse DNS for your IPv6 address.

(Reverse DNS? They require the forward address to point to you before they'll let you set it, as a "security measure".)

I'm using the host for offsite monitoring, but we'll see if it stays. Cheap, but perhaps too cheap.

> (Reverse DNS? They require the forward address to point to you before they'll let you set it, as a "security measure".)

Many hosters I've dealt with have required this to prevent abuse from someone reading the rDNS and assuming the IP address is related to someone it is not.

I've heard this argument before, but it doesn't make a lot of sense to me.

(Perhaps more annoyingly there is no ability to set reverse DNS for IPv6 the address allocated to a server.)

> You can't make outgoing SMTP (25/587) access without giving them copies of passports, etc.

That's odd, I simply opened a ticket and they enabled it for me.

I got in touch and their reply was as follows:


According to a security issue about your account, we would like to verify your identity to unlock the situation.

For this we will need you to send to this mail address XXXX the following document :

If you are are an Individual , we invite you to send :

        * A copy of your ID
         * A picture of yourself holding your ID
If you are are a company, we invite you to send :

        * A business Certificate
We will process your document as soon as we received them and will make contact with you.

Why asking you these documents ?

As you know some swindler steal the identity of other people and use their credit card number getting them in unscrupulous way. By asking you these proof, we will be all protected from these people.

The process link to this identification request as a goal to fight against fraud and unpaid bill.

All the document which will be sent will be received by departement taking care of fraud.


Hm, I wonder if it's some fraud heuristic situation where depending on your "risk level" they'll either approve/request more into.

It's interesting how most of the companies where I've heard of them asking for ID/passport (e.g. Hetzner, Online.net, Scaleway) are European.

Most American companies (e.g. DigitalOcean, Google Cloud, Vultr) will let you just create an account (often with free credit) and do whatever you want.

ID for outgoing SMTP is anti-spam measurement. Many other cloud have their IP blacklisted by different spam list.

Other ID works too. If you are using a credit card, just ask their support

I've been a customer for 3 years. During the first year, I had some annoying problems, like instances that wouldn't restart (got "frozen") or disks that wouldn't attach to my instance. But their support fixed those issues a few hours after creating the tickets.

However, for the last 2 years or so, I've had no issues at all and the platform seems to be evolving constantly. Their prices are insanely competitive.

If you learn just a little bit of Ansible and Terraform, you can easily setup a Kubernetes cluster there.

I can totally recommend them.

Disclaimer: just a happy customer :)

I use them to run ArchiveTeam[1] projects (which consist of downloading data and then uploading it again, usually to the Internet Archive). For that purpose they work very well.

When I initially set up my servers a few years ago I did have an issue where I was getting randomly corrupted packets on a few of my instances. I spent about a week going back and forth with their support about it and they eventually determined it was a hardware fault with the VM host and had me migrate them to a different physical server. I haven't had any problems with them since.

[1] https://www.archiveteam.org/

> When I initially set up my servers a few years ago I did have an issue where I was getting randomly corrupted packets on a few of my instances.

I had the exact same issue, and found at least two reports of similar issues on public sources. This was very frustrating, as configuring the server over SSH was basically impossible since the sshd would drop the connection as soon as it detected a corrupt packet (invalid MAC), resulting in dropped SSH shell every couple of minutes.

I contacted support multiple times. I even pointed them to resources showing that this was likely an issue with their hardware, and some other guy had had the issue resolved by moving the instance to a different rack. Their support's reply was basically telling me to try to re-start the server as many times that it would spawn on a working rack.

Needless to say I had very little interest to continue working with them after this experience.

One thing that is annoying about Scaleway is that their underlying infrastructure relies on niche things like NBD storage, in-house odd ball booting, etc. So, if you want to run something other than their fixed list of Linux distros, you're in for a lot of hurt. BSD variants are especially frustrating there. Even if you get it working, they will change something to the point where you can't reboot.

To be fair, their standard images work fine, and you can't beat the price.

Their modern VPS instances use regular local drives. There is a local boot option too… but it's broken for now on the ThunderX ARMv8 instances. The docs say "will be enabled in a few days" but they still haven't done it. My FreeBSD/aarch64 install can't boot right now :(

I am a happy customer of their cheapest baremetal arm server since 2015. Maybe they could publish the number or available servers of each kind so that clients could organize accordingly.

> bad reviews about Online.net network. Anyone got any experience on this?

I have never had any network issues.

I use Scaleway Amsterdam-based instances for more than two years. I used to run about a dozen of permanent servers: from small ARM-based, to bare-bone, to "X64 Pro Workload intensive" VPS (which are now apparently replaced with EPYC); and countless amount of short-lived instances. I did have the only single serious issue when production server was completely gone, including LSSD data, due to underlying "hypervisor critical failure". Unfortunately, I had no server snapshot that time, but that's another story and it's on me.

> reports of not being able to launch an instance from control panel

They do occasionally (quite rarely) have minor issues. Just recently I was unable to remove their "X64 Pro" instance neither within web interface nor scaleway-cli client. However, this issue was resolved within a couple of hours after reporting the ticket (with no paid support).

In general, I am very pleased with the Scaleway, considering the quality/price ratio. If only they'd have servers in Scandinavia/Northern Europe region, I wouldn't look for other offers such as Hetzner/Helsinki.

I confess I wonder if the switch to EPYC is cost based or speculative execution bug based.

> Nice. The only thing keeping me away from Scaleway are several bad reviews about Online.net network. Anyone got any experience on this?

I never had any issue with my Online.net servers (one at a time, but I upgrade it from time to time) nor my Scaleway one.

> When I contacted support to ask if they have plans to mitigate this, their full response was "sometimes we are victims of our own success" (nice, good for them, but some real information would be useful).

The two datacenters they use for public hosting (DC2 and DC3) are full. They recently finished building a new one (DC5) and commissionned a first room a few months ago.

I've been customer of Online, since it opened back 10 years ago or something (was called dedibox back then), and customer of scaleway since beta.

Bandwidth has always been vastly superior to any competitor, not sure if this has something to do with the fact that the mother company Iliad also contains a famous french ISP (Free). Not to mention another sister company that sells domains without making a marging on it.

I have restarted thousand of instances, never had a problem that was not a PEBKAC. Actually, I even have a production site there since the beta and never had a problem.

online.net themselves (proper dedicated servers) are fine, and their network is reliable, fast and they have good uptime.

Scaleway's hardware and software is a huge problem, it's unreliable to the point of being unusable. I'd only use it for cheap hosting of build server workers. Their disks for example have no RAID, you're expected to mount multiple volumes and perform software raid yourself, except that there's no easy way to do that and also you often run into limits for how many disks you can attach to a VM!

The customer support has sucked most of the times that I have been in contact with them.

The C14 cold storage product seems to have a ton of bugs in it, like how you'll open up an archive and you can't connect to it for an hour.

I already moved my small servers to Digital Ocean and I'm thinking of replacing C14 with Glacier.

It's not a bug, it's a feature, you just don't understand what C14 is and the problem it's trying to solve.

It's not a feature. Once the archive has been created for the first time, you should be able to connect to it immediately once it says "Available", not an hour after you created it and it went to status "Available".

Having just switched from Online.net to Hetzner, I vastly preferred the Online.net service.

However they were none too pleased when my machine Tor Exit node was listed in some email spam list, so I had to move.

Besides the comment I left... I've seen successful ARP poison attempts and lots of junk broadcast traffic (people running PLEX) on Online.

> full response was "sometimes we are victims of our own success"

A very French reply

active customer for like 2 years, no issues with my 3 euro/mo instances

I have a Scaleway VM and the networking has lots of trouble. Would not recommend.

It's difficult to host a Tor hidden service on Scaleway because half the time the clients just can't connect. I have not figured out why this is.

I also find that my SSH session sometimes just randomly drops. It's nothing to do with keepalive. Sometimes I can leave it for a few hours and come back to it, sometimes if I look away for 10 seconds, the SSH session will be gone when I come back.

EDIT: Here's a chart showing my findings for Tor hidden services: https://img.jes.xxx/1940 the cht1 machine is the one on scaleway. The red charts show the proportion of requests which failed, you can see for cht1 this is normally more than half.

Does anyone know how Scaleway compares with other providers such as Hezner or OVH or Digital Ocean?

I've used the cheap VPS plans from OVH, Hetzner, and the 'baremetal ARM' instance from Scaleway (c1). This is not exactly an apples to apples comparison and all very anecdotal but:

OVH would typically have one extended outage a year, usually related to something like an excavator severing fibre cables or diesel generators not starting in a brownout. The supporting infrastructure was nice, you could get e.g. free primary DNS hosting for external domains, and the domains they sold were cheap too. RESTful API was nice. SSD performance & storage space (bumped up only recently) was on the low side. Quite a few datacenter locations to pick from.

Scaleway 'baremetal ARM' was a mixed bag. I didn't like the ipv4 NAT and related networking, they took a long time before you could use your own kernel (a feature promised at launch), crypto acceleration module on their Marvell ARM processors (CESA) didn't work for a long time due to a borked devicetree and after it did support was vestigial (had to patch and maintain your own libraries etc). Networked SSDs were a bit of a pain and a kernel update once broke them leaving all my instances on recovery console unable to mount anything... Performance was meh. It was the cheapest way to have a dedicated instance though.

I've been now on Hetzner for about two years and I don't have any particular complaints. It's solid. There was one bigger outage last year but didn't affect my datacenter. Pricing is very competitive, API is functional, processor crypto extensions (AES-NI) are exposed through the KVM (I think this wasn't the case with OVH but perhaps changed now). Only two datacenter locations (DE and FI). Cheap snapshots and backups. I barely ever need to log into the management console. HTH.

Thank you for posting such an informative reply. Kudos!

I've used OVH for several years (a few small Cloud VPSes). Never had any outages, but their web interface is very complex and unintuitive. I left because they messed up billing, then immediately threatened to delete my servers within hours of their mistaken billing (my creditcard was charged but their systems somehow didn't register it), then was charged a second time. They refused to acknowledge the error (even with screenshots, I had to do a chargeback instead). And their support took 2-3 days to respond to tickets.

I'm now a happy Hetzner customer since half a year. Support was quick, interface is simpler, and their small VPSes are even a bit more attractive.

Cheaper GPU offering for one. Feel free to add.

I tried Scaleway a year ago, and it was almost impossible to just set up the firewall due to some low level kernel bug.

Who would use hosting where firewall is insanely hard to set up?

A friend of mine went another route, he built the infra first and wanted to secure it after. He hit the same problem. Firewall was impossible to set up.

Not to mention before I figured this out I had to recreate 5+ instances because it was locking me out. I will not use Scaleway in the future.

Love how their extra small is 4 cores and 16gb ram (about $45/mo).

They have other products that are much cheaper. I’ve been running a small server there for 3€ for a few years.

Their cheapest instance is actually just 0.5EUR/month if you don't need a public IP.

The cheapest one I can see is 1.99EUR/m [1]. Maybe that's an old plan?

[1]: https://www.scaleway.com/pricing/

Both of:

* The pricing page includes a public IP for all instance types

* It's an old plan they don't offer anymore

So the cheapest one at the moment is START1-XS, which is 1EUR/month without public IP.

They're actually better than anyone else I know in terms of their cheap instance (around $5/month).

What is the difference between BareMetal C2M which offers 8 cores, 16GB memory, and 50GB of SSD storage for €17.99 vs GP1-XS which offers 4 AMD EPYC, 16GB memory, and 150GB of storage for €39.99? Isn't the BareMetal better because it is dedicated, yet it is less expensive?

Ah thanks. Since my workload is more memory bound, might be ok with Atom.

Worth noting: ovh have stunningly good rates for RAM, on dirt cheap instances, and you just have to accept there'll be unplanned reboots and their support is, well, they're dirt cheap, it's minimal.

For things that suit those constraints, they're amazing value for money though.

Ovh "VPS Cloud RAM 3" is 30GB for €30 exVAT, is it cheap considering e.g. Hetzner's bare metal EX42 with 64GB for €34 exVAT?

No one beats Hetzner. Hands down

I used both Hetzner and Scaleway. I trust Hetzner way more tbh. With Scaleway we had several issues other commenters also mentioned: instances not rebooting, random freezes, instance types out of stock. In the meantime we have both bare metal and VPS servers on Hetzner that have years of uptime.

I love Hetzner but you have to compare like for like. The EX42 is a dedicated machine which is great for perf and HD space but bad for maintenance, flexibility and setup fees. It’s also desktop hardware so no ECC memory.

Hetzner does have cloud vps though e.g. CX51 32 gb 8 vcpu at 29 eur ex vat. So a bit cheaper but not as much DC location choice as you get with OVH.

The C2M appears to use an Intel Atom chip: https://denbeke.be/blog/servers/benchmarking-a-baremetal-sca... (which are not known for their raw performance)

The Avoton, though, isn't a typical Atom. It's not a high end Xeon, but is closer to them in performance than the anemic Atom processor most people think of. I think it was a disservice to call the Avoton processors "Atoms". At the time they were released, they were faster than, say, a 5 year old Xeon.

About 60 weeks ago I had to apply a mailing list patch on top of the most recent GCC master prior to the patch, so GCC would know -mnative for them. It was on the branch that is becoming GCC9. If it wasn't backported, I'll wish you luck getting decent perf out of them. I recall a double-digit boost in CPU perf compared to GCC7 at -O3 when only allowing the vectoe instructions of Avoton, but not being able to get full instruction scheduling optimization.

They are not slow though, I use them for e.g. irssi/toxic bouncers and such. Uptime is sufficient, if you have backups as you should have.

do they support running your own kernel (or even one from a distro) yet?

Applications are open for YC Summer 2019

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact