
Upthere, a cloud storage service, wants to make file syncing a thing of the past - dannylandau
http://www.theverge.com/2015/10/29/9632904/upthere-cloud-file-storage-bertrand-serlet
======
kylec
I'm a bit skeptical of this. The way the CEO brushes off the issue of what
happens when the user is offline leads me to think that they don't have a
great answer for that. Services like Dropbox and iCloud are great because you
don't need constant connectivity to use them - with Dropbox you always have
all your files on your computer, and with some exceptions (Photos) the same is
true for iCloud. This company might be living in a Silicon Valley bubble of
perfect connectivity if they think that people always have a fast and reliable
internet connection.

I'm also interested to see if they have a way for you to run your own Upthere
server instead of using their service. I'm not eager to have to trust yet
another third party with my private data, and if I can't host it myself
somehow I'll probably avoid using the service.

~~~
api
He talks about caching. As soon as you add caching, you now have a problem
that isn't just close to syncing. It's utterly, absolutely identical to
syncing. Syncing is just distributed caching where you only ever invalidate a
cache entry when someone tells you to by deleting the local file.

You can "skip" the cache consistency problems by "just" always treating "the
cloud" (which is internally probably a distributed cache) as the authority,
but now you have an unreliable distributed system from a user's point of view
since the thing you are skipping is how to handle real-world network weather.
You can't eliminate problems by pretending they don't exist.

You also can't skip this problem with "the cloud." I am SO damn sick of "the
cloud" being talked about as magic beans. If you're building a global
distributed drive, then you are doing a distributed cache -- SYNCING -- in the
cloud between cloud nodes. If you can do it there, why not do it on the end-
user device too and let me cache my data? If you are using someone else's
boxed cloud service to do this, then you are letting them build a distributed
system and operate it for you in the cloud and if you ever have problems with
it you better hope they have good support/SLA... and you're also doing
something trivially duplicated. If that's what you're doing why don't I just
use s3fs?

I sincerely hope there's more to it than this. I can trivially do this now
with sshfs or Expandrive if I want a pretty GUI, and it's only tolerable with
cloud storage that's physically very close to me (about <500mi).

~~~
stuntprogrammer
I agree with many things you say. Take AWS S3 as an example: it reduces the
problem by making objects immutable. Internally, they can cache aggressively
without scaling fine-grained consistency on the objects as such. As an S3 user
I can aggressively cache on the client too.

Now we've reduced the problem to consistency on the metadata structure which
aggregates objects for the user. There are "well known" ways to do this for
traditional trees. Other well known options include doing an all search based
approach (i.e. always talk to a server for metadata, perhaps with local result
caching), and so on.

~~~
api
The last part you wrote is what I was getting at: you cannot eliminate
distributed systems problems from distributed systems. You can only either (a)
push them around into different parts of your design or (b) outsource them by
using stuff like Amazon's DB-as-a-service stuff. Using immutable objects is
(a): now you have a distributed metadata problem.

If they're doing (a) then it's just another Dropbox but with less tolerance
for disconnected operation. If they're doing (b) they're just reselling and
maybe with a nice client. That's not doomed as a business of course but it's
not all that technically interesting.

If there's a (c) it's not in this article.

~~~
stuntprogrammer
I strongly agree with you re 'conservation of distributed systems problems'.
I'm being vague because I know what they're doing, but I'm limited in what I
can comment on.

Their public comments are that they are running their own servers and not
reselling other storage.

Personal opinion: no one can do (c) because of (a). That is, any possible (c)
must tackle the fundamental hardness in the distributed systems problem, and
this is what we agree (a) is doing. Using immutable objects as in S3 just
shifts the problem elsewhere, while it reduces it, it doesn't solve it.

~~~
api
Immutability does reduce the problem by reducing the data footprint of stuff
that needs to be synchronized.

So let me guess:

They're doing a distributed _metadata_ store where the cloud is the tie-
breaker combined with flexible caching of immutable content-addressable
(identified with SHA512 or similar) objects.

In that case it sounds like cloud-hosted-only ipfs plus a nice client to
access it from a host.

Cold? Warm? :)

------
blixt
While the solution going forward is most likely to go back to dumb terminals
(thin clients, if you prefer) I think just putting it in "the cloud" as it
stands today is not the solution. Granted, it's not very clear what they are
offering, but it does appear to be a gated service (think Dropbox) rather than
an open technology (think HTTP). It just doesn't seem to solve problems any
better than what's already out there, but I'd be happy to be proven wrong.

Personally, what I'd like to see is mainstream software bundled with
[https://ipfs.io/](https://ipfs.io/) technology. If content could be served
via a DHT plus all the extra things IPFS offers, we'll be a lot closer to a
world where content doesn't get lost. At that point I'd gladly pay a company
to store my personal files (by simply giving them a list of hashes which
resolve to my computer so they can download them), and trust that it's safe
because they were encrypted on my end and the company is just ensuring that
the content can be redownloaded should my computer explode.

On a sidenote, IPFS would also solve a lot of problems with public data, where
everything can be reliably synced (and even updated with IPNS) for people with
no persistent access to the Internet (and people on space stations or the Moon
or Mars, should we ever put bases on them).

------
niels_olson
This seems to be a recurring theme: Unix (and Windows) use files. Keeping
track of files is hard. If we could just use enough Unix (internet, mobile,
the cloud, oh my), we could make files go away.

And then pictures, video, music, etc come along and wreck everything.

Maybe there are better machines to rage against than a hippie OS.

Also, I wonder if they've done the math on how much data you can actually cram
into the airwaves. You can't instamagically move more data through a finite
spectrum at rush hour on a Wednesday.

~~~
Natanael_L
They're essentially making end user devices into thin clients. Which I'm
actually a fan of, _if only_ I get to run the backend on my own server
hardware.

We badly need standardized home servers that can run 24/7 and managing storage
and processing tasks for our mobile device. It would solve sooo much of the
problems derived from unreliability of mobile devices.

~~~
valine
I personally love this idea. Its a mini cloud running out of your home and it
puts you in complete control of your data. I would even take it a step further
and say that as long as you have a home server running 24/7 why not run a mail
server as well? It would resemble something like an apple tv. Small, quiet,
power efficient. You plug it in, run through a simple setup wizard, pay ten
bucks for a domain name, and off you go.

~~~
Natanael_L
A mix between a basic game console (see Steam machines), router (see OnHub)
and a plug computer could work, maybe.

Something that the regular users have an incentive to buy, because the primary
advertised features are something they need or really want, and that then also
offers a wide variety of additional services, such as mail, VoIP, coalescing
proxy for your mobile devices to save their battery, storage, etc...

------
jamescun
The Verge, TechCrunch and their website, and I still have no idea what their
product is. They have an impressive team, that has been put across. Other than
that, just buzz words.

~~~
jozzas
Yeah, I must be missing the point as well.

"Upthere says it plans to offer an API in hopes that developers will make it
the default storage solution for their own apps"

So... how is that in any way different to using the AWS API to upload stuff to
S3? Being a "storage solution provider" (and reselling other companies' cloud
offerings at that) has some razor thin margins and a whole lot of competition.
I don't understand what makes these guys different.

~~~
inframouse
They talk about running their own hardware so assume something like open
compute or backblaze. But without the size or reliability of S3. Can't imagine
they'll be able to do it at lower cost than Amazon either. Like you I have no
idea why not use S3.

~~~
jozzas
Like you, I can't see how they'll compete on price if they own their own
hardware. Microsoft, Amazon, Google, Facebook have all been working on this
stuff for years. Even adopting Opencompute or similar, they will have
comparatively no buying power, no users and no data.

------
eps
Woah, getting a strong whiff of "we will XML everything" from the late 90s.
Lots of big words, grandiose vision, but all reeking of technical
inefficiency, which when resolved will yield... a form of an SQL database. 4
years in stealth mode also don't instill much confidence that they will live
up to the claims.

Nice logo though.

------
cargo8
Even more info here [http://techcrunch.com/2015/10/29/whats-
upthere/](http://techcrunch.com/2015/10/29/whats-upthere/)

~~~
PhantomGremlin
They're doomed. From that link:

    
    
       The ground-up approach includes the entire stack.
       Hardware to software. The company has designed
       its own servers down to the bend of the metal
       and the motherboards used.
    

Why, oh why, did these guys think they needed to design their own
motherboards? I couldn't think of a more commodity product. You want to do
that eventually, then sure, do it. But not as a startup. Focus first on what
will differentiate you from everyone else. Hint: IT'S NOT THE FUCKING
MOTHERBOARDS!!!

The founder spent 22 years working for St. Steven, and yet he didn't learn one
of the most important lessons:

 _“People think focus means saying yes to the thing you 've got to focus on.
But that's not what it means at all. It means saying no to the hundred other
good ideas that there are. You have to pick carefully. I'm actually as proud
of the things we haven't done as the things I have done. Innovation is saying
no to 1,000 things.”_

~~~
MichaelBurge
It says they designed the servers rather than the motherboards, so I'm
guessing it just means they shopped around for different off-the-shelf
motherboards rather than, say, hiring hardware engineers to lay out gates and
chips.

~~~
stuntprogrammer
Correct

------
neilellis
I signed up because, hey I'm curious. But ...

It's hard not to think of this: [http://mashable.com/2012/10/17/color-shuts-
down/#](http://mashable.com/2012/10/17/color-shuts-down/#)

High-concept stuff doesn't seem to fair well, the small and growing always
seems to do a lot better. That said good luck to them.

------
willyt
A little bit of feedback: Quite large parts of the UK still have patchy 2G
reception so this would not be practical for here. People on HN have been
saying don't worry the network will catch up for years, but where I live the
network coverage and speed is crap and hasn't changed at all in the last 5
years in the 80 miles between here and the nearest major city.

~~~
TeMPOraL
I really, really wish startups stop assuming everyone has unlimited data plan
and broadband Internet available 24/7\. I'm not even begging for the common
engineering sense anymore (don't route around half of the globe the data that
has to travel only 5 meters in physical space; some data does not belong on
the Internet, etc.), or even common decency. For practical reasons and the
love of God, stop with the requirement of being constantly on-line to use a
goddamn app.

~~~
voltagex_
I really need to register bandwidthprersevationsociety or something a bit
shorter. I was initially going to say that developing nations are going to be
left behind but that's not right, a lot of them have very good high-speed net
access. Australia on the other hand - a lot of places are going to have a
maximum of 25 mbit / 1 mbit for a long time due to the geniuses running our
government.

I'm on what is effectively 8/1 and I'm starting to struggle with the size of
Windows Updates (8->8.1 failed multiple times resulting in a repeated 2GB
download), XBox One games (20-40GB), streaming and RDP/Citrix/VPN for working
from home.

> I'm not even begging for the common engineering sense anymore

OneDrive! It doesn't even implement any kind of LAN Sync - you'd think MS
would have bolted on some of the newer SMB stuff like HomeGroup (which is
discovery+SMB over IPv6 AFAICT)

>For practical reasons and the love of God, stop with the requirement of being
constantly on-line to use a goddamn app.

Ubisoft managed to lock the whole of Australia out of one if its recent betas
with this mentality.

~~~
Natanael_L
"Data locality preservationists" :)

~~~
TeMPOraL
I love it <3.

~~~
voltagex_
savethebytes.io? (and can you or anyone else chip in for that domain?)

~~~
TeMPOraL
Count me in. I'm shooting you an e-mail.

~~~
Natanael_L
Don't forget me! I'm sending you an email too

~~~
niels_olson
I like it. Maybe a tie in with slow food?

~~~
voltagex_
Well we certainly have slow internet here. It'd be a good joke/political site
to do a "slow internet movement" in the style of the "slow food movement"

------
lewisl9029
Syncing will eventually be a thing of the past for most people, but I imagine
what will make syncing obsolete is not cloud storage alone, but rather cloud
storage in combination with the emergence of a new class of portable devices
that is compact and efficient enough to carry around for personal
communications, powerful enough to get real work done quickly, and flexible
enough to support large displays and VR for media consumption and gaming: a
single device that literally does it all.

Cloud storage will still be important in this new single-device computing
model, but only as a means for archival and backup. Sync won't come into play
anymore because you only have the cloud and a single device to manage, and
changes will tend to only propagate in a single direction from the device to
the cloud.

~~~
Natanael_L
What I'm imagining is a "wearable server", an "anchor device", that power all
your screens and input devices. Maybe you've got "smart clothes" with
communication and power lines embedded so it can communicate with your smart
glasses, or it talks wirelessly with your smartwatch, etc.. Doesn't really
matter how exactly it works, just that it manages your connectivity and holds
your local data. Maybe you have it in your bag, or on a necklace, it it _is_
your necklace, or on your wrist (maybe it IS your smartwatch) etc...

------
zomg
i was so unsure of what they do, i signed up for their beta!

------
inframouse
Reading through the spin.. it's just an object store and pubsub with some demo
apps? Like S3 and SNS on Amazon or Google Datastore and Cloud pubsub? That's
it? Why would anyone use this instead of AWS or Google?

------
josu
Syncing everything to the cloud may be great if you have an unlimited plan,
but as long as each GB costs around $10USD these kind of initiatives will have
a very limited impact.

------
api
So is this now officially the "mainframe 2.0" era, with endpoint devices now
being dumb terminals again?

~~~
jozzas
Pretty much, but large multinationals are the only ones that own the
mainframes, and it's only cost-effective to have a few very large providers.
That's the way it's all moving.

------
chatmasta
> "NO ONE DID WHAT WE ENVISIONED. SO WE STARTED FROM SCRATCH."

Yet you were the "senior vice president of software engineering for Apple."
You were in a better position than almost anyone else to "do what you
envisioned." What stopped you? If you could not do it at Apple, why will you
be able to do it at a startup?

The ability to start a startup is a new bargaining chip for high value
corporate employees. Why build an innovative feature at your company -- and
give them all the money -- when you could just leave and do it yourself, and
keep all the money?

There is a moral hazard for corporate employees that poses a large risk to
corporate innovation. If an employee at a company stumbles upon an innovative
opportunity, it can be economically sensible to for the employee to avoid
implementing it, in preparation for a future startup that will attract easy VC
money.

The best solution to this is probably just for big companies to grant more and
more stock to employees to keep incentives aligned.

~~~
mattzito
Well, because starting a company is a huge risk with a very low percentage of
success, even with an awesome idea. Some ideas will require corporate backing
to be successful, either because it is more of a feature than a product or
because you need to leverage the infrastructure of the corporation.

~~~
chatmasta
Way less risk when you are former VP of engineering at apple. That guarantees
at least enough investment to hire a team the size of what you had at Apple.
Presumably as former VP you also have the experience to increase chance of
success.

So no, not a huge risk, and higher percentage chance of success than anyone
else starting the same company.

~~~
mattzito
Some of the worst startup founders I have seen are people who come from being
execs or management at very large companies, because they have no frame of
reference for how things should be done when you have 5 people.

So I don't think you can generalize on that point. And as far as this goes:

> That guarantees at least enough investment to hire a team the size of what
> you had at Apple.

There's also no way this is a guarantee. Sure, it'll be easier to get money
than an 18 year-old walking in off the street will, but again, you can't
compare the steady job, steady paycheck, "let's build a feature internally"
with "let's go start a startup".

I will definitely agree that having the experience of a vp of engineering is
better than not having it, but that's a far cry from saying that quitting your
highly paid job as a VP of engineering for the biggest company in the world to
start one of many cloud storage companies is "not a huge risk".

