
The Great Data Flood Ahead - SemiTom
https://semiengineering.com/the-great-data-flood-ahead/
======
devicetray0
> While data in the cloud is heavily secured, data on the way to the cloud is
> not.

This article is nonsense and completely pretends that TLS does not exist.

------
eesmith
The use of "flood" reminded me of a few years ago when I was researching the
post-war information technology era. What struck me was how many papers used
phrases like "flood" or "deluge" of data.

For examples:

> The problems of adequate storage, preservation and service for the
> increasing flood of periodical literature coming into their collections are
> of special urgency for librarians. Many studies have been made, all of which
> view with deep concern the rapidly increasing rate of growth of American
> libraries.’ Such growth, if continued even at the present rate, will in a
> short time result in collections of almost unmanageable proportions, both as
> to physical size and servicing.

("The Use of High Reduction Microfilm in Libraries", J. Am. Doc. Summer 1950 -
[https://search.proquest.com/openview/14f723869613e43376c4a76...](https://search.proquest.com/openview/14f723869613e43376c4a7646f583183/1?pq-
origsite=gscholar&cbl=41135) ).

> With the advent of the IBM card programmed calculators, actual calculating
> time on the data was materially diminished, leaving the problem of reading
> and processing the data standing as a very real bottleneck. It therefore
> became evident to responsible personnel concerned that a system would have
> to be devised that would allow either automatic or semi-automatic processing
> of much of the data incurred at the Air Force Flight Test Center if the
> Center was to survive this deluge of data.

("A centralized data processing system", 1954,
[https://dl.acm.org/citation.cfm?id=1455227](https://dl.acm.org/citation.cfm?id=1455227)
)

Even "deluge of data" is still common, says
[https://duckduckgo.com/?q=%22deluge+of+data%22&t=ffsb&ia=web](https://duckduckgo.com/?q=%22deluge+of+data%22&t=ffsb&ia=web)
.

65 years later and the data waters keeps rising.

------
bob1029
I don't see any need for action here. If Amazon runs out of storage space or
bandwidth for keeping track of everyone's botnet kitchen appliance statuses, I
don't think the connected world will grind to a halt. We will simply begin
dropping said fridge temperature status updates on the floor in favor of more
important things like bank transactions. 99% of data today is garbage from the
microsecond it was created, and virtually all of it is garbage after a few
months elapses.

That said, I am not against new iterations on the idea of the internet and how
we move data from point A to point B. Enabling a vast 'ocean' of a trillion+
devices to send data easily to/from any node is a compelling problem (which
we've arguably already solved). Trying to agonize over whether the data is
important or not in some overarching manner is not a compelling problem. Data
is very ad-hoc in nature. I don't see any other way about this if you want to
maintain its utility to everyone. You don't apply QOS rules to payment
processing networks in the same way you do the xbox live network. Both parties
would argue their data is very important, but neither party cares at all about
the other party's data requirements.

------
rdlecler1
The problem is that data is going to be walled off. There will be a lot but it
will be siloed and inaccessible.

------
caust1c
This is a pretty vapid article. Surprised to see it on the front page.

------
antisthenes
If the costs of storing data rise faster than data is generated, people will
simply be more selective with what data is retained permanently.

Most data is junk anyway, and is only useful in the immediate or short-term.
It's also somewhat pointless to debate without talking about a specific type
of data. Some of it is highly compressible and is a non-issue.

------
noobiemcfoob
My summary of the article: Most data is or will soon be trash. Efforts should
be made to get to that designation as fast as possible from the moment of data
generation.

------
fabiofzero
This is an information-free article. There's been a flood of those popping up
these days.

