> evolution from “data as value” to “data as risk”
This jumped out at me in the blog post. IMHO "data as value" is 3/4 of the problem.
Storage evolution has made data hoarding cheap and the hoarding desire is mostly driven by "monetization" goals.
Those who bitch and moan the loudest about regulatory compliance are generally those who hoard the most for the least necessary reason.
Keep only the data you require and the amount that needs securing when compliance comes knocking is far more manageable than attempting to secure an ocean of guff.
> ...field-level encryption should be a starting place for new application development...
What does the state of the art look like here? Particularly when it comes to familiar tech like Postgres and Redis?
> Conversely, this will generally be a pain to work with ... (You can, of course, build something like a trigger to allow retrieving the keys within the database, but that would significantly undermine the value of this entire enterprise.)
Is this limitation inherent to the concept, or are there some creative ways to retain the benefits without having to lose the relational aspect of the DB?
* Some environments might be able to use a structure where a PII table is fully tokenized while less-sensitive data is more broadly available to developers.
* Postgres can be configured with row/column-level security.
* I believe https://basistheory.com/ offers the per-user/entity key setup. This is an additional service/cost but the UI is slick and easy to provision/remove access to specific fields.
Its like being in manufacturing in the late 60s, noticing the EPA is coming, and wondering how to continue operating.
When you can't work in a geographic location anymore, the answer has always been to offshore or close. Silicon Valley in 2040 will probably look a lot like Detroit does today. Remember that Detroit in 1960 looked a lot like SV now, correcting for inflation and technology, etc. Actually Detroit in 1960 was a much nicer place to live than SV now.
"Relatively soon" the concept of storing the general public's data for them, while officially or unofficially sharing with everyone for profit, will disappear from most countries.
My guess is some kind of weird legal fiction where I have something like an encrypted google drive that I "own" and google can't access, and everyone storing data about me is actually looking at documents they signed and encrypted that I store "for" them in "my" storage, although I probably can't even decode the data I'm storing for them.
By analogy, the easiest way for Citibank to extend me credit card available credit is to have a big list on their side of everyone they've ever extended credit to, which I'm sure many people would love to steal and sell for aggregated marketing data and similar nonsense. However, in theory, citibank could store nothing at all, and extend several thousand one dollar credits to me in the form of letters or documents and a software infrastructure could trivially prove my ownership of those credits. Kind of like block chain but none of that pesky privacy stuff nobody wants but the endusers and nobody cares what endusers want, so we're not going to get that.
This is how the paper coupon market worked back when people used paper coupons. They were pretty big with the WWII "ration book" generation, not so popular now. Back in 1981 I was never on a list at company HQ of dudes allowed to buy a can of Pepsi for 20 cents instead of 25 cents, I just had a paper token claiming five cents off.
Technically in the old days dental records were on paper in folders and you'd carry them from dentist to dentist as you move. Somehow I lost mine when I moved and the old dentist went out of business, those things happen, so I have no dental records from before I was 30 or so. Anyway you can't steal my records from the dentist if he doesn't have my records because my records exist solely on a flash drive in my pocket. Hopefully I'll keep backups but "the way things are" will have to change to tolerate some percentage of your customers losing all their data every year. Most data is trash anyway so not much loss. There's a lot of misplaced faith in data that my tooth records could somehow improve my QoL or make someone a pile of money, but IRL my tooth records are not in practice of any value and if a dentist wants to know if I have a filling he can just take a look at the tooth, its pretty obvious.
> Silicon Valley in 2040 will probably look a lot like Detroit does today.
I don't live within 2,000 miles of SF, so I have no dog in this race. However, the fact that SF has the most consistently pleasant microclimate of anywhere in the mainland US (climate change notwithstanding) means that there will always be major demand for living there, unlike in Detroit, whose original draw was as a center for trade on the great lakes and later as an industrial base.
I buy that more people prefer SF weather to Detroit, but how is it the best in the US? As you go South along the coast you pass many places I think most people would prefer (ex: Santa Barbara [1]) and even as far South as LA weather is likely something more people would prefer [2].
(Personally I'm very happy with Boston and like having seasons and occasionally getting snow, but I recognize that this is a minority view)
Yea that's what I mean - the same is true in the SFBA. SJ/South Bay, Berkeley/Oakland/Richmond, the rest of Alameda county, the peninsula, etc are all quite different just like the San Fernando Valley, Inland Empire, Westside, and Orange County near LA.
> Personally I'm very happy with Boston and like having seasons and occasionally getting snow, but I recognize that this is a minority view
Grandparent commenter here, yeah, this is why I also live in Boston. :P But averaging 70F every day year-round with consistently sunny skies is pretty appealing to those who don't need seasons to anchor their memories. It's the same reason why people will always choose to live in Hawaii, regardless of economics.
There are really two different groups of compliance programs: privacy compliance programs like GDPR and CCPA, and security compliance programs like SOC2, PCI and HITRUST. What's been happening over the last few years is that some of these security compliance programs like SOC2 or HITRUST are adding privacy concerns (usually as optional components) so that companies can do one audit for everything instead of doing multiple audits.
As I once heard a lawyer explain, laws, regulation, compliance are mostly like software: created by hardworking individuals to be as accurate, fair and efficient as possible. She mentioned that the primary difference compared to software development is feedback from compiler, automated tests, telemetry, user feedback, etc.
So, complying to CCPA,GDPR,SOC2,PCI,HITrust is like running on your code 5 different platforms, none of which were tested during construction.
This jumped out at me in the blog post. IMHO "data as value" is 3/4 of the problem.
Storage evolution has made data hoarding cheap and the hoarding desire is mostly driven by "monetization" goals.
Those who bitch and moan the loudest about regulatory compliance are generally those who hoard the most for the least necessary reason.
Keep only the data you require and the amount that needs securing when compliance comes knocking is far more manageable than attempting to secure an ocean of guff.