Hacker News new | past | comments | ask | show | jobs | submit login
Project Silica proof-of-concept stores movie on quartz glass (microsoft.com)
191 points by hongzi 13 days ago | hide | past | web | favorite | 102 comments





One of the startup ideas I've had is to modernize death.

Most humans want to "live forever." Given that we can't do that, we resort to procreation, and leaving behind some pieces of ourselves to be remembered by, with lots of cultures having some type of funeral practice to ensure this occurs (i.e. tombstones).

With the proliferation of our digital lives, many of us have more photos, videos, and other media than we know what to do with. Furthermore, we probably don't have a great way to preserve this information.

Imagine a funeral service that could comb through your digital life, archive it all, and keep it accessible to your family for generations. Bonus points for summarizing it into a beautiful montage to be played to your family at your service.

Financially, it would work a lot like how funeral homes operate with deposits, insurance, and annuities, but the marginal cost should be much lower due to it being a digital vs. physical product.

One of the problems I've been mulling over is the fact that digital storage isn't perpetual, which means that someone has to ensure the data is maintained in some way.

Therefore, something like the quartz glass storage medium in the article would be perfect. Your data would be accessible via traditional means for 10-20 years after your death. Afterwards, it would still be accessible, but it would be on cold storage, so it wouldn't be instantaneous. It would mean you'd "never really die" since any of your descendants would be able to see what your life was like.

I think that would be enough of a "hard problem to solve" to build a startup around. Of course, you could extend it to preserve other forms of data, like your DNA, or your mind like Westworld, where your descendants could communicate with you whenever you wanted.

It could easily become one of the first perpetual corporations.


> Imagine a ... that could comb through your digital life, archive it all

No, and no thanks. Every company already does that. :)

A better idea would be something like a GitHub or Dropbox option that makes your content public after your death.

Unfinished projects for others to take up, ideas you never got to work on, thoughts you couldn't speak out when alive, secrets you didn't dare spill, stuff like that.


I think gp's point is that nobody does this in an organized way that's not tied to a platform. When Facebook and Google eventually die, their archives will die too. You'd ideally have some format with the openness and durability of an ASCII file that could be stored in a single location and survive multiple storage medium migrations trivially; something like a photograph in a box, which anyone can interact with easily regardless of whether the camera maker is still in business. And while a GitHub or Dropbox sounds great, the issue is that most people don't have the technical ability to collate their data from all the places its stored, and their descendants also mostly won't have the ability to pore through it unless it's pithy and organized--and if that's the case, then that it just means that the deceased came up with their own ad hoc durable "life archive" data format.

Yes, it wasn't cool of me to cut them off at one sentence. Sorry.

Still though, instead of having a firm scour ALL your data, including things that you may not wish for others to discover, wouldn't it be better to let you choose what you want to be released after your death?

For example, besides marking select files/repositories, and setting my own criteria for releasing them (e.g. 1 year after I haven't logged on), I would have the option to notify specific individuals when my data is released, or even have it automatically posted on places like HN/Reddit/etc.

Other people would then have the choice of what to do with my data.

They could archive it, enshrine it, build upon it, whatever.


Mind uploads? A digital afterlife is one of the dystopian trolls of post scarcity fiction. Bonus points for providing a 'digital hell' for those whose transgressions merit it.

It would be much better to gracefully accept death as a natural part of the cycle of life, than offer fake simulacra of eternal life IMO.

But that's just me. Let the markets decide!


I want a life-sized hologram deep-fake, coded with all of that stuff. People could come and ask questions or chat. It would take what they said and try to either recall or synthesize what my response might be. I would not want it online, insisting instead that people make a special trip to interact with my digital remains.

If you like hard sci-fi I'd recommend reading Zendegi by Greg Egan. One of the main plot-lines is about a father creating a digital version of himself to leave for his son when he dies.

- https://en.wikipedia.org/wiki/Zendegi

- http://www.gregegan.net/ZENDEGI/ZENDEGI.html


This is quite brilliant and much more preferable. No one is going to want to sift through someone’s old vacation slide show. But to interact with a close facsimile of an ancestor...priceless.

Have you seen the “Black Museum” episode of Black Mirror?

The earliest appearance of this idea I can remember is the Dixie Flatline [1] construct in Neuromancer (1984), but I'm sure there are examples preceding that.

[1] https://williamgibson.fandom.com/wiki/Dixie_Flatline


I've thought about this too, but I think you've got two big marketing problems.

One is that discerning customers will be reluctant to deal with a startup. Most startups go out of business, and it will be challenging to convince people that you'll really be around even as long as the service purchaser, let alone their great-grandchildren.

The other is that non-discerning customers won't be able to tell a truly well-designed and well-endowed long-term archive company from one that just talks about it. So you'll have competitors that will charge 10% of what you do, and once who will charge the same but put 5x the money toward advertising.

I suspect this is a market only truly available to companies with long-lived brands. It's sort of like how cloud computing took off once Amazon put their brand behind it; if what you're selling requires deep trust, startups face big hurdles.


Like the scene in "Firefly", graveyard with moving pictures to remember loved ones by.

Interesting idea. I am not sure I want that level of detail preserved. I also am pretty sure those to come would want it.

Lots to think about with this.


Tombstones with pictures exist, so moving pictures would be a natural evolution.

The first time this happened, to a friend of mine whose parents out a photo for her, I found it really creepy. I still don't like to visit her grave. It makes me curious how differently people process grief.

Agreed. But this concept goes way deeper.

I had a similar idea a little while ago, but it was primarily aimed at storing DNA of your loved ones.

Not only would a backup of the DNA be stored inside a physical product, but the most variable regions of the genome would be sequenced and used as input for a set of lights embedded in the device, so that in the end every product is visibly different from ever other one in a way that's related to their actual genetic makeup.


You've just described the film "The Final Cut".

We kind of have this with social media. I have a good friend who passed away and I occasionally go look at his Instagram to relive the good times. Guess I should probably archive it in case they delete it.

This is almost exactly what the LifeNaut foundation aims to achieve: https://www.lifenaut.com/

Even more tantalizing idea: Some advance versions of GANs can learn from huge amount of data from your life and recreate you in digital world. Future generation can talk to that generative model of yourself which talks like you and says things you used to say and have beliefs you used to have. We shall call it Reanimation GAN.

Actually, there's this really neat network called Arweave which solves this permanent data archiving problem quite elegantly. Then, you only have to pay one fee upfront and the data will be there for a very, very long time if not forever.

How do you offset the fact that your operating costs are going to grow to infinity?

Charge $500 up front and use it as an endowment. Operating costs shouldn’t be more than hosting after the first year.

Storage gets cheaper over time.

Subscription, your remaining family pays 25 per year for each relative "digitally available" (otherwise it gets deleted)

> Given that we can't [live forever]

It's not can't it's don't currently.

That's the "hard problem to build a startup around."


Better sens money to SENS and the likes

Dixie Flatline

In the movie Kryptonian interactive holographic media is stored on irregular crystal shards. It's disappointing that this quartz media instead just look like squares of clear glass. I bet they're not even bothering to store them in a super cool Fortress of Solitude somewhere deep in the arctic wilderness.

the arctic is melting so not safe to store it there

Maybe Superman needs to learn how to chill.

They mention in the article that some of the old black-and-white Superman film is stored on glass as well - but what really amuses me is that (if I recall correctly) Superman HIMSELF stores film on glass (or long crystal shards) in the 1978 film that is the subject of this article. My sister and I used to pretend in winter that icicles were our VHS tapes and we’d put them in the snow to “watch” them.

I think they said the old glass recordings were radio serials, not movies.

Ah ok - thanks!

Is that why this particular film was chosen?

Of course! The movie is hardly a masterpiece...

There's been "M-Disc" [1] for a decade or so.[1] It's a form of DVD/Blu-Ray disc that, instead of using a dye that reacts to laser light, uses a drive with a more powerful laser to blast an indentation into the recording material. This is supposed to last a thousand years, and is much more resistant to heat and water than dye-based discs.

The blanks cost about twice as much and you need a drive with the M-Disc feature, which is not expensive.

[1] https://en.wikipedia.org/wiki/M-DISC


> The hard silica glass can withstand being boiled in hot water, baked in an oven, microwaved, flooded, scoured, demagnetized and other environmental threats that can destroy priceless historic archives or cultural treasures if things go wrong

How about dropping it on the floor?


Thick glass like that is pretty durable, especially fused silica. I work with glass artistically and it's surprising how strong borosilicate marbles are, for instance. You can throw a 35 mm marble on concrete repeatedly before it will chip. They work like bouncy balls for a time. I doubt these thick sheets would shatter easily.

That was my first question, too.

I linked another company doing the same thing and they claimed it could withstand 0.5 tons of force before breaking. I’m not sure how that applies to dropping it, necessarily if there are other parameters that would cause a weakness that way.


The key there is what type of force. Glass is very strong against compression, but not strong against tension.

Multiple copies, and always keep some of them in padded containers.

> Microsoft senior optical scientist James Clegg loads a piece of glass into a system that uses optics and artificial intelligence to retrieve and read data stored on glass.

How are they using "artificial intelligence" for this?


I'm pretty sure in this instance its pop sci speak for "somewhat better than usual statistical error correction system"

Fuzzy logic 2.0. ECC is just so last century.

"machine learning algorithms decode the patterns created when polarized light shines through the glass"

Unclear why ML is required versus old-school methods.


"As fans of The Mote in God's Eye already know, one remaining question must be answered for any data storage method expected to last for millennia—what happens when the technological and cultural context surrounding a storage medium collapses? Silica addresses this problem also, by using initial "ground truth" tracks. The team is using machine learning algorithms to re-read Silica's data, and in the event of the loss of those trained algorithms, fresh algorithms can train very rapidly on the "ground truth" tracks, which teach them how to interpret the rest of the data." https://arstechnica.com/gadgets/2019/11/microsofts-project-s...

You just need to get your civilisation back the point where you can write ML algorithms first.

Are our brains so inferior? Wouldn't someone who studies those ground-truth records long enough learn to see the data?

Yes, they would - and it actually says that in the article. But you'd then need some non-ml way to read & play it back, which would still require readily available computers & lasers, etc...

ML is the new way to refer to old-school methods such as regression, statistical inference, etc.

Ohh, ML is the new term for AI now.

We've finally accepted the computer doesn't have any information in it yet, and that we have to (somehow) feed some in.

Heh.


That was my thought. If the data is written in some specific format, shouldn’t it be able to be read just as easily, without resorting to ML?

I think the idea is to make it readable even if civilisation ends and we don’t have any record of the format when the tech is re-invented afterward — A built-in Rosetta Stone.

But you'd still need to be able to run the classifier, right? What's the difference between that and a piece of code that just knows the format? There's still a format here, regardless of how it's being decoded.

I’m not sure I follow. The idea (if it works is another matter) is that a future civilisation can say “we don’t know what this format is, let’s throw machine learning at it” and even though that machine learning system doesn’t know anything about the format when it starts, it can work it out on its own.

That would work regardless of the format, so why use ML today? Today you just write a decoder that assumes the format, in the future it can be reversed engineered using whatever means they have available to them.

Would it? I have not heard anyone suggest that JPEG could be decoded using only a self-training ML system.

Why wouldn't it? What makes a format more amenable to being decoded via ML?

This format comes with its own metaphorical Rosetta Stone.

The information required to decode the data encoded on the glass is encoded on the glass. The AI/ML is apparently a testing mode, to ensure that someone who knows nothing about what is on the glass is able to extract video of the Superman movie from it.

If the bootstrap section is good enough, the naive, untrained, know-nothing AI should be able to decode the movie. If it isn't good enough, the AI cranks for a long time and then puts Superman in a purple, orange, and green costume, and won't even know that those are all villain colors.


The hard silica glass can withstand being boiled in hot water, baked in an oven, microwaved, flooded, scoured, demagnetized and other environmental threats that can destroy priceless historic archives or cultural treasures if things go wrong.

The worst environmental threat is when your media is a proprietary format with secret specifications or even worse, DRM... even if the media could last 1000000 years, that's a moot point if the specifications have disappeared before then, or the encryption keys lost. Given the tie-ins with the entertainment industry mentioned in this article, the possibility of DRM causing the death of a format is too big to ignore.


One of the figure captions says they are storing 75.6 GB on one of these sheets. But I couldn't find any data on how long it takes to read and write, other than it's slow. Does it take minutes, hours, days, weeks?

And I guess at least the reading can be somewhat parallelized to speed up?



So why are they so happy when there's already 100 GB M-DISC which is much faster?

https://en.wikipedia.org/wiki/M-DISC


Can't find much info on M-DISC, but https://www.verbatim.com/subcat/optical-media/m-disc/ has it as hundreds of years. Silca is claiming a thousand years.

Its a 1 Mbit / sec write speed if I calculate it correctly.

Hopefully someone at Universal Music Group reads this story and reaches out. I'm not being sarcastic either.

https://en.wikipedia.org/wiki/2008_Universal_Studios_fire


Agreed. That was a tragedy.

The article seems to talk about the same concept as a https://en.wikipedia.org/wiki/Bubblegram .

I like write-once optical storage media because it protects me from accidentally modifying or deleting files. Also I don't have to worry about HDD reliability issues like head crashes, circuit board failures, etc., and it's easier to keep optical discs in cold storage with zero maintenance effort.

While hard drives have been getting bigger and cheaper every year, is anyone else disappointed that write-once media technology has stagnated?

A rough timeline of when writable optical discs became popular and affordable: CD-R 700MiB (~2000); DVD+/-R (+DL) 4.7GB/8.5GB (~2005); BD-R 25GB (~2010); BD-R XL 100GB (~2015). Various holographic discs have been proposed but are vaporware. The problem today is that all these disc formats are tiny compared to the 10TB HDDs you can pick up easily.


Raw storage per GB is also more expensive now compared to getting a giant hard drive. 100GB triple-layer BD-XL's are often ~$10/disk (~$0.10/GB). 10TB NAS-friendly HDD's can be had for ~$265 (~$0.0265). It sure was an interesting time where average desktop drives were only the size of a few DVDs which could be burned for less than a dollar. I hope improvements of commodity optical media happen but I think I'll be disappointed as consumers move to The Cloud. Chances are long term optical media archival will continue to move towards these esoteric implementations for massive corporations instead of something you can plug in to a normal computer (whatever a "normal computer" is in 20 years).

That said, there are other positives of going with write-once optical media. Magnetic or NAND media loses its charge over time, requiring refreshes to ensure data isn't lost. Its also more susceptible to outside EMI. Archive quality optical media is usually rated to safely store its contents for hundreds of years. I know I have cheap burned CDs from the 90s which still read with 100% accuracy, while most hard drives I have from that era have many errors and lots of corruption. Keep the media free from scratches, stored in the right temperature and humidity, and it'll probably outlive you without having to touch it.


My big issue is that optical media reliability isn’t where it needs to be for me to want to trust it for long term archival storage. Yes, some brands and media types are better than others, but it’s still enough a concern that I feel I can’t trust it in general.

You can try M-DISC that's supposed to store data a thousand years. No idea if it keeps its promises, though.

https://de.wikipedia.org/wiki/M-DISC



What technology do you trust for long term archival storage? Magnetic and NAND media need refreshing for long term storage. I'm unaware of any kind of storage media which will last decades at a time on a shelf, other than printing it on acid-free paper with inorganic inks/dyes.

> What technology do you trust for long term archival storage?

Lots of copies on different platforms / storage formats sadly.


I like optical media for the same reasons you indicate. Write once and have reasonable expectation it will last for a good long time. I do wish Optical media wasn't so stagnated as well. Write speeds could be improved by adding more laser heads to work in parallel on the disks, no reason for us to be stuck with a single head like vinyl had to for obvious reasons. That could increase disk R/W quite a bit and make it comparable to USB 2/3 but write once.

I'd love to get a holographic disk or heck an isolinear chip-shaped slab to write to because I like Star Trek.


I'm not so demanding when it comes to archival read/write speeds. Even if a drive writes 10 MB/s, it will crunch one terabyte in about a day. My gripe is that with small discs, you need to manually change discs and set up burns.

I am more than willing to leave my computer on overnight to write to a 1 TB optical disc with no human intervention. And this action is only for cold storage; HDDs are bigger anyway and I would have the data in hot storage if I needed to browse or edit it.


When it comes to Blu-Ray read speeds, good drives will already outperform USB 2.0. 16x read speeds on Blu-Ray is 576Mbit, faster than USB 2.0's 480Mbit peak.

https://en.wikipedia.org/wiki/Blu-ray#Drive_speeds


I think storing movie archives is really a nice fit for this technology. Reading/writing data on glass is too slow right now to host applications with interactions. I would imagine read/write movie archive only happens once per month or even years. That said, glass storage should also apply to never-change log data like bank transaction history (or stale blocks in block chains)

We should probably store a bunch of video evidence of historical events before we get the ability to do cheap and easy photo-realistic CGI on demand. There's going to be so many alternative video evidence for everything that historians are going to appreciate a known record that is difficult to update.

What if some one mischievous records the fake video instead of the real video? What if they get swapped post etching. I think the storage medium itself will not secure stuff for the future!

I hope to get this someday even at crazy prices. Permanent cold-storage of digital data can 'kinda' be done with M-Disk Blu Rays, but that is estimated good for 1000 years...which probably translates in reality to 100 or less years. DVDs, HDDs etc. have severe bit rot in comparison to that and are all likely to be gone within a decade +- a few years depending on medium and storage.

This glass? Write it and don't physically smash it and it should be good to go for the rest of your life +, if the claims in this article hold up. Then digital data will have a cold-storage equivalent to a piece of paper. Stick it in a box in the attic, get it out in 80 years and as long as its not smashed or burned it should be GTG.


I am confused by your comparison of M-Disk Blu Rays (estimated 1000 years, but actually only 100 years) with the glass (rest of your life+).

Are you anticipating living a lot longer than 100 more years?

And even then, isn't the M-Disk a better tech by your math?


The M-Disc claim of 1000 years is based on a few estimated tests and while they mostly hold up, we have objective proof that glass left alone over 1000 years does not change to the point that if you wrote something on year x, time-travelled with your reader to year 1000x , the reader would be unable to read it.

I don't expect to crack 90 years myself, but my great-great-grandchildren might want or need whatever data there is on that glassdrive, even if it is only pictures and not something important like legal documents.

The 'rest of your life+' claim is simply my shorthand for saying that while the glass will likely last for thousands of years, how long the readers exist that can read that particular glassdrive and interpret its data (ANSI or Unicode text, PNG or PDF display capability) may or may not last as long as the glassdrive itself can (see Voyneuch manuscript which exists, but we no longer can read and interpret it).


I am aware of a company that user to sell a long term DVD like storage disk that they claimed would last for 100+ years, 5-6 years in disks started to lose data, I now take lifetime claims with a big pinch of salt.

Indeed, I think of those claims as a mixture of optimism, wishful thinking, and marketing drivel. How can someone claim that it lasts for 100 years, unless you have tested that claim by waiting 100 years and then verifying that the data is still there ?

After many years of experience with CD-R and DVD-R, one can say that those disks are good for at least 3 years. But we can only say that because lots of people have successfully read CD-Rs that were 3 years old.

Also, they can sell those "100-year" disks with these claims, and when customers lose data after 5-6 years, the company just says "oh that's too bad, maybe you stored them where the temperature fluctuated by more than +/- 0.1 degrees Celsius ?" and you have no recourse other than not giving them more of your money in the future.


Exactly. But glass is glass, so the amount of believing we have to do with a pane of glass / quartz with this system is much less of a leap than with a pressed circle someone made in cheap-as-possible-for-profit-margins-increases factories, organic dye or not.

Presumably, this is to store things that future generations might also want to have.


Anyone else disappointed the quartz glass isn’t shaped like the stick-like storage units that appeared in the Superman movies?

Mark Russinovich gets to play with some cool tools. I still remember how advanced Winternals software was for sys admins. Microsoft really got some sharp guys in that acquisition.

Is this related to what Vint Cerf called “Digital Vellum” [0]

[0]: https://m.youtube.com/watch?v=0IabXOLdbAQ

*PS not sure if I have done the reference thing twice.


> For years, they had searched for a storage technology that could last hundreds of years...

I'd love to see a project preserve data for tens of thousands of years. The Egyptians did this by creating tombs. Surely we could do better.



> The team has baked, boiled, microwaved, demagnetized and scoured similar pieces of glass with steel wool — with no loss to the data stored inside.

That's really cool.


How do they know that it's going to last 1000 years? What kind of properties do you have to control for to insure that?

ensure

Neat - lossless archiving of stuff is actually pretty hard. This approach seems to check a lot of critical boxes.

First we got Communicators... now Isolinear Chips... I can't wait for Transporters!


That looks like a fun job.



Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact

Search: