Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Others out there, such as:

    - DataVerse: https://dataverse.org
    - Omeka: https://omeka.org/s/
    - also many internally developed archival systems
To us it's mostly up to the granularity of the metadata, and wether metadata fields support e.g. ISO standards for validation (which is why it's difficult for a single metadata standard to rule all, unless all you need is free text descriptions). Another need is a way to batch ingest.

    > Compatible with hundreds of formats
I'm becoming increasingly critical of file format verification as a gatekeeper for digital preservation, but perhaps someone can explain why I'm wrong. Striving towards open specifications is of course a must for long term preservation (cave paintings persevere, while digital data is fragile in so many ways), but if conversion incurs data loss, parallel archiving should be an option. Worse, we've had issues with old systems that deny archiving data althogether because of an automated format checker standing in the way (we'll use FITS [0] in the future, but that just uses a bunch of other type checkers under the hood and doesn't seem bomb proof from the little testing I've done). Formats such as MP4 also seem like a nightmare to validate (granted, lots of cameras out there ignore following specifications). But archiving nothing at all over a few proprietary formats every now and then is a horrible outcome (see experience above). At the very least, it must be possible to override automated format checking if necessary.

    [0]: https://projects.iq.harvard.edu/fits


You'll want to be sure to test FITS thoroughly for your use case. It really needs some love. It's using some outdated tools and has a number of bugs. I submitted a bunch of PRs to the project this summer, but maintaining FITS is currently not a priority at Harvard, though I was assured that its not abandoned.

As a digital preservation aside, folks may be interested in OCFL[1], which is a preservation focused storage layout specification that had its 1.0 release last year.

[1] https://ocfl.io/


Ouch, thanks for the heads up. :/ We'll be stuck with FITS this time around, though.

Also, someone should put OCFL on the front page, while HN is in an archiving mood.


I now use https://archivebox.io/ and now am preparing to do a backup of my pinboard.in to it.




Consider applying for YC's Winter 2027 batch! Applications are open till November 2.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: