Hacker News new | comments | show | ask | jobs | submit login

Size is unfortunately completely unimaginable....

A small marketing firm I happen to admin the email servers for... sends 15 million marketing emails per month (all opt-in, CAN-SPAM / Australian Spam Act compliant). This alone is nearly 1.5TB of data. We only send marketing campaigns for about 8 national brands and about the same number of regional brands to a country of twenty something million people.

Now It might be suitable for a conversation or private chat room. It would work really well in an IRC / small group situation. But I can't imagine a single global system ever scaling past a few dozen people without some significant hardware.

As far as deleting things, the problem there is who is responsible for deleting and what if I don't? The number of bounces we get from campaigns from defunct mail boxes that have filled to quota is quite high. Who removes this email that no one reads? At the moment the storage problem is isolated, but a central chain makes it impossible.

The more I think about this makes me think you might get it to work with a block chain per domain... but then whats the point over simply setting up your own email server you trust and using encryption?




Actually if you go with the concept of pointers to messages detailed here http://news.ycombinator.com/item?id=3615862 you enormously cut down on the size needed to store those messages (one message with a million recipients is just one message + a million recipients, not one million copies). Also with regards to deletion you could leverage the economics to pay for size, for example a user with a 1tb mailbox store will have to pay more mailcoins than a user with a 200kb mailbox store.


The pointers method is used by most commercial mail archival software that I've seen (the kind big companies deploy for SabOX compliance).

The problems with Pointers & de-dupe is that at least marketing emails are personalised, each are sent independently often with at least a minimum of personalisation, for example calling you by your name (people are more likely to read it if it calls them by name). You can do de-dupe, but your limited to the efficiency and overhead of your block size. Compression would also take the storage required down to ~ 1/10th (excluding attachments), but I don't think thats still enough. Even if deduping and pointers gave you a 500x storage saving, the data size for a few days email is still so unfathomable that it won't work as a single global system.

As far as trying to solve it with economics, its been tried (as a method to stop spam) but the question is who has to pay who and for what? Every such scheme has failed because you end up missing one of the big groups of users as it becomes un-economical for their mail pattern to continue. If you make the senders pay, few comercial entities want to play, and few people will provide free email. If you have to pay to store then how do you stop being billed for someone flooding you.




Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | DMCA | Apply to YC | Contact

Search: