
Show HN: New storage technology - lcsantosgm
Hi,<p>I have developed a new technology to storage data in computer hard drive. It does have a native compression method that is transparent to OS and makes disk access faster than traditional SSD.<p>It is enabled by some math formulas combined with existing programming principles.
A practical implementation has been done in a windows virtual driver.<p>As an example:
You can create a virtual machine disk and install a 10GB Windows while in physical storage it only costs a few hundred megabytes, 100MB, 300MB depending on optimization.
This short storage allow for memory caching making access way much faster while it remains non-volatile.<p>Imagine companies with 10,000 VMs with much faster desktops and storage cost reduction on orders of magnitude. That is what can be done here.<p>Few video illustrations below.<p><a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=X-ZR3bSntHY" rel="nofollow">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=X-ZR3bSntHY</a><p><a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=iDfMVzwLWvI" rel="nofollow">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=iDfMVzwLWvI</a><p><a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=bjcwjZfFHBw" rel="nofollow">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=bjcwjZfFHBw</a><p>Problem I&#x27;m having is I emailed several research groups and email is ignored, people don&#x27;t even click the video link. Any suggestions to take this further?<p>Thanks
Lucas
======
nihil75
Hi Lucas, I'm a bit sceptical, as most people will probably be. As mentioned
by other here - more concrete benchmarking data will greatly help. Preferably
include a comparison with other technology bench results using the same data.

On the topic of data - I wonder what was actually on those compressed disks
you've shown in the videos, because as you probably know compression
algorithms performance depends on the complexity and entropy of the data.

Also, NTFS and FAT filesystems are mostly zeroes, so they do shrink down a lot
when compressed, and virtual disks can even be "sparse" to begin with..

In short - more information is required.

As for reaching out to research groups / investors - It can be difficult, and
I know this from experience running my own startup.

Between you and me - most investors won't even understand what you've created.
Assume they are non technical people. If it were the 90's again - most of them
would be working in the stock market, not technology.

So dumb it down, publish more papers/data, and get people excited - In this
business it's more about appearances than substance I'm afraid.. If there's
hype - they will be interested.

Hope this helps

~~~
lcsantosgm
Hi Nihil,

This definitely helps, in fact this was the nicest post I've seen and I will
try to reply equal care.

Sceptical - Yes I expected this, I didn't expect as much skepticism though.

benchmark - It is difficult to compare to something else as there's no one
doing something similar (afaik) and would be comparing apples and oranges,
nonetheless I will provide a comparison details at the bottom which is easy to
understand to general public.

DATA - The disks have a clean installation of Windows XP, chosen due to
relatively small size of 1.5GB compared to Windows 10 with 10GB and makes
processing of the model much easier. If you compress the Windows XP image with
Winrar it shrinks to ~500MB so around 30%, my method requires only 200MB
physical space so down to 13%.

It also differs from traditional compression in that with more powerful
hardware I can shrink the storage further preserving fast I/O, but it gets
exponentially more expensive for each %. Still large companies can afford the
benefit which is multiplied for thousands VMs.

Sparse file system - no, it's much better :-)

Additionally I'm having trouble to name my technology, it has much similarity
with compression but it is not traditional compression, it uses some math
functions such as number pairing function and others (I can't disclose the
sweet secret) and achieves an I/O much better than a traditionally compressed
file system.

Performance is in fact better than plain NTFS as can be seen in the third
video, a Windows 10 boot in 5-8 seconds is better than an SSD can do (around
25 seconds). Ultimately apps and systems can be written to take most
advantages of the technology characteristics.

Your last paragraphs were the most helpful I agree I need to get people
excited, I was hoping I could do this mostly anonymously but perhaps not
(don't need people to know my name). I will need to think what is best
strategy.

Some benchmark. HOST \----------------------------------- OS: Windows 10
Enterprise Processor: Intel Core i5 8 Gen Memory: 16GB DDR Base Storage: SSD
\----------------------------------- GUEST 1 (Normal)
\----------------------------------- OS: Windows 10 Education Processor: 8
Virtual Processors Memory: 2GB dedicated VM Storage: SSD Boot time: 18 seconds
\----------------------------------- GUEST 2 (New technology)
\----------------------------------- OS: Windows 10 Education Processor: 8
Virtual Processors Memory: 2GB dedicated VM Storage: Enhanced File System Boot
time: 8 seconds \-----------------------------------

Lastly, this is not only application for VMs, same technology can be used for
e-mail servers, databases, etc. Any more comments are greatly appreciated.

Thanks

~~~
nihil75
Thanks for the detailed reply. Was a very interesting read, and definitely
helped soothe my scepticism.

Regarding benchmarking - I think boot-time is not a very good test, as it
varies greatly depending on the guest VM. A fresh install will be pretty
consistent, but it's not a very reliable and _repeatable_ test.

You need some hard data - graphs, charts and counts. Maybe check out DiskMark
or some other tool like it, Then run test inside normal VM and your encrypted-
disk VM.

Second - I wonder if there's a way you could let people experiment with your
solution without "giving it away". for example-

1\. Compression-As-A-Service - a simple website, people can upload a file, get
the compressed file back. This way they can compare the size to
Zip/7Zip/whatever, And you don't actually give them the algo (You'll probably
say - "without my algo they can't open the file or use it, so they won't see
the crazy fast IOPS." , But it might be enough, at least as a first step).

2\. Compiled algo, limited distribution - I have a feeling you wrote this in
some low-level programming language, not Python and such. Could it be compiled
to a binary? If so it might be safe to distribute, as it will require serious
reverse-engineering to decipher. You could also "watermark" each executable
you send someone, so it has to "dial-home" via internet before it works, and
will help you identify who released it to the public in case someone does.

Hope this helps, and please feel free to connect with me on other channels as
I'd love to help some more :) my mail is nihil75 at gmail.com And I'm "nihil"
on mastodon.sdf.org

------
gus_massa
Can you win the Hutter Prize?
[https://en.wikipedia.org/wiki/Hutter_Prize](https://en.wikipedia.org/wiki/Hutter_Prize)

It's a well known challenge and it's a simple way to have an independent
validation of your method. The problem is that most people think it's too good
to be true, so they'd like to see an independent validation.

~~~
lcsantosgm
Actually no, there's no one size fits all with compression, in order to win
that prize you need an algorithm specialized in compressing natural language
text. Mine is specialized in binary data and fast I/O so it's a different
nature.

Maybe I will try to email the organizer though to see if there's interest in
what I'm doing. Thanks!

------
mtmail
Any reason the technology isn't explained in text? A video how fast a program
starts under Windows isn't exactly a scientific measurement.

~~~
lcsantosgm
thanks for your comment, added second and third paragraph to the question. I
choose video to demonstrate materially that this is real without disclosing
too much implementation details too soon.

Again, trying to figure out the best approach.

~~~
mtmail
The paragraphs help a lot, thanks for adding.

What is the goal? I mean you don't want to disclose the implementation so it's
a teaser of sorts. Of a coming product? Patent? Selling the idea to companies?
Getting a job or funding at a research firm? Does it have a name?

~~~
lcsantosgm
Yes correct. I can understand when you hear "technology breakthrough" the
first thought that comes to mind is BS, but it is what it is.

I intend to sell it as a trade secret to an investor or company and they do
the patent. Why don't I patent myself? Because I don't have a board of lawyers
to do the fighting When it is copied.

As for name, I don't have a final name yet but NextGen Hard Drive sounds fit.

------
emayljames
Hi Lucas, please keep us updated about progress in the future. I, and others,
will be very happy to hear your story, as well as this breath taking
invention.

~~~
lcsantosgm
I will, I got bogged down a little trying to win the Hutter Prize but it might
take a while, at the minute my adapted model can only match winzip for that
task.

------
wattengard
Is this not DriveSpace all over again?
[https://en.wikipedia.org/wiki/DriveSpace](https://en.wikipedia.org/wiki/DriveSpace)

~~~
lcsantosgm
How interesting the new stuff I can learn in an open forum like this!

Well no it's not the same it has several differences but the two major are:

One, it can shrink storage up to 100 times, unlike traditional compression
which saves 50-70% with prohibiting performance loss. Two, data access can
actually be up to 100 times faster, while in traditional compression data
access for many small files can be too slow.

------
pell
It would be helpful if you could provide some benchmarks.

------
pnunesc
Here can I find more info about this project?

PS: Windows XP?!

~~~
lcsantosgm
Third video is windows 10 :) People claim on a fast SSD Windows 10 boot in 25
seconds. That is 5 seconds on the video.

I'm looking for guidance to take this to right attention and get a deal.

