A company I worked for was started by someone doing this kind of work in biology. In particular trying to find matches in the ACTG strings and when there is a huge volume of data.
The startup was doing network compression - put an appliance either side of a WAN link and transparently compress the data inbetween. The core loop of that is examining IP and TCP traffic and determining if you have seen something like that before and how much of a match you have. If there is a match then you can just send the match details to the other side rather than the actual data. Typical compression ratios were 20:1 / 95% - ie you could stuff 20 times as much data through the link.
The standard zip algorithm has a window of 32kb (ie it looks roughly 32kb behind to see if data has been repeated). bzip2 uses 900kb. That is usually less than a second or two on standard WAN links! The founder's algorithm worked over hundreds of megabytes.
(The company name was Peribit Networks - you'll find Wikipedia rather unhelpful.)
The startup was doing network compression - put an appliance either side of a WAN link and transparently compress the data inbetween. The core loop of that is examining IP and TCP traffic and determining if you have seen something like that before and how much of a match you have. If there is a match then you can just send the match details to the other side rather than the actual data. Typical compression ratios were 20:1 / 95% - ie you could stuff 20 times as much data through the link.
The standard zip algorithm has a window of 32kb (ie it looks roughly 32kb behind to see if data has been repeated). bzip2 uses 900kb. That is usually less than a second or two on standard WAN links! The founder's algorithm worked over hundreds of megabytes.
(The company name was Peribit Networks - you'll find Wikipedia rather unhelpful.)