I worked on the memory allocator for a feature called Flex Tables - https://www.vertica.com/docs/9.2.x/HTML/Content/Authoring/Fl... which involved loading unstructured data into the column store database. I worked with two incredible engineers (who I believe were sniped by Facebook later). One was a distributed systems expert who spent the entire summer on and off debugging a multi-server crash where the root cause and failure occurred 30 minutes apart. When he finally crushed it I went outside for lunch that day and he was sitting in a lawn chair on this corporate office drinking a beer and staring in to the distance. I had this feeling that he was untouchable and alone in the universe that day, having conquered a mountain few people will ever climb.
The second guy more or less taught me how to use Vim. And not just use it, but move buffers around and navigate the filesystem so fast that it seemed he was wired directly into the computer. Up to that point I had not worked with a very very experienced programmer who was still practicing. He would write very complex regexes directly in the vim prompt to find the right parts of log files.
The overall team had been acquired by HP 18 months before I arrived and you could tell that people were beginning to move along. However many of the folks there had worked with Stonebreaker at MIT and then at C store. There was a feeling that they all knew things about column store databases that were not widely known. In particular the engineering manager once walked us through the very first project he implemented at Vertica, which was backup and restore. He more or less derived the whole system from a few principles of when data should be saved over the course of 2 hours.
It was an extremely technical place to work with data sizes and problems I had never encountered. It gave me a sense of the the sheer scale that some companies operated at and what was possible by starting with C++ and building the system out bit by bit over years. Many of the approaches there actually mirrored architecture work I did later on a multi-camera tracking system.
The second guy more or less taught me how to use Vim. And not just use it, but move buffers around and navigate the filesystem so fast that it seemed he was wired directly into the computer. Up to that point I had not worked with a very very experienced programmer who was still practicing. He would write very complex regexes directly in the vim prompt to find the right parts of log files.
The overall team had been acquired by HP 18 months before I arrived and you could tell that people were beginning to move along. However many of the folks there had worked with Stonebreaker at MIT and then at C store. There was a feeling that they all knew things about column store databases that were not widely known. In particular the engineering manager once walked us through the very first project he implemented at Vertica, which was backup and restore. He more or less derived the whole system from a few principles of when data should be saved over the course of 2 hours.
It was an extremely technical place to work with data sizes and problems I had never encountered. It gave me a sense of the the sheer scale that some companies operated at and what was possible by starting with C++ and building the system out bit by bit over years. Many of the approaches there actually mirrored architecture work I did later on a multi-camera tracking system.