Ask HN: How did you implement GDPR compliant Global Log Collection system?

[rishiloyola]

Hello Guys, I am running hundreds of servers across the globe. Right now all these servers are pushing the data to centralized ELK cluster. I want to make a regional log collection system. Like all EU servers should send all logs to the EU cluster likewise for Japan, US, India servers.

Maintaining ELK clusters per region can be a pain and paying a huge amount to AWS Elastic Search Service is not a good idea.

Do you guys have some better suggestions? Any tools or best practices I should follow for this project?

[idoh]

I'm the PM in charge of GDPR issues at the company I work for. I don't know if I can give you any black and white answers, it really depends on a lot of factors. Some directions though:

- Try to minimize personal data in the logs or exclude it entirely

- It may be OK to have personal data in logs (e.g. IP addresses) if there is a basis for it. One example is for infosec - keeping logs to review for breaches, and pin down how the breach happened may be a good enough reason. You should disclose it as part of the terms of use.

- I don't know if you are really obligated to do per-region logs. If you are located in region A, and logs are located in region B, you still have to transfer data from A to B to look at it, so not really sure what it gets you.

I don't know if this is what you are looking for or not. Hope it helps though!