

Ask HN: When do you index your logs? - redcircle

Do you write your log-worthy events to a file, and then parse them when you need to analyze them (e.g., Splunk); or do you write the structured event to a structured data store, such as a SQL database, which maintains indices in real time?  Both approaches can be configured to perform the parsing and indexing on a remote node.
======
ScottWhigham
I do it nightly to a SQL database. It's my morning routine to review the logs,
add firewall rules based on various "things" I find, or order new/replace
equipment based on what I see. I have a heavily partitioned SQL database and
so my load times for one day are very fast (< 3 mins typically).

