Conversation
Jeff the Alien
hackdefendr
#TIL
How to not analyze 8 gigabytes worth of #IIS logs inside a #Jupyter #Notebook.
You gotta break those logs down into months first, which slows down the investigation, but in the end you don't have to reboot the host server multiple times.
Using these notebooks are very powerful for drilling down into mass amounts of logs, but be careful, because without preset limits in place, Jupyter Server will go to town on resources.
How to not analyze 8 gigabytes worth of #IIS logs inside a #Jupyter #Notebook.
You gotta break those logs down into months first, which slows down the investigation, but in the end you don't have to reboot the host server multiple times.
Using these notebooks are very powerful for drilling down into mass amounts of logs, but be careful, because without preset limits in place, Jupyter Server will go to town on resources.
1
0
0
Jonah Stein
jonahstein
@hackdefendr Pardon my ignorance but isn't log file analysis a perfect use case for one of these new-fangled generative AI instances? Even better, couldn't it be trained on all of the old log files in an enterprise so it was well tuned to the system?
1
0
1
Jeff the Alien
hackdefendr
@jonahstein Hey...let me get back to you on this. You ask a great question. The short answer is we are a law firm, and they don't want me using the AI (at work). I used it outside of work to create a notebook to data science the shit out these logs. More on this later.
0
0
0