I've been wondering about this for some time, and I really started thinking about it today after reading this post. Actually, what really got me thinking about it was when I configured auditd on a red hat system last week. In roughly 3 hours, without network connectivity, just me on the box, 3 gig of logs. And that's just the audit logs. Sure, that much information is great after you're owned and you're looking over the logs (assuming you're logging to a remote server), but really, with that much information, aren't you just creating more of a hassle than anything else? No organization has enough staff to watch that much log traffic. Even after the fact, do all those logs get you anywhere? Sure, you can say "Ok here's where they came in and what they did" but that's about it. Is that worthwhile?
Now that I think about it, I guess if you're concerned about someone coming in and sitting on your systems indefinitely without being detected, then serious logging might be warranted, but again, who can watch that volume of logs? I saw something recently that I thought was really cool -- Juniper's IDP device has an option where you can allow it to profile your 'normal' traffic. It watches and watches and gets a baseline of what usually happens on your network, so if anything abnormal happens, you are alerted. This isn't new, but I hadn't really seen it before, so it's newish to me. Is there something similar for logging? That would be a worthwhile application in my opinion -- generate events OUTSIDE the usual mundane.
I'm torn on this issue. Logs are like a warm blanket; verbose logging means you can know what's happening on your systems if you keep up with the logs. At the same time, logs become a burden very very easily, and they are easy to ignore. I guess each admin determines the line between verbosity and noise, but for me and my systems, I need to think about this more. I'm not sure I'm comfortable running systems with no logging. That seems like a liability issue -- a problem waiting to happen.
3 comments:
wootness :)
imo logs are worth nothing if you aren't reading them.
that's one reason i should dig out the syslog project from back in the day. if you can use regexp and some reasonable ways to apply those regexp to groups of hosts, it's pretty easy to get value from your syslog server.
basically, most logs are full of crap, so we tune just like ids. so i can regexp for windows, redhat, bsd, etc machine specific chatter and pull out all of the normal meaningless crap. if you're working w/ these systems and know what's normal, eventually you're left only getting logs when something worth paying attn to is happening on a given host. i feel like i managed to achieve that in production back in the day.
anyway, as far as doing things other than pattern matching, you're in good company. bruce potter is giving that talk at bh, but i (we?) already saw it at notacon... looks like mb some updates ;)
we were talking about it at the last CitySec actually... whether or not you'll be able to that type of analysis on malicious network traffic running over port 80 co-mingling w/ web2.0 traffic... imo, it'll still be possible to do. i'm kicking around a project w/ that atm (which i'll surely never complete) and have aquired what i believe is a port80 http based malware sample ;)))
further updates as warranted...
You're right about tuning, and yes, I did think about that, but even after meticulous tuning, if/when you have an incident, chances are you're overwhelmed with logs very quickly, no?
I don't remember the net flow talk, will definitely have to catch it now. You touched on the malware over port 80 issue -- that's really valid, and that's more along the lines of what I'm thinking with this post. Even with all the tuning and configuration and awesome ninjitsu with logs, how much benefit do you really get these days? I'm thinking very little, especially in a large organization. The fact that people need to read the logs is almost a moot point since you can always have more people reading logs.
I do like this idea of watching for everything outside the norm, but that assumes your attacker will do something outside the norm -- not a very safe assumption. I guess finely tuned logs and a net flow type IDS system would work pretty well in combination.
personally, i can't imagine having too many logs during incident response. between grep/sed/awk i'm not really seein the downside... ;)
"how much benefit do you really get these days? I'm thinking very little, especially in a large organization. The fact that people need to read the logs is almost a moot point since you can always have more people reading logs."
i can't say i've been to too many clients where they were like "man, the logs are done now, can we get some real work to do?" :P
i need to post about the bell curve stuff...
Post a Comment