The what, why and (simplified) how of SIEM
Security · Apr 1, 2016
Guest contributor and TELUS security pro Josh Brierton breaks down the x,y,z of SIEM and explains why it’s become so popular for finding “weird” stuff on the network.
SIEM! It’s literally everywhere. Media. Vendors. Industry events.
But what is SIEM really? What does it do? Why do we need it?
Let’s start with the definition. According to Wikipedia:
Security information and event management (SIEM) software products and services combine
Great. Seems pretty jargon heavy though. What does that definition mean in the real world? To uncover the answer, let’s start at the beginning and look at the origins of SIEM.
It's the 1990's. Hanson's hit song
Weirdly enough, it looks like something out of a movie. A huge wall of screens showing charts and graphs, maybe a few security cameras monitoring a server room and possibly the news playing. But the bulk of the room is taken up with row upon row of people staring at two to three monitors of data, which are refreshing at an unbelievable rate.
What is going on? Basically there are people manually monitoring logs coming from various devices around the network, and they're looking for anomalies. If this sounds somewhat impossible to do manually, you're not entirely wrong. These people build filters to scrub out some of the data, but for the most part, the first row of people are looking for stuff that has been deemed "weird."
If they see something weird, they pass it to the team behind them who delves deeper to determine whether weird is really weird. If there is something outside of the norm that could indicate a possible attack, the event is finally passed to the next team. Each team has a number, making up the "tier" system -- with Tier 1 handling the mundane and Tier 3 (or higher depending on the size of the organization) handling the truly unusual or malicious.
As you can imagine this can be mind-numbing work. A single corporate firewall can generate millions of logs per day.
Now most of the data generated is fluff that isn't worth anyone’s time to review. But we're back in the 90's, and aside from making generic, error-prone filters, the only way to determine if something is good or bad is by examining it.
The idea for SIEM originated as a collection of intricate scripts people made to automate some of the monitoring tasks until they had nearly a full product’s worth of code. Essentially they wanted to automate approximately 80% of the manual work of examining boring logs, leaving them time to focus on important stuff. Below is a diagram I use to explain what a SIEM solution does (very basically).
Starting at the bottom, you have a deluge of logs coming from as many devices as you want to monitor in your network. Sometimes this may only be the perimeter devices. Sometimes this could literally be every piece of hardware connected to a network (I've seen deployments that collected logs from door badge access systems for instance). All of these logs come in looking different from each other, which brings us to the first key functionality of a SIEM solution. It interprets/normalizes logs into one easy access form.
Why are these logs different from one another? Two reasons:
Each vendor’s device logs things differently and, as a result, presents different information.
Making a device log is boring work. Seriously. If you ask programmers if they’d like to code a new feature, fix a bug, write a document or format logs properly, they'll choose anything BUT the last option every time.
Once collected and interpreted, the results go to long-term storage along with the raw logs that produced them. Reporting often happens at this stage, providing a general overview of network activity.
At the same time, a select subset of events goes to the real-time tier that houses the rules. This is where the solution hunts for the weird stuff using filters, simple rules, correlation rules and lists, which act like miniature temporary databases.
The why of SIEM
SIEM solutions were introduced to solve two problems: automate the hunt for weird stuff and normalize all the data from across the network into one readable format. It includes the following capabilities:
Something to collect and interpret logs
Something to store the logs for reporting over long time periods
Something to process events in real time to tell you if something "weird" happened
With these capabilities, analysts have a simpler and more effective way to explore logs and identify strange behaviour on the network, which inevitably contributes to a more a proactive approach to security and optimally, better overall protection.
Strengthen your cyber security stance with