As the very first blog in the SOC Analyst series, I thought it would be good to write about what your life may be like as an analyst in the SOC. Most people have no idea what to expect. I know I had no clue when I first started out.
I also think that before you devote yourself to a particular tradecraft, you should know what you are getting into. That way, you go into things with your eyes wide open. This, in turn, helps you understand the why’s behind training recommendations and other advice.
In this blog, you will see underlined terms. These are buzzwords or terminology that you should be familiar with. The words may also come up in an interview or during your workday. At the end, you will see a section on common questions about the role. I did my best to remember what people ask, but feel free to comment with more questions, and I will append them to the article as they come in.
Join me on this journey into the SOC!
In my career, I have worked in two different Security Operation Centers, known affectionately as the “SOC.” It is here that I cut my teeth on security.
When I started out, I worked for a Managed Security Services Provider, or MSSP for short. These organizations are to who companies outsource SOC work if they don’t have a big enough internal team to handle the workload.
So, for instance, say a company, we will call them Acme, only has a team of three security people that handle everything security-related. This includes firewall management, vulnerability scans, investigations, compliance, etc. The Acme team doesn’t have enough time to review hundreds to thousands of security alerts a day.
This is where the MSSP and SOC-as-a-Service come in. The MSSP handles what are considered Tier 1 tasks; Reviewing alerts and escalating those that require further investigation.
Don’t get me wrong. This isn’t all MSSP’s may do. Services vary between organizations, and sometimes they carry out in-depth investigations and conduct incident response activities. Other MSSP’s will handle the engineering requirements for security appliances. And still, others may do penetration testing and sell appliances. It all depends on the business model of the MSSP. But for this post and simplicities sake, we will say the MSSP handles Tier 1 activities.
After all, in my first job, we only did tier-one analyst work and handed off anything that required further investigation.
Now that you have an understanding of why the role exists let’s dive into a typical day.
The SOC Analyst’s Day
Most Tier I SOC Analysts work in shifts. This is because the enemy doesn’t sleep. Attacks could happen anytime, whether it is a holiday or even when no one is looking.
The shifts will be designed to cover every hour of every day. Yes, this means you will most likely need to work holidays and weekends.
In most places, holidays will be manned by a skeleton crew, and that is to say, there will be a minimal amount of analysts on shift. Sometimes this is also the case for the weekends.
When you come into work, there will usually be a handoff period between the shifts. The shift leads will typically discuss issues with equipment, updates on current investigations, and let the incoming lead know of any tasks that still need to be completed.
This is one of the areas that differ in SOC’s. But what was previously mentioned is pretty typical of what I have seen in the industry.
The shift lead then briefs their team, assigning outstanding tasks that need to be completed.
For those without a task, analysts will begin to triage alerts.
Actually, the main focus of a Tier 1 Analyst is to triage. Alerts typically come through some sort of ticketing system that allows analysts to leave notes and manage the state of the alert. Once an analyst chooses an alert, they begin to investigate.
Investigations are done using a SIEM, EDR tool, and other security appliances in the client’s environment. These tools allow you to look at events that occurred around an alert to help figure out the context in which the alert fired.
Context is one of the most important concepts to remember. For instance, if you see the calculator app running in one log. You may think it is no big deal. That is pretty normal. However, if you see in another log that calculator is communicating with a malicious-looking domain on the internet, you have a problem. Calculator should not be making network communications like that.
This is where knowledge and experience come in. It helps you determine if a behavior is normal. A more experienced analyst can determine if an activity is normal more quickly. Where a junior analyst may need to spend time researching to make a conclusion.
From a company’s perspective, candidates with security fundamentals under their belt will learn this skill much faster. This is, by the way, why the technical interview process dives so deep into the basics. The more you know coming in the door, the easier you will be to teach. The faster you get up to speed, the more value you will have to the MSP. If the MSP only has so many positions, they will snag up those candidates with more knowledge.
Study. Study. Then study some more. You want to be at the top of their interview list.
Now back to the ticketing.
MSP’s benefit from ticketing in several ways. By maintaining notes about the various alerts, they have a historical account of who worked on the alert, when it was completed and what was identified during the investigation. These notations are often used by leadership to identify training opportunities for analysts.
For instance, if a breach was discovered and an analyst annotated that a related alert was not malicious, the shift lead could use the mistake to help the analyst learn. This would set the junior analyst up for success the next time similar behavior occurs.
However, most companies understand, they are hiring junior people with little to no experience. They also appreciate that mistakes are inevitable. Don’t be scared to make mistakes. It is going to happen.
But I digressed. The state of the alert is where analysts mark the alert as in progress or complete. Marking is essential so that their coworkers don’t duplicate efforts by investigating the same alert in parallel.
SOC Analysts diligently review every alert triggered by the security appliances and notate whether the alert needs to be investigated further or is nothing of concern. The terms often used to describe the second type of notation are false positives and anomalous but explained.
False positives mean that the rule for which the alert was written triggered incorrectly. Often due to a logic flaw. When these are identified, a team is notified that a rule needs some attention. People who maintain the detection logic are commonly called Threat Detection Engineers and Content Developers.
Anomalous but explained means that the rule fired as intended. However, the activity is not considered malicious. If the rules are written well, these will account for most of your findings. Logic often detects non-malicious activity because hackers often use similar tools and tactics of administrators. The logic is written in a way to allow some non-malicious activity in, to reduce the chances that malicious activity is missed.
Suppose an alert needs to be investigated further. In that case, the analyst notes all the information they have discovered and why they believe the activity requires a closer look. This info is then passed on to a Tier 2 team or to the client for further investigation.
At this point, the alert is complete. And the analyst moves on to the next one.
This cadence of investigating and closing alerts is the lifeblood of an analyst. It happens all day, every day, until there are no more alerts.
Often in the early morning, late evening, holidays, and outside of business hours, alerts are slow to trigger. This is simply because people aren’t working; therefore, few changes occur in the client’s environment. Resulting in few to no alerts.
When I was a shift lead, I used this time to train my analysts or provide them time to teach themselves. As practitioners, we have so much to learn in this industry, and it’s really a non-stop educational adventure. You should use your free time to grow as a professional and collaborate on research.
In my opinion, this is where average analysts and rock stars set themselves apart. The rock stars use this time wisely and add value to the organization with research, projects, and education. The average analysts use this time to relax. They take advantage of the breathing room to chill. One will develop their skills much faster than the other.
I am not saying you should kill yourself over work. Take breaks when you need them but also don’t be afraid to push yourself.
With that, we have concluded a workday. In summary, an analyst's job is to triage security alerts all day every day.
Now let me tackle common questions I have heard about the role. If you have any others, comment below and I will add them to the end of the blog.
If I Miss A Security Event, Will I Get Fired?
Some organizations may not be so nice when an analyst misses something. They also don’t look at these as training opportunities. If you look at the issue from the company’s perspective, you will see embarrassment and customer complaints. Rightly so.
Poor leaders may chastise the analyst for missing stuff or document it as a mess up. However, I haven’t heard of anyone being fired for one-off mistakes.
If you continue to make the same mistakes over and over, don’t expect to be kept around. When someone is fired for poor performance, the issue is usually systemic. The person doesn’t care about the quality of their work and isn’t looking to improve. In short, they usually deserve the outcome.
Learn from your mistakes by looking at them as opportunities to improve, and you will be fine.
How do you pronounce SIEM?
For those unfamiliar with the word SIEM, it is a controversial word pronounced differently, much like gif. I am in the group who pronounced it like stem without the “T.” Although you may hear it pronounced like seam.
What Exactly is a SIEM?
A SIEM is a log aggregator. In a traditional enterprise environment, logs are stored everywhere: computers, servers, appliances, the cloud, etc.
The analyst would have a heck of a time going to each source and parsing the logs. This is where the SIEM comes in handy. Organizations push the logs from all these places to the SIEM. The SIEM then allows a user to search through the logs much like a search engine for security.
Not every log is forwarded to the SIEM due to storage and processing requirements. Engineers will choose only the most important logs to forward to the SIEM.
If I Am Investigating, Why Does Someone Else Need To Investigate?
This is a good question. Junior Analysts may have the ability to conclude something seems weird, but they may lack the technical skills to dive deep enough to make a definitive conclusion. Multiple tiers exist for this reason. The higher the tier, the more knowledgeable the analyst.
These analysts may also have access to tools the tier 1 analyst does not. For instance, I have seen junior analyst only have access to the SIEM. This tool is great, but not everything is logged. This is where direct access to the security appliances comes in handy. A more senior analyst can dig in and view the context of the activity.
Also, let’s not forget about forensics and incident response procedures. These tasks both fall out of the purview of junior analysts. If the analyst found something malicious, a team will need to determine the who, what, when, where, and how of the activity. These investigations involve more senior people and can last for weeks or even months. Often organizations bring in a third-party to complete the investigation, if their staff doesn’t have the time, tools, or capabilities to successfully investigate an incident.
Will You Get Burned Out as a SOC Analyst?
Yes. Unfortunately, that is the reality of SOC work. It is very fast-paced and sometimes overwhelming. Your brain just can’t handle that workload for so long. Most people last eight months to two years in this role. Moving on to a different specialty such as engineering, leadership, pen testing, etc.
Realize that the SOC isn’t a forever role. But it is a great place to start. Get your experience and then move on to something more interesting to you.
I was in the SOC as an analyst for six months, then took a shift lead role for about four months before getting a shot at pen testing. Obviously, I took the sweet pen testing gig because that was my ultimate goal.
My advice is to stay if you are happy and if you not, move on to something that brings you joy.
Why Are There So Many Alerts?
There are tons of alerts because of the quality of rules and the way the alerts are configured. When analysts are inundated with alerts and start missing things, the industry term is “alert fatigue.” It is a real issue that is well known. That is why I said that missing stuff is ok.
More mature organizations will have automated efforts and use correlation techniques that help with the issue. Automated measures include tools such as SOAR (Security Orchestration, Automation and Response), which use playbooks to decrease the time it takes to close an alert. It automates tasks that analyst repeatedly do, like resolving an IP address or looking up the reputation of a domain.
Correlation techniques, on the other hand, use multiple triggers before an alert can fire. For instance, one failed login attempt is normal. But when you correlate, or in other words, look at over a hundred within a second for a single account, the activity is abnormal. Correlating events increases the fidelity, or in layman’s terms, the accuracy of alerts.
Will My Employer Provide Training?
Sometimes employers have a structured training program for all incoming analysts. Other times it is all on-the-job training. When I started my first job, on-the-job training consisted of shadowing a couple of days and then triaging increasingly difficult alerts. My notes were then reviewed by a fellow team member before I could close the alert.
My personal experience with training is that immature SOCs have no structured training. Which makes sense by the way. Then as the organization gets more experience, they develop a formal program.
For instance, my first role did not initially have structured training and then integrated training into the onboarding process.
My second SOC role also didn’t have a very good, structured training program for junior analysts. Before I left, one of my last projects was to create a bootcamp that taught all the basic skills needed to succeed.
Each place is different, and this would be a great question to ask at the end of your interview.
Infosec Analyst Training Program
Entry-Level Infosec Jobs
Top 2 Ways To Make A Junior Analyst Resume Pop
Top 8 Junior SOC Analyst Interview Questions To Study