Monday, October 31, 2016

Audit Logs, XML Filters, and PowerShell, Oh My!

My last post documented the big breakthrough for me - being able to limit internet access for a few specific programs that had little business accessing the internet and were being abused by malware, pen testers, and researchers alike. Now I had a flexible tool that I could use to greatly limit this abuse.

Now what do I do with it?

Initially I thought it would be sufficient to allow all inside network access for the applications that I chose to limit and then just allow internet access as needed. I had a policy ready to go that would enforce this, but after some initial testing I learned that there was more activity by these applications than I initially thought. To deploy this it would be critical to log connections in the production environment and collect the information needed to make informed baselines before enforcing any application specific rules.

Easy enough, there was a place in the policy configuration where I could set up a log file which would output to a simple text file.



Looks pretty straight forward. Once the configuration was enabled I browsed to the log file to see what I had to work with. Here is a sample:

What's missing here?
My heart sank. There is some useful information in this log, but the application entry was missing! The key feature that drove me to use this tool was the fact that it could be configured to control application access to IP addresses. In order to effectively do that without the rollout becoming a resume generating event I needed a candidate configuration that allowed everything, but logged it, so I could create a baseline; and this wasn't going to let me do that. The other piece of information that this was missing was the rule that it matched against. Lots of data, but nothing to see here. I disabled this log and moved on.

After some time searching I found something that had potential: Windows Filtering Platform Auditing. Windows Filtering Platform is a set of services and an API in the Windows OS that allows traffic to be filtered. It is the underlying set of features that Windows Firewall with Advanced Security runs on top of. (For more, see MSDN) Auditing can be enabled by a policy located in the following areas:

Local:
Local Security Policy\Security Settings\Advanced Audit Policy Configuration\System Audit Policies - Local Group Policy Object\Object Access
GPO:
Computer Configuration\Policies\Windows Settings\Security Settings\Advanced Audit Policy Configuration\Audit Policies\Object Access

In these audit policies there is a subcategory for Audit Filtering Platform Connection. If Success and Failure are configured for this subcategory an event will be created in the system Security log with information about both blocked (event ID 5157) and allowed (event ID 5156) connections. These logs are much more detailed, including the following information:


  • Source User ID Edit 5/9/2017: After additional research I have learned that the source user ID is not in these logs. :-(
  • Process ID
  • Application Name and Path
  • Direction
  • Source Address/Port
  • Destination Address/Port
  • Protocol
  • Filter Run-Time ID (rule ID match)


...and a handful of other pieces of information. Having this information allows us to create a policy that allows all connections but audits them. We can then go through the audit logs and use them to determine a baseline to use when creating our application rules - of course only after thoroughly vetting the source application and every destination IP... right?

These logs have the type of information I was only able to dream about when working with network firewalls - with those I could trace traffic back to a machine and make a good guess at which user it was from but that was about the limit of what I could do. Seeing these logs take the username from the credentials used to launch a process along with the process name and path is a dream come true and provides valuable troubleshooting and forensics data that can be analyzed and consumed later.

Now that we have the data we need, how do we make use of it? On a single computer this is pretty straight forward. These audit settings generate a large volume of events (size log files for retention appropriately), but can be quickly filtered in or out of the event viewer console with the basic filter or with the XML filters.

For example, if you can find a sample event for an application you would like to write a rule for, an XML filter can be used to limit the packets to just what matches one or a few fields. Here is part of the XML view of an event where chrome.exe is connecting to www.google.com over port 443:

This can be obtained by opening the event in the Security event log, selecting the Details tab, and then selecting the "XML View" radio button.

The fields that are shown can be used in the XML filter to get only the specific log entries we might be looking for, even allowing us to use things like =, !=, multiple matches with AND/OR, etc. (see this TechNet blog for more info). I have been able to use up to 7 conditions to narrow down what I was looking for. (As a side note, more than 7 conditions seems to cause the filter to start behaving unexpectedly, but I can't find any information on why that is. If anyone knows why I would be very interested in hearing it.)

But let's not stop there! GUIs don't scale well. Let's imagine we would like to automate this process and have it generate some interesting data for us across thousands of machines so that we could build a GPO and have the confidence to enforce it! I ♥ PowerShell for this - "Get-WinEvent" with the "FilterXPath" switch allows us to write this all in a script to generate a match and then output it to a file for later consumption. If we use an event centralizing technology like Windows Event Forwarding then we have all of these logs in one place and the tools to make them useful! If we have access to a SIEM or logging platform that can take the logging hit, we have a recipe for large scale success.

Hopefully more on Get-WinEvent, WEF, and maybe event ELK in the next couple of posts.

Until then, work hard and spend time with your family.

Branden
@limpidweb

Monday, October 10, 2016

The Key to Success - Enabling Blocklists with Inverse Ranges

Have you ever been doing some task and have some random connection happen in the back of your mind regarding a completely unrelated subject and leaving you with no idea how it happened? That is what happened to me after my initial roadblock when trying to configure Windows Firewall with Advanced Security (WFwAS) to limit internet access for some applications. I knew that if it could control applications in this way it would mitigate all sorts of app AppLocker bypasses and quite a bit of malware. I had shelved my work on the matter though, until I had one of these particular random connection events happen to me while I happened to be taking a shower.

To recap my last post, I had been looking into the ability of WFwAS to limit access to IP addresses or ranges based off of individual executables, but ran into a dead end with the peculiar way that it handles rule order of operations. I didn't yet have enough data or comfort with the tool to deploy it in a way that only allowed pre-defined applications to my entire organization, and had to shelve my active work on it because I couldn't find another way to deploy it. If only I had a way to block some high risk applications that I was pretty sure had little legitimate internet use!

So there I was washing my hair in the shower, and then it hit me. "What if I block all IP addresses except the range that I want to allow, and then just configure the default rule to 'Allow' to handle the rest?" I thought it through for a little while and realized that this could be the key to solving my problem!

After that I couldn't wait to get to work and when I did I jumped right in and started setting it up on my test machine. Sure enough it behaved as expected and worked immediately! Unable to contain myself I bubbled over and had to share my success with a coworker who had been patiently listening to my drama about Windows' firewall. The key to the whole thing was something I called "inverse ranges". But what were these "inverse ranges"? To understand that, let's look back at the example from my last post.

This type of rule is what I spent years configuring and is what defined how my mind thought about rule sets.
My last post illustrated why this wouldn't work with WFwAS. To overcome this I had to think outside of the allow range and instead take the same end goal (blocking the stuff I didn't trust, allowing the stuff I have to trust) and make it fit in the WFwAS "process blocks first" mentality.

The key for this was to block what was not allowed, and then default allow to permit the rest. 

Here is the logic spelled out:
  1. Rule to block IP ranges not allowed that were before the IP address range I want to allow.
  2. Rule to block IP ranges not allowed that were after the IP address range I want to allow.
  3. Default allow the rest.
In other words define what I don't want and assume what I do want, thus the "inverse". Block what I want to block, the default allow permits the rest.

Let's take a look at what that looks like in our WFwAS configuration. I will use the 64bit version of PowerShell.exe as an example, and deny it to everything but RFC1918 IP address space. After watching our pen testers, studying malware in my test lab, and reading other people's blogs I knew that this was becoming a common method used by criminals to use initial execution on a machine to download and execute malicious code and would be high on my list to mitigate. As cool as that sounds, it was probably also due in large part to PowerShell web connections being one of the few things I could actually reproduce for testing that still had real world application.

(I apologize for the picture roll, especially if you are reading this on a mobile device).

Key here is that it is enabled and the Action is set to "Block the connection".
A reasonable name helps too.

Here is where we get to define the path and executable.
Remember that most systems have 32 bit and 64 bit .exe for many system files, so they will take at least 2 rules.
Also, check out that "Services" section... is that what I think it is? You bet, a great topic for another post!

Here is the "inverse range" concept in action. Look closely at the IPs and remember that these are being blocked.


Hard to see, and can't get it all on a single shot, but here is what part of the rule looks like after being configured.
Technically this could be done with subnet ranges, but I decided to simplify my rules by taking advantage of a WFwAS feature by using basic start-finish ranges. This allowed me to take what would require quite a few subnet statements and put it into a single basic range and greatly simplify managing my rule sets.

That's it, my secret is out. Isn't that cool? Well, I think it is anyway. I was able to get a successful PowerShell web download from a server on my allowed range (my internal network), but when I tried something from the internet that was previously working it puked red text onto my console window. But this time it was a welcomed and good puking of red text... in a twisted sort of way where my anthropomorphisation of a consoles behavior begins to quickly fall apart. Because it wasn't just my command puking; no, it was every piece of malware and crypto ransomware that used PowerShell to do its dirty deeds that would be puking all over consoles that were running hidden, and that could now be ignored and left to suffer alone in a world of STDERR!!!

So my next task was to take this and apply it to one of my original goals. I wanted to mitigate Casey Smith's clever use of system utilities to download code from the internet and execute it while bypassing application AppLocker. So I went over to his GitHub page and decided to create a rule for each item on his list of AppLocker techniques that used an executable to download code. I couldn't stop them from executing, but I could substantially limit their capabilities by limiting which IP addresses they needed access to.

I wrote up rules that covered the 32 and 64 bit locations for the utilities on the list and then proceeded to test it in our environment. There were a few applications that did have legitimate needs to access IP address space outside of my organization, but it was easy enough to just add in the appropriate "inverse ranges" to the block statements. There were a few challenges to be sure, but I had an effective mitigation that covered a wide swath of malicious activity. It would be worth my time to continue down this path and see how far it would take me.

With the basic functionality operating, I now had to tackle the challenges presented by long lists of IP addresses across multiple rules. To top that off, several of my applications were running up against the bane of many a firewall administrator - large lists of IP space owned by companies such as Microsoft, Amazon, and Akamai. I decided to tackle it, but that will have to wait until a future post for details.

I really hope that others see the benefit in this and find it helpful. Sound off in the comments or on Twitter if you have any questions about it. I plan to have more good things to come!

Until then, work hard and spend time with your family.

Branden
@limpidweb

Monday, October 3, 2016

Second fail brings me closer...

To recap my last post, I was trying to limit the ability of individual applications on domain workstations from being abused to bypass other restrictions (such as application allow lists) to download malicious content from the internet. EMET provided the capability to do this with an on/off switch, but I needed something more granular to roll this out at scale.

I knew that Windows Firewall with Advanced Security (hereafter WFwAS)  was capable of limiting access by application, but I had only worked with it to limit traffic inbound to a workstation, which is the capability that the basic Windows Firewall has had since Windows XP SP2. I hadn't spent much time with the new features introduced in Windows Vista when the name was appended "with Advanced Security". So, I thought I would give it a go.


To give a frame of reference, here is the console for WF on Windows 10

I began looking into the Outbound Rules section of the configuration. I was immediately overwhelmed with a very long list of complicated rules. Microsoft had contributed helpful stuff they thought I would need. In addition, all sorts of apps had put in custom rules, usually allowing various applications access to any IP address with any protocol. This would have to be cleaned up. But, the one key thing was here. There are rules that were allowing outbound traffic with specific applications listed in the rule. So far this was looking very promising!

Now that I knew that the rules could be limited to outbound traffic from specific applications, the next problem to tackle was limiting an application to some approved destination and then blocking it from all of the rest. My mind immediately went to my past as a network firewall admin... this was a problem that I learned to solve back in my Cisco Network Academy days when my instructors drilled IOS ACLs into our heads.

  1. Allow traffic from a source IP/range to a destination IP/range
  2. Deny all traffic from the source IP/range to ANY
  3. Continue with rest of list
  4. Default Deny statement at the end.
Quick IOS ACL example I whipped up in notepad to limit TCP access to approved telnet and HTTP while blocking all other TCP connections... not very practical in the real world, just using it to illustrate a point.
Easy enough, right? I wasn't quite sure how these applications I wanted to restrict were being used, but I was pretty sure some of them were being used on the corporate network. I decided that initially I wanted to allow access to my internal private IP address range from these applications but deny access to everything else. This would solve my problem of these apps downloading code from some random site on the internet, hopefully without upsetting the IT cart at my organization. So I had two problems to solve. First, writing the allow/deny rules, and second finding out where the default action was for the WFwAS. I decided to start with PowerShell.exe, which was easy enough to test using the Invoke-WebRequest cmdlet.

Since I didn't like the complexity and openess of the default rule set, I cleared all of the default firewall rules. I then added in my rule to allow PowerShell.exe access to the internal network range. Problem one solved.

Now on to problem two. After looking around a bit I found out that the default action can be set by right clicking "Windows Firewall with Advanced Security..." in the WFwAS console and selecting "Properties". The default for Outbound connections was set to "Allow (default)". Since I had removed a whole list of rules I knew that switching this to "Block" was not an option. If I did, I would kill things like DHCP and DNS and various other critical traffic unless I explicitly listed each app and service and what they were allowed to get to. At this time I just wanted to restrict PowerShell.exe. I left the default action to allow and then went back to the "Outbound rules" and set out to configure a Deny to mimic the IOS example above. I figured I would just allow PowerShell.exe to the internal network, deny PowerShell.exe to Any, get them in the right order, and then let the default action tackle the rest.

I guess that means I was back to problem one. Just a minor setback though. I already had an allow rule, so I added in my deny rule for any traffic from PowerShell.exe to destination Any. I was able to get the rule order correct on the screen by deleting and re-adding the allow rule, and started my testing. (Those of you with experience with WFwAS will know exactly what happened next.) Access to the internet was successfully blocked!!! So far so good. Next I wanted to make sure access to my internal network was also allowed. Easy enough, quick test to a webserver on 192.168.40.11...

Interesting...
And with that wall of red text I was introduced to WFwAS rule processing order of operations. I have worked on at least 7 major firewall/router vendors, all of which process rules from top down, first match wins. So why didn't it work? Simply put, WFwAS doesn't process rules that way. In fact, the order listed in the console didn't seem to make any difference at all. So, I went to the interwebs and hit the search button and landed on TechNet. Here is the WFwAS rule processing order, in summary:
  1. Windows Service Hardening (huh?)
  2. Connection security rules (IPSEC... I knew a little about this from a coworker)
  3. Authenticated bypass rules (more IPSEC magic)
  4. Block rules
  5. Allow rules
  6. Default rules
In my case it was the last 3 that really mattered (hopefully more on the first three in a later blog). WFwAS processes all of the Block rules, and if there is a match the traffic is denied. It then processes all of the Allow rules, and if there is a match the traffic is permitted. If there is no match, it applies the Default action defined in the properties dialog box.

With my PowerShell example, even though I had a more specific allow rule ordered before a broad deny rule, the deny rule was processed first and denied all of my PowerShell traffic and there was never an opportunity to match the allow I had set.

Whahh??

To date this is still the only firewall I have come across that processes rules in this way. It took me quite a while to get comfortable with the idea. I would love an hour with the committee at Microsoft who came up with this, um, most interesting solution. I tell myself it made sense at the time, right? I hope it has something to do with some dependency I don't yet understand.

So, my first attempt failed. After the initial elation I was a bit deflated. Perhaps I could switch to a rule that allowed all apps access to the internal network, things like web browsers access to all of the internet, and PowerShell to just the external ranges I knew it needed, and use a default action of "Block" at the end? This would be the "allow list" approach. I knew it would work, but I didn't know which applications needed access to the internet outside of web browsers. I knew that there were things like "web ex"perience meetings and meetings that I could "go to" that would need access to all sorts of IPs and that had random names and various hard drive locations they were launched from. I also knew that lots of installers relied on web connectivity to download applications. In short, I just wasn't ready for a pre-defined allow list, and I really just wanted to limit access to PowerShell and a few other applications.

I had my problem before me. Allow access to internal network for restricted apps, allow access to some external IP address ranges, block all else - without processing block rules first, with a default action of "Allow" for the rest (with logging enabled to start building my allow list - I hope to have more on this later as well). I was stumped and hungry and it was late and I was frustrated with Microsoft and I wanted to go home and see my family. I shelved my skunkworks project for a few days and mulled it over in the back of my head.

Turns out there is a way to do what I wanted, but I had to stop thinking like a firewall admin and think outside of the allow range. But, this post is already longer than I expected, so I will plan to go over how that was accomplished next week. I'll see if I can keep a cadence of a post a week.


Until then, work hard and spend time with your family.

Branden
@limpidweb