Logstash—uses ElasticSearch for full-text indexing. ▫ Kibana is an alternative
WebUI for Logstash. ▫ Graylog2—ElasticSearch for full-text indexing &. MongoDB
...
John McLeod & Mike Pilkington
SANS DFIR Summit 2013
Disclaimer: The opinions expressed in this presentation are our own and may not reflect the opinions of our company.
Work for a Fortune 500 company in the oil & gas industry Mike is also a SANS Instructor John is retired AFOSI Combined 30+ years of computer security experience Both hold various certifications and higher level degrees Oh…and Mike is a UT grad. 3
A bit about evil System Center Configuration Manager (SCCM) Windows Host logging Even though we have the “talking stick,” feel free to ask questions
4
Hide in plain sight Rename Windows files Cmd.exe, AcroRd32.exe, diskmgmt.exe, etc.
Name files the same for years Ctfmon.exe
Persistent mechanisms …\Start Menu\Programs\Startup
known
unknown
known
unknown
known knowns
known unknowns
e.g. Known EVIL file is located in a known place
e.g. Known EVIL file is located in an unknown place
Unknown knowns
Unknown unknowns
e.g. An unknown file is located in e.g. An unknown EVIL file is an known place somewhere in your enterprise
SQL knowledge Read-only, backend access to SCCM database Microsoft SQL Server Management Studio Although this presentation details SCCM, the concepts can be used on other similar platforms
7
System Center Configuration Manager (SCCM) Formerly Systems Management Server (SMS) Product by Microsoft for managing large groups
of Windows-based computer systems Puts agent on host; sends data back Backend: MSSQL
Feature of SCCM Designed to track the usage of applications Provides the identity which software applications (and executable files) are being used, and who is using them Software metering client agent must be enabled in order to find EVIL using SCCM
Structured Query Language (SQL) Special-purpose programming language designed
for managing data held in a relational database management system (RDBMS)
The most common operation in SQL is the query, which is performed with the declarative SELECT statement More information http://www.w3schools.com/sql/ 10
Microsoft does not publish the schema, but… Creating Custom Reports By Using Configuration Manager 2007 SQL Views http://www.microsoft.com/en-us/download/details.aspx?id=22052
Configuration Manager 2007 SQL View Schema Microsoft Visio
document (CM2007SQLViewsSchema.vsd): Provides the Configuration Manager 2007 SQL views schema, organized by category. Configuration Manager 2007 SQL Views Excel spreadsheet (CM2007SQLViews.xls): Provides a list of the Configuration Manager 2007 SQL views with descriptions, a list of the SQL views and columns ordered by SQL view name, and a list of the SQL views and columns ordered by column name.
Asset Intelligence Client Deployment Client Health Collection Desired Configuration Discovery Hardware Inventory Mobile Device Mgt Network Access Protection Operating System Deployment Query
Reporting Schema Site Security Software Distribution Software Inventory Software Metering Software Updates Status Wake On LAN
AD Domain Name AD Site Name IP Address Netbios Name Hardware info User ID OS Service Pack
Registered User Last Boot Time Logon Date Domain System Type Autostart Software
SELECT LastUsedTime0, Name0, FolderPath0, ExplorerFileName0, OriginalFileName0, FileDescription0 FROM v_GS_SYSTEM INNER JOIN v_GS_CCM_RECENTLY_USED_APPS ON v_GS_SYSTEM.ResourceID = v_GS_CCM_RECENTLY_USED_APPS.ResourceID WHERE v_GS_CCM_RECENTLY_USED_APPS.TimeStamp > '2012-03-01 00:00:00.000' AND FolderPath0 LIKE 'c:\%' AND FileDescription0 LIKE 'Windows Command Processor' AND NOT ExplorerFileName0 LIKE 'cmd.exe‘ ORDER BY LastUsedTime0
LastUsedTime0
Name0
7/18/2012 7:11 TESTBOX5 5/28/12 20:03
TESTBOX3
3/30/12 0:39
TESTBOX1
3/2/12 5:19
TESTBOX4
FolderPath0
ExplorerFileName0 OriginalFileName0 FileDescription0 Windows Command C:\Intel\ ctfmon.exe cmd.exe Processor Windows Command C:\WINDOWS\system32\ sethc.exe cmd.exe Processor Windows Command C:\WINDOWS\system32\ sethc.exe cmd.exe Processor Windows Command C:\WINDOWS\system32\ sethc.exe cmd.exe Processor
SELECT LastUsedTime0, Name0, FolderPath0, ExplorerFileName0, OriginalFileName0, FileDescription0 FROM v_GS_SYSTEM INNER JOIN v_GS_CCM_RECENTLY_USED_APPS ON v_GS_SYSTEM.ResourceID = v_GS_CCM_RECENTLY_USED_APPS.ResourceID WHERE v_GS_CCM_RECENTLY_USED_APPS.TimeStamp > '2012-03-01 00:00:00.000' AND FolderPath0 LIKE 'c:\%' AND NOT ExplorerFileName0 LIKE ‘%.exe‘ ORDER BY LastUsedTime0
LastUsedTime0 Name0 7/18/2012 7:11 TESTBOX6
FolderPath0 ExplorerFileName0 OriginalFileName0 FileDescription0 C:\Users\(userid)\ Bat.vbs NULL NULL C:\Users\(userid)\App 7/4/12 20:03 TESTBOX9 Data\Local\Temp\ System.gif testest.exe testtest 5/30/12 0:39 TESTBOX2 C:\WINDOWS\system32\ wqrtuiwl.dll NULL NULL C:\Users\(userid)\App Data\Roaming\Microsof t\Windows\Start FFmpeg video 4/2/12 5:19 TESTBOX4 Menu\Programs\ flvvideo.dll ffmpeg.exe converter
SELECT LastUsedTime0, Name0, FolderPath0, ExplorerFileName0, OriginalFileName0, FileDescription0 FROM v_GS_SYSTEM INNER JOIN v_GS_CCM_RECENTLY_USED_APPS ON v_GS_SYSTEM.ResourceID = v_GS_CCM_RECENTLY_USED_APPS.ResourceID WHERE v_GS_CCM_RECENTLY_USED_APPS.TimeStamp > '2012-03-01 00:00:00.000' AND FolderPath0 LIKE 'c:\%' AND FolderPath0 LIKE '%Programs\Startup%' ORDER BY LastUsedTime0
LastUsedTime0
Name0
7/18/12 14:56 TESTBOX1
3/28/12 20:03 TESTBOX2
3/30/12 0:39 TESTBOX3
4/2/12 5:19
TESTBOX4
FolderPath0 ExplorerFileName0 OriginalFileName0 FileDescription0 C:\Documents and Settings\(userid)\Start Menu\Programs\Startup\ Update.exe NULL NULL C:\Users\(userid)\AppDa ta\Roaming\Microsoft\Wi ndows\Start Menu\Programs\ Update.exe Update.exe Document Updater C:\Documents and Settings\(userid)\Start AVG Basic Menu\Programs\Startup\ Ctfmon.exe NULL Interface C:\Users\(userid)\AppDa ta\Roaming\Microsoft\Wi ndows\Start FFmpeg video Menu\Programs\ flvvideo.dll ffmpeg.exe converter
Once a host to investigate has been identified run an SCCM history for that host ▪ WHERE Name0 LIKE ‘hostname'
The output may contain a group of commands showing evil activity
LastUsedTime0
Name0
FolderPath0
7/18/2012 7:11
TESTBOX5
C:\Intel\
7/18/2012 7:12
ExplorerFileName0 OriginalFileName0
FileDescription0
ctfmon.exe
cmd.exe
Windows Command Processor
TESTBOX5 C:\WINNT\system32\
net.exe
net.exe
Net Command
7/18/2012 7:25
TESTBOX5
hil.exe
NULL
Installer
7/18/2012 7:27
TESTBOX5 C:\WINNT\system32\
ping.exe
ping.exe
TCP/IP Ping Command
7/18/2012 7:32
TESTBOX5 C:\WINNT\system32\
sc.exe
sc.exe
7/18/2012 7:40
TESTBOX5 C:\WINNT\system32\
tasklist.exe
tasklist.exe
Task List
find.exe
Find String (grep) Utility
7/18/2012 7:40
C:\Intel\
TESTBOX5 C:\WINNT\system32\
find.exe
7/18/2012 7:41
TESTBOX5 C:\WINNT\system32\
netstat.exe
netstat.exe
TCP/IP Netstat Command
7/18/2012 7:41
TESTBOX5 C:\WINNT\system32\
taskkill.exe
taskkill.exe
Kill Process
Lot’s of false positives until you know your environment NOT and wildcards are your friends Timestamps
Last time used vs time received
Do not consider software metering to be authoritative Attacker could send invalid information to SCCM
Information is deleted periodically Only the most current data is in the database
25
Enabling & locating useful logs for IR SCCM Metering Logs Event Logs Others too, which we won’t have time to discuss… ▪ Windows Firewall Logs ▪ Commercial AV Logs ▪ MS Antimalware Logs ▪ Scheduled Tasks Logs
Remote searching & collection using PowerShell 26
Provides a local archive of inventoried software
Located in either: 32-bit: %systemroot%\system32\ccm\logs\mtrmgr.* 64-bit: %systemroot%\syswow64\ccm\logs\mtrmgr.*
Example Entry:
Often many deleted versions in unallocated Therefore, search/carve for entries with the signature “
When configured appropriately, Event Logs can tell us a ton, such as:
Software installed or uninstalled Processes run and by who, including parent process Services installed, started, and stopped User logons, including source IP and hostname if remote Users added and removed Groups modified
Determining the current audit policy can be tricky
Providing recommended auditing levels is tricky too, due to the impact heavy logging can have 28
Depending on how audit policy is configured, Local Security Policy (secpol.msc) could incorrectly indicate auditing disabled:
29
Despite the Local Security Policy settings , many logs are being created on the machine (184K over 4 months):
Unfortunately you cannot trust the Local Security Policy or Group Policy reporting tools such as GPRESULT or RSOP. Explanation here:
http://blogs.technet.com/b/askds/archive/2011/03/11/getting-theeffective-audit-policy-in-windows-7-and-2008-r2.aspx
Instead, use AUDITPOL…
30
Need to run “auditpol” to determine actual settings. For example: auditpol /get /category:*
31
It depends! Requires consideration and testing for your individual environment.
A great new resource is Microsoft’s whitepaper titled “Best Practices for Securing Active Directory” Download from http://aka.ms/bpsad Developed by Microsoft’s Information Security and Risk Management
(ISRM) organization Paper covers: ▪ ▪ ▪ ▪
Common attacks against Active Directory Countermeasures to reduce the attack surface Monitoring Active Directory for signs of compromise Offers recommendations for recovery
My Overview of the Best Practices paper: ▪ SANS Forensics Blog: http://computer-forensics.sans.org/blog/2013/06/20/overview-of32 microsofts-best-practices-for-securing-active-directory
Start with Microsoft’s strongest recommendations Setup test groups via Group Policy Weigh value of acquired logs to impact on performance
Where necessary, tighten the audit policy to reduce performance impact
Use Randy Franklin Smith’s site to supplement your analysis Identify event types that were not in your sample http://www.ultimatewindowssecurity.com/securitylog/encyclopedia/default.aspx 33
Site also gives pre-Vista ID # (and vice versia)
34
35
Many good commercial solutions: SEIMs such as ArcSight, Splunk, QRadar, etc.
Good (Splunk-like) open source options too: ELSA—uses Sphinx for full-text indexing Logstash—uses ElasticSearch for full-text indexing
▪ Kibana is an alternative WebUI for Logstash Graylog2—ElasticSearch for full-text indexing & MongoDB for stats 36
Can’t always forward the logs For example, often hard to get logs off of end-user
machines
For IR, PowerShell may be useful for querying remote logs
Specifically, use PowerShell “Remoting” Processing takes place on remote hosts – only
results are returned 37
WinRM is designed to provide management communications services for any number of applications PowerShell “Remoting” being one of those
Event Log Forwarding is another
HTTP is default transport, yet still encrypted via Negotiate or Kerberos SSP
Installed by default on Vista and higher. Available for XP and Server 2003 R2. 38
Processing occurs on remote machine Run commands interactively (ssh/telnet-like) via Enter-PsSession –ComputerName
Run commands non-interactively, but still processed on remote machine, via InvokeCommand cmdlet: Invoke-Command –ComputerName -command { }
Allows concurrent parallel connections rather than sequential connections 39
Testing by Jason Hofferle, as discussed on his site: http://www.hofferle.com
/powershell-remotingperformance/
His test gathered last 20 security event ID 4624 logs from each machine across US WAN links 40
Does not result in an interactive logon! Thus no exposure for passwords, hashes, or
Kerberos tickets
Does not create delegate-level impersonation token, allowing “double-hop” To allow delegation, must enable a specific feature
(CredSSP) on both client & server – don’t do this!
Network authentication via Kerberos
Machines are mutually authenticated 41
By using Invoke-Command, all processing within –Command parameter’s {braces} occurs on remote machine
Event Log Example: Find Security events after 5/31/2013 with message contents “ATTACK-ACCT”: Invoke-Command -ComputerName CompA,CompB -Command {Get-EventLog -LogName Security -After "Friday, May 31, 2013 12:00:00 AM" -Message *ATTACK-ACCT* }
Log-file Example: Find SCCM Meter log entries with filename that includes “.jpg”; output to interactive table with sortable columns: Invoke-Command -ComputerName CompA,CompB -Command {Select-String –Path C:\Windows\syswow64\ccm\logs\mtrmgr*.log -Pattern “.jpg“ } | Out-GridView 42
The Big Con: Does not work with legacy (EVT) logs
The Big Pro: Provides stronger filtering through XPath Query Language
Example—Find Event ID 4624 logon from user MIKE between 6/1/13 and 6/25/13: Get-WinEvent -ComputerName CompA,CompB -FilterXml ' *[System[(EventID=4624) and TimeCreated[@SystemTime >= "2013-0601T00:00:00.000Z" and @SystemTime