Skip to content

Commit

Permalink
Merge branch 'SigmaHQ:master' into master
Browse files Browse the repository at this point in the history
  • Loading branch information
hieuttmmo authored Jun 30, 2021
2 parents 579220d + 19962c6 commit dceb13f
Show file tree
Hide file tree
Showing 776 changed files with 17,046 additions and 3,846 deletions.
27 changes: 0 additions & 27 deletions .github/workflows/pypi-publish.yml

This file was deleted.

10 changes: 6 additions & 4 deletions .github/workflows/sigma-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ on:
branches:
- "*"
pull_request:
branches: [ master ]
branches: [ master, oscd ]

jobs:
test-sigma:
Expand All @@ -22,10 +22,12 @@ jobs:
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r tools/requirements.txt -r tools/requirements-devel.txt
pip install pipenv
pipenv lock
pipenv install --dev --deploy
- name: Test Sigma Tools and Rules
run: |
make test
pipenv run make test
- name: Test SQL(ite) Backend
run: |
make test-backend-sql
pipenv run make test-backend-sql
3 changes: 2 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -57,8 +57,9 @@ test-sigmac:
$(COVERAGE) run -a --include=$(COVSCOPE) tools/sigmac -rvdI -t arcsight -c tools/config/arcsight.yml rules/ > /dev/null
$(COVERAGE) run -a --include=$(COVSCOPE) tools/sigmac -rvdI -t arcsight-esm -c tools/config/arcsight.yml rules/ > /dev/null
$(COVERAGE) run -a --include=$(COVSCOPE) tools/sigmac -rvdI -t qradar -c tools/config/qradar.yml rules/ > /dev/null
$(COVERAGE) run -a --include=$(COVSCOPE) tools/sigmac -rvdI -t stix -c tools/config/stix.yml -c tools/config/stix-qradar.yml -c tools/config/stix-windows.yml rules/ > /dev/null
$(COVERAGE) run -a --include=$(COVSCOPE) tools/sigmac -rvdI -t stix -c tools/config/stix-custom.yml -c tools/config/stix-shifter.yml -c tools/config/stix2.0.yml rules/ > /dev/null
$(COVERAGE) run -a --include=$(COVSCOPE) tools/sigmac -rvdI -t limacharlie -c tools/config/limacharlie.yml rules/ > /dev/null
$(COVERAGE) run -a --include=$(COVSCOPE) tools/sigmac -rvdI -t chronicle -c tools/config/chronicle.yml rules/ > /dev/null
$(COVERAGE) run -a --include=$(COVSCOPE) tools/sigmac -rvdI -t carbonblack -c tools/config/carbon-black.yml rules/ > /dev/null
$(COVERAGE) run -a --include=$(COVSCOPE) tools/sigmac -rvdI -t qualys -c tools/config/qualys.yml rules/ > /dev/null
$(COVERAGE) run -a --include=$(COVSCOPE) tools/sigmac -rvdI -t netwitness -c tools/config/netwitness.yml rules/ > /dev/null
Expand Down
9 changes: 6 additions & 3 deletions Pipfile
Original file line number Diff line number Diff line change
Expand Up @@ -10,13 +10,16 @@ elasticsearch = "~=7.6"
elasticsearch-async = "~=6.2"
pytest = "~=5.4"
colorama = "*"
setuptools = "*"
stix2 = "*"
attackcti = "*"

[packages]
requests = "~=2.23"
urllib3 = "~=1.25"
requests = "~=2.25"
urllib3 = "~=1.26"
progressbar2 = "~=3.47"
pymisp = "~=2.4.123"
PyYAML = "~=5.1"

[requires]
python_version = "~=3.8.2"
python_version = "3.8"
657 changes: 425 additions & 232 deletions Pipfile.lock

Large diffs are not rendered by default.

42 changes: 23 additions & 19 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,9 +40,9 @@ The SANS webcast on Sigma contains a very good 20 min introduction to the projec

# Why Sigma

Today, everyone collects log data for analysis. People start working on their own, processing numerous white papers, blog posts and log analysis guidelines, extracting the necessary information and build their own searches and dashboard. Some of their searches and correlations are great and very useful but they lack a standardized format in which they can share their work with others.
Today, everyone collects log data for analysis. People start working on their own, processing numerous white papers, blog posts and log analysis guidelines, extracting the necessary information and build their own searches and dashboard. Some of their searches and correlations are great and very useful but they lack a standardized format in which they can share their work with others.

Others provide excellent analyses, include IOCs and YARA rules to detect the malicious files and network connections, but have no way to describe a specific or generic detection method in log events. Sigma is meant to be an open standard in which such detection mechanisms can be defined, shared and collected in order to improve the detection capabilities for everyone.
Others provide excellent analyses, include IOCs and YARA rules to detect the malicious files and network connections, but have no way to describe a specific or generic detection method in log events. Sigma is meant to be an open standard in which such detection mechanisms can be defined, shared and collected in order to improve the detection capabilities for everyone.

## Slides

Expand All @@ -52,7 +52,7 @@ See the first slide deck that I prepared for a private conference in mid January

# Specification

The specifications can be found in the [Wiki](https://github.com/Neo23x0/sigma/wiki/Specification).
The specifications can be found in the [Wiki](https://github.com/Neo23x0/sigma/wiki/Specification).

The current specification is a proposal. Feedback is requested.

Expand All @@ -62,7 +62,7 @@ The current specification is a proposal. Feedback is requested.

Florian wrote a short [rule creation tutorial](https://www.nextron-systems.com/2018/02/10/write-sigma-rules/) that can help you getting started.

## Rule Usage
## Rule Usage

1. Download or clone the repository
2. Check the `./rules` sub directory for an overview on the rule base
Expand Down Expand Up @@ -106,7 +106,7 @@ merges multiple YAML documents of a Sigma rule collection into simple Sigma rule

```bash
usage: sigmac [-h] [--recurse] [--filter FILTER]
[--target {arcsight,es-qs,es-dsl,kibana,xpack-watcher,elastalert,graylog,limacharlie,logpoint,grep,netwitness,powershell,qradar,qualys,splunk,splunkxml,sumologic,fieldlist,mdatp,ee-outliers}]
[--target {sqlite,netwitness-epl,logpoint,graylog,netwitness,arcsight,carbonblack,es-rule,ala,elastalert-dsl,splunkxml,fieldlist,sysmon,arcsight-esm,kibana,csharp,qualys,powershell,es-qs,mdatp,humio,grep,qradar,logiq,sql,sumologic,ala-rule,limacharlie,elastalert,splunk,stix,xpack-watcher,crowdstrike,es-dsl,ee-outliers}]
[--target-list] [--config CONFIG] [--output OUTPUT]
[--backend-option BACKEND_OPTION] [--defer-abort]
[--ignore-backend-errors] [--verbose] [--debug]
Expand All @@ -131,7 +131,7 @@ optional arguments:
tag that must appear in the rules tag list, case-
insensitive matching. Multiple log source
specifications are AND linked.
--target {arcsight,es-qs,es-dsl,kibana,xpack-watcher,elastalert,graylog,limacharlie,logpoint,grep,netwitness,powershell,qradar,qualys,splunk,splunkxml,sumologic,fieldlist,mdatp}, -t {arcsight,es-qs,es-dsl,kibana,xpack-watcher,elastalert,graylog,limacharlie,logpoint,grep,netwitness,powershell,qradar,qualys,splunk,splunkxml,sumologic,fieldlist,mdatp}
--target {arcsight,es-qs,es-dsl,kibana,xpack-watcher,elastalert,graylog,limacharlie,logpoint,grep,netwitness,powershell,qradar,qualys,splunk,splunkxml,sumologic,fieldlist,mdatp,devo}, -t {arcsight,es-qs,es-dsl,kibana,xpack-watcher,elastalert,graylog,limacharlie,logpoint,grep,netwitness,powershell,qradar,qualys,splunk,splunkxml,sumologic,fieldlist,mdatp,devo}
Output target format
--target-list, -l List available output target formats
--config CONFIG, -c CONFIG
Expand Down Expand Up @@ -172,13 +172,13 @@ Translate a whole rule directory and ignore backend errors (`-I`) in rule conver
```
tools/sigmac -I -t splunk -c splunk-windows -f 'level>=high' -r rules/windows/sysmon/
```
#### Rule Set Translation with Custom Config
#### Rule Set Translation with Custom Config
Apply your own config file (`-c ~/my-elk-winlogbeat.yml`) during conversion, which can contain you custom field and source mappings
```
tools/sigmac -t es-qs -c ~/my-elk-winlogbeat.yml -r rules/windows/sysmon
```
#### Generic Rule Set Translation
Use a config file for `process_creation` rules (`-r rules/windows/process_creation`) that instructs sigmac to create queries for a Sysmon log source (`-c tools/config/generic/sysmon.yml`) and the ElasticSearch target backend (`-t es-qs`)
Use a config file for `process_creation` rules (`-r rules/windows/process_creation`) that instructs sigmac to create queries for a Sysmon log source (`-c tools/config/generic/sysmon.yml`) and the ElasticSearch target backend (`-t es-qs`)
```
tools/sigmac -t es-qs -c tools/config/generic/sysmon.yml -r rules/windows/process_creation
```
Expand Down Expand Up @@ -209,7 +209,9 @@ tools/sigmac -t splunk -c ~/my-splunk-mapping.yml -c tools/config/generic/window
* [LimaCharlie](https://limacharlie.io)
* [ee-outliers](https://github.com/NVISO-BE/ee-outliers)
* [Structured Threat Information Expression (STIX)](https://oasis-open.github.io/cti-documentation/stix/intro.html)
* [LOGIQ](https://www.logiq.ai)
* [uberAgent ESA](https://uberagent.com/)
* [Devo](https://devo.com)

Current work-in-progress
* [Splunk Data Models](https://docs.splunk.com/Documentation/Splunk/7.1.0/Knowledge/Aboutdatamodels)
Expand All @@ -228,16 +230,18 @@ It's available on PyPI. Install with:
pip3 install sigmatools
```
Alternatively, if used from the Sigma Github repository, the Python dependencies can be installed with:
Alternatively, if used from the Sigma Github repository, the Python dependencies can be installed with [Pipenv](https://pypi.org/project/pipenv/).
Run the following command to get a shell with the installed requirements:
```bash
pip3 install -r tools/requirements.txt
pipenv shell
```
For development (e.g. execution of integration tests with `make` and packaging), further dependencies are required and can be installed with:
```bash
pip3 install -r tools/requirements-devel.txt
pipenv install --dev
pipenv shell
```
## Sigma2MISP
Expand All @@ -251,7 +255,7 @@ Example:
*misp.conf*:
```
url https://host
key foobarfoobarfoobarfoobarfoobarfoobarfoo
key foobarfoobarfoobarfoobarfoobarfoobarfoo
```

Load Sigma rule into MISP event 1234:
Expand All @@ -266,7 +270,7 @@ sigma2misp @misp.conf --same-event --info "Test Event" -r sigma_rules/

## Evt2Sigma

[Evt2Sigma](https://github.com/Neo23x0/evt2sigma) helps you with the rule creation. It generates a Sigma rule from a log entry.
[Evt2Sigma](https://github.com/Neo23x0/evt2sigma) helps you with the rule creation. It generates a Sigma rule from a log entry.

## Sigma2attack

Expand All @@ -291,7 +295,7 @@ Result once imported in the MITRE ATT&CK® Navigator ([online version](https://m

## S2AN

Similar to **Sigma2attack**, [S2AN](https://github.com/3CORESec/S2AN) is a pre-compiled binary for both Windows and GNU/Linux that generates [MITRE ATT&CK® Navigator](https://github.com/mitre/attack-navigator/) layers from a directory of Sigma rules.
Similar to **Sigma2attack**, [S2AN](https://github.com/3CORESec/S2AN) is a pre-compiled binary for both Windows and GNU/Linux that generates [MITRE ATT&CK® Navigator](https://github.com/mitre/attack-navigator/) layers from a directory of Sigma rules.

S2AN was developed to be used as a standalone tool or as part of a CI/CD pipeline where it can be quickly downloaded and executed without external dependencies.

Expand All @@ -317,11 +321,11 @@ These tools are not part of the main toolchain and maintained separately by thei
* [uncoder.io](https://uncoder.io/) - Online Translator for SIEM Searches
* [THOR](https://www.nextron-systems.com/2018/06/28/spark-applies-sigma-rules-in-eventlog-scan/) - Scan with Sigma rules on endpoints
* [Joe Sandbox](https://www.joesecurity.org/)
* [ypsilon](https://github.com/P4T12ICK/ypsilon) - Automated Use Case Testing
* [ypsilon](https://github.com/P4T12ICK/ypsilon) - Automated Use Case Testing
* [RANK VASA](https://globenewswire.com/news-release/2019/03/04/1745907/0/en/RANK-Software-to-Help-MSSPs-Scale-Cybersecurity-Offerings.html)
* [TA-Sigma-Searches](https://github.com/dstaulcu/TA-Sigma-Searches) (Splunk App)
* [TimeSketch](https://github.com/google/timesketch/commit/0c6c4b65a6c0f2051d074e87bbb2da2424fa6c35)
* [SIΣGMA](https://github.com/3CORESec/SIEGMA) - SIEM consumable generator that utilizes Sigma for query conversion
* [SIΣGMA](https://github.com/3CORESec/SIEGMA) - SIEM consumable generator that utilizes Sigma for query conversion

Sigma is available in some Linux distribution repositories:

Expand All @@ -333,10 +337,10 @@ If you want to contribute, you are more then welcome. There are numerous ways to

## Use it and provide feedback

If you use it, let us know what works and what does not work.
If you use it, let us know what works and what does not work.

E.g.
- Tell us about false positives (issues section)
- Tell us about false positives (issues section)
- Try to provide an improved rule (new filter) via [pull request](https://help.github.com/en/articles/editing-files-in-another-users-repository) on that rule

## Work on open issues
Expand All @@ -345,7 +349,7 @@ The github issue tracker is a good place to start tackling some issues others ra

## Provide Backends / Backend Features / Bugfixes

Various requests for sigmac (sigma converter) backends exist. Some backends are very limited and need features. We are working on a documentation on how to write new backends but our time for this project is currently mostly spent for issue resolutions.
Various requests for sigmac (sigma converter) backends exist. Some backends are very limited and need features. We are working on a documentation on how to write new backends but our time for this project is currently mostly spent for issue resolutions.

## Spread the word

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
title: Always Install Elevated Parent Child Correlated
id: 078235c5-6ec5-48e7-94b2-f8b5474379ea
description: This rule will looks any process with low privilege launching Windows Installer service (msiexec.exe) that tries to install MSI packages with SYSTEM privilege
#look for MSI start by low privilege user, write the process guid to the suspicious_guid variable
#look for child process from the suspicious_guid, alert if it's Windows Installer trying to install package with SYSTEM privilege
status: experimental
author: Teymur Kheirkhabarov (idea), Mangatas Tondang (rule), oscd.community
date: 2020/10/13
references:
- https://image.slidesharecdn.com/kheirkhabarovoffzonefinal-181117201458/95/hunting-for-privilege-escalation-in-windows-environment-48-638.jpg
tags:
- attack.privilege_escalation
- attack.t1548.002
logsource:
product: windows
category: process_creation
detection:
system_integrity:
IntegrityLevel: 'System'
system_user:
User: 'NT AUTHORITY\SYSTEM'
image_1:
Image|contains|all:
- '\Windows\Installer\'
- 'msi'
Image|endswith:
- 'tmp'
image_2:
Image|endswith: '\msiexec.exe'
child_of_suspicious_guid:
ParentProcessGuid: '%suspicious_guid%'
condition: write ProcessGuid from (event_id and image_2 and not system_user) to %suspicious_guid%; then if (child_of_suspicious_guid and event_id and image_1 and system_user) or (suspicious_guid and event_id and image_2 and system_user and integrity_level) -> alert
fields:
- EventID
- IntegrityLevel
- User
- Image
ParentProcessGuid
falsepositives:
- System administrator usage
- Penetration test
level: high
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
title: Stored Credentials in Fake Files
id: 692b979c-f747-41dc-ad72-1f11c01b110e
description: Search for accessing of fake files with stored credentials
status: experimental
author: Teymur Kheirkhabarov (idea), Ryan Plas (rule), oscd.community
date: 2020/10/05
references:
- https://image.slidesharecdn.com/kheirkhabarovoffzonefinal-181117201458/95/hunting-for-privilege-escalation-in-windows-environment-13-638.jpg
tags:
- attack.credential_access
- attack.t1555
logsource:
product: windows
service: security
detection:
selection:
EventID: 4663
AccessList|contains: '%%4416'
ObjectName|endswith:
- '\%POLICY_ID%\Machine\Preferences\Groups\Groups.xml'
- '\%FOLDER_NAME%\Unattend.xml'
condition: selection
fields:
- EventID
- AccessList
- ObjectName
falsepositives:
- Unknown
level: high
44 changes: 44 additions & 0 deletions rules-unsupported/win_remote_schtask.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
title: Remote Schtasks Creation
id: cf349c4b-99af-40fa-a051-823aa2307a84
status: experimental
description: Detects remote execution via scheduled task creation or update on the destination host
author: Jai Minton, oscd.community
date: 2020/10/05
references:
- https://drive.google.com/file/d/1lKya3_mLnR3UQuCoiYruO3qgu052_iS_/view
tags:
- attack.lateral_movement
- attack.persistence
- attack.execution
- attack.t1053.005
logsource:
product: windows
service: security
definition: 'The Advanced Audit Policy setting Object Access > Audit Other Object Access Events has to be configured to allow this detection (not in the baseline recommendations by Microsoft).'
detection:
selection1:
EventID: 4624
Logon_Type: 3
selection2:
EventID:
- 4698
- 4702
filter1:
Source_Network_Address:
- '::1'
- '127.0.0.1'
filter2:
Source_Network_Address: '-'
timeframe: 30d
condition: (selection1 and not filter1) or selection2 and not filter2
# where:
# selection1: TargetLogonID = selection2: SubjectLogonID, grouped by host over 30seconds | eventcount > 1
# Rule should trigger where the SubjectLogonID from event 4698 or 4702 is the same as the TargetLogonID from event 4624 with a Logon_Type of 3, in a 30second period, provided its from the same host.
# This logic would be similar to the Splunk 'Transaction' operator which groups related events over a timeframe.
# This takes both field values (e.g. Logon_ID), and an expression provided (e.g. startswith=(EventCode=4624) maxspan=30s) which occurs over the raw event log to find events, at which point a Union based on the criteria provided occurs to merge these events into a single transaction.
# This is similar to stats as an aggregation function, but allows you to see the raw text of events rather than to calculate stats on then, and it retains the raw event to allow an eval expression to occur for grouping. This is beneficial as fields such as LogonIDs are reused over time.
# By having this you can group logon events to their remote schtask creation event (as it is searching for a logon followed by a schtask creation) even by using a search timeframe over a long period of time e.g. 30days without running the risk of incorrectly grouping a logonID at one time, to a task creation at another.
# Rule logic is currently not supported by SIGMA.
falsepositives:
- Unknown
level: medium
Loading

0 comments on commit dceb13f

Please sign in to comment.