Skip to content

Commit

Permalink
Expected result pattern not always shows (aquasecurity#784)
Browse files Browse the repository at this point in the history
* Add expectedResultPattern to invalid test

when testing and try convert to numeric we didn't set expectedResultPattern value.

* check for auditconfig before using it

The current state is that when ever audit output is not what we search for we check for auditConfig output which is sometime empty and therefore create empty expected result as described in aquasecurity#694

* Fix issue about expectedResultPattern

expectedResultPattern not always shown and wasn't accurate enough 
Issue aquasecurity#705

* Add tests for ExpectedResult and fixes

Add tests for ExpectedResult with the new output and the verify that the fix is working

* Add missing flags

In some cases not having audit or audit_config flag would fail the test.
So added just a simple commands like echo something to solve this issue 
Also add bitmask checks

* Add example IAM policy

* Pass RotateKubeletServerCertificate related checks if it's not found (aquasecurity#767)

* Allow for environment variables to be checked in tests (aquasecurity#755)

* Initial commit for checking environment variables for etcd

* Revert config changes

* Remove redundant struct data

* Fix issues with failing tests

* Initial changes based on code review

* Add option to disable envTesting + Update docs

* Initial tests

* Finished testing

* Fix broken tests

* Add a total summary and always show all tests. (aquasecurity#759)

Whether the total summary is shown can be specified with an option.

Fixes aquasecurity#528

Signed-off-by: Christian Zunker <[email protected]>

* Update Readme.md file with link to Contribution guide (aquasecurity#754)

* Update License with the year and the owner name

Please add this to make your license agreement strong

* Updated Readme.md file with license and proper documentation links

I have added a proper license agreement to the documentation. Also shortened the links to the issues so that it does not break in any on the forks.

* Update LICENSE

* Update README.md

* Update README.md

* Remove erroneous license info

Co-authored-by: Liz Rice <[email protected]>

* Support auto-detect platform when running on EKS or GKE (aquasecurity#683)

* Support auto-detect platform when running on EKS or GKE

* Change to get platform name from `kubectl version`

* fix regexp and add test

* Update Server Version match for EKS

* try to get version info from api sever at first

* Change expected expectedResultPattern

Now expectedResultPattern is more verbose

* Update ops tests

* Fix unit tests

* Fix bitmask output syntax

* Changes to be committed:
	modified:   check/check.go
	modified:   check/test.go
	modified:   check/test_test.go
fix unit testing and test.go to resolve conflicts.

* Change found to flagFound

* add missing }

* change found to flag found

Co-authored-by: yoavrotems <[email protected]>
  • Loading branch information
lizrice and yoavrotems authored Dec 24, 2020
1 parent b6f619c commit 6452df7
Show file tree
Hide file tree
Showing 4 changed files with 696 additions and 392 deletions.
14 changes: 11 additions & 3 deletions check/check.go
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ type Check struct {
AuditOutput string `json:"-"`
AuditEnvOutput string `json:"-"`
AuditConfigOutput string `json:"-"`
DisableEnvTesting bool `json:"-"`
DisableEnvTesting bool `json:"-"`
}

// Runner wraps the basic Run method.
Expand Down Expand Up @@ -220,14 +220,22 @@ func (c *Check) execute() (finalOutput *testOutput, err error) {
// Try with the auditOutput first, and if that's not found, try the auditConfigOutput
t.auditUsed = AuditCommand
result := *(t.execute(c.AuditOutput))
if !result.found {

// Check for AuditConfigOutput only if AuditConfig is set
if !result.flagFound && c.AuditConfig != "" {
//t.isConfigSetting = true
t.auditUsed = AuditConfig
result = *(t.execute(c.AuditConfigOutput))
if !result.found && t.Env != "" {
if !result.flagFound && t.Env != "" {
t.auditUsed = AuditEnv
result = *(t.execute(c.AuditEnvOutput))
}
}

if !result.flagFound && t.Env != "" {
t.auditUsed = AuditEnv
result = *(t.execute(c.AuditEnvOutput))
}
res[i] = result
expectedResultArr[i] = res[i].ExpectedResult
}
Expand Down
69 changes: 49 additions & 20 deletions check/data
Original file line number Diff line number Diff line change
@@ -1,28 +1,31 @@
---
controls:
id: 1
text: "Master Checks"
text: "Test Checks"
type: "master"
groups:
- id: 1.1
text: "Kube-apiserver"
text: "First Group"
checks:
- id: 0
text: "flag is set"
audit: "echo \"Non empty command\""
tests:
test_items:
- flag: "--allow-privileged"
set: true

- id: 1
text: "flag is not set"
audit: "echo \"Non empty command\""
tests:
test_items:
- flag: "--basic-auth"
set: false

- id: 2
text: "flag value is set to some value"
audit: "echo \"Non empty command\""
tests:
test_items:
- flag: "--insecure-port"
Expand All @@ -33,6 +36,7 @@ groups:

- id: 3
text: "flag value is greater than or equal some number"
audit: "echo \"Non empty command\""
tests:
test_items:
- flag: "--audit-log-maxage"
Expand All @@ -43,6 +47,7 @@ groups:

- id: 4
text: "flag value is less than some number"
audit: "echo \"Non empty command\""
tests:
test_items:
- flag: "--max-backlog"
Expand All @@ -53,6 +58,7 @@ groups:

- id: 5
text: "flag value does not have some value"
audit: "echo \"Non empty command\""
tests:
test_items:
- flag: "--admission-control"
Expand All @@ -63,6 +69,7 @@ groups:

- id: 6
text: "test AND binary operation"
audit: "echo \"Non empty command\""
tests:
bin_op: and
test_items:
Expand All @@ -73,6 +80,7 @@ groups:

- id: 7
text: "test OR binary operation"
audit: "echo \"Non empty command\""
tests:
bin_op: or
test_items:
Expand All @@ -87,38 +95,29 @@ groups:

- id: 8
text: "test flag with arbitrary text"
audit: "echo \"Non empty command\""
tests:
test_items:
- flag: "644"
- flag: "permissions"
compare:
op: eq
value: "644"
value: "SomeValue"
set: true

- id: 9
text: "test permissions"
audit: "/bin/sh -c 'if test -e $config; then stat -c %a $config; fi'"
audit: "/bin/sh -c 'if test -e $config; then stat -c permissions=%a $config; fi'"
tests:
bin_op: or
test_items:
- flag: "644"
- flag: "permissions"
compare:
op: eq
op: bitmask
value: "644"
set: true
- flag: "640"
compare:
op: eq
value: "640"
set: true
- flag: "600"
compare:
op: eq
value: "600"
set: true

- id: 10
text: "flag value includes some value in a comma-separated list, value is last in list"
audit: "echo \"Non empty command\""
tests:
test_items:
- flag: "--admission-control"
Expand All @@ -129,6 +128,7 @@ groups:

- id: 11
text: "flag value includes some value in a comma-separated list, value is first in list"
audit: "echo \"Non empty command\""
tests:
test_items:
- flag: "--admission-control"
Expand All @@ -139,6 +139,7 @@ groups:

- id: 12
text: "flag value includes some value in a comma-separated list, value middle of list"
audit: "echo \"Non empty command\""
tests:
test_items:
- flag: "--admission-control"
Expand All @@ -149,6 +150,7 @@ groups:

- id: 13
text: "flag value includes some value in a comma-separated list, value only one in list"
audit: "echo \"Non empty command\""
tests:
test_items:
- flag: "--admission-control"
Expand All @@ -159,6 +161,7 @@ groups:

- id: 14
text: "check that flag some-arg is set to some-val with ':' separator"
audit: "echo \"Non empty command\""
tests:
test_items:
- flag: "some-arg"
Expand All @@ -169,7 +172,10 @@ groups:

- id: 15
text: "jsonpath correct value on field"
audit: "echo \"Non empty command\""
audit_config: "echo \"Non empty command\""
tests:
bin_op: or
test_items:
- path: "{.readOnlyPort}"
compare:
Expand All @@ -189,6 +195,8 @@ groups:

- id: 16
text: "jsonpath correct case-sensitive value on string field"
audit: "echo \"Non empty command\""
audit_config: "echo \"Non empty command\""
tests:
test_items:
- path: "{.stringValue}"
Expand All @@ -209,6 +217,8 @@ groups:

- id: 17
text: "jsonpath correct value on boolean field"
audit: "echo \"Non empty command\""
audit_config: "echo \"Non empty command\""
tests:
test_items:
- path: "{.trueValue}"
Expand All @@ -229,13 +239,17 @@ groups:

- id: 18
text: "jsonpath field absent"
audit: "echo \"Non empty command\""
audit_config: "echo \"Non empty command\""
tests:
test_items:
- path: "{.notARealField}"
set: false

- id: 19
text: "jsonpath correct value on nested field"
audit: "echo \"Non empty command\""
audit_config: "echo \"Non empty command\""
tests:
test_items:
- path: "{.authentication.anonymous.enabled}"
Expand All @@ -246,6 +260,8 @@ groups:

- id: 20
text: "yamlpath correct value on field"
audit: "echo \"Non empty command\""
audit_config: "echo \"Non empty command\""
tests:
test_items:
- path: "{.readOnlyPort}"
Expand All @@ -256,13 +272,17 @@ groups:

- id: 21
text: "yamlpath field absent"
audit: "echo \"Non empty command\""
audit_config: "echo \"Non empty command\""
tests:
test_items:
- path: "{.fieldThatIsUnset}"
set: false

- id: 22
text: "yamlpath correct value on nested field"
audit: "echo \"Non empty command\""
audit_config: "echo \"Non empty command\""
tests:
test_items:
- path: "{.authentication.anonymous.enabled}"
Expand All @@ -273,6 +293,8 @@ groups:

- id: 23
text: "path on invalid json"
audit: "echo \"Non empty command\""
audit_config: "echo \"Non empty command\""
tests:
test_items:
- path: "{.authentication.anonymous.enabled}"
Expand All @@ -283,13 +305,16 @@ groups:

- id: 24
text: "path with broken expression"
audit: "echo \"Non empty command\""
audit_config: "echo \"Non empty command\""
tests:
test_items:
- path: "{.missingClosingBrace"
- path: "{.missingClosingBrace}"
set: true

- id: 25
text: "yamlpath on invalid yaml"
audit: "echo \"Non empty command\""
tests:
test_items:
- path: "{.authentication.anonymous.enabled}"
Expand All @@ -300,6 +325,8 @@ groups:

- id: 26
text: "check regex op matches"
audit: "echo \"Non empty command\""
audit_config: "echo \"Non empty command\""
tests:
test_items:
- path: "{.currentMasterVersion}"
Expand All @@ -310,6 +337,7 @@ groups:

- id: 27
text: "check boolean flag with no value"
audit: "echo \"Non empty command\""
tests:
test_items:
- flag: "--peer-client-cert-auth"
Expand All @@ -320,6 +348,7 @@ groups:

- id: 28
text: "check boolean flag with false value"
audit: "echo \"Non empty command\""
tests:
test_items:
- flag: "--peer-client-cert-auth"
Expand Down Expand Up @@ -587,7 +616,7 @@ groups:
path: '{.readOnlyPort}'
set: false
scored: true
- id: 15
- id: 16
text: "parameter and config file don't have same default - parameter has bad value and config is not present - failing"
audit: "echo '--read-only-port=1'"
audit_config: "echo ''"
Expand Down
Loading

0 comments on commit 6452df7

Please sign in to comment.