diff --git a/.github/workflows/shellcheck.yml b/.github/workflows/shellcheck.yml new file mode 100644 index 000000000..13d2b53a2 --- /dev/null +++ b/.github/workflows/shellcheck.yml @@ -0,0 +1,15 @@ +name: Check all shell scripts +on: pull_request + +jobs: + shellcheck: + name: Run shellcheck on shell scripts + runs-on: ubuntu-22.04 + steps: + - uses: actions/checkout@v3 + with: + submodules: recursive + - name: Install dependencies + run: sudo apt-get update -y && sudo apt-get install -y shellcheck + - name: Run shellcheck + run: make check diff --git a/Makefile b/Makefile new file mode 100644 index 000000000..67a27b506 --- /dev/null +++ b/Makefile @@ -0,0 +1,3 @@ +.PHONY: check +check: + shellcheck generator/build/*.sh diff --git a/README.md b/README.md index 018d3c429..0b2be87be 100644 --- a/README.md +++ b/README.md @@ -1,9 +1,9 @@ # CFEngine Documentation This repository holds the sources for the technical -[CFEngine documentation](https://cfengine.com/docs) in +[CFEngine documentation](https://docs.cfengine.com/docs/) in markdown files. If you have a texteditor, know how to write -[markdown](http://daringfireball.net/projects/markdown/syntax) and +[markdown](https://daringfireball.net/projects/markdown/syntax) and English and would like to contribute to the CFEngine documentation, then we'd like to invite you to collaborate here! @@ -18,9 +18,9 @@ CFEngine users by correcting some of them. ## Writing Documentation The CFEngine documentation is written in regular -[markdown](http://daringfireball.net/projects/markdown/syntax), with some +[markdown](https://daringfireball.net/projects/markdown/syntax), with some extensions as documented below. Check out the -[cheatsheet](https://cfengine.com/docs/master/markdown-cheatsheet.html)! +[cheatsheet](https://docs.cfengine.com/docs/master/markdown-cheatsheet.html)! To keep the source readable in Git and workable with a broad range of tools, keep the line length in paragraphs below 78 characters. @@ -48,13 +48,15 @@ The content of the file will generate a HTML page - docs.cfengine.com/version/path-of-categories.html + docs.cfengine.com/docs/version/path-of-categories.html with equivalent location in the left-hand navigation tree. The documentation generation inserts a link to the source markdown in the footer of each page. +Note: Do not use markdown files prefixed with `.`. They are not picked up by the documentation generator. You can use `.` prefixed page titles. + ### META tags Additional META tags you should set are: @@ -96,7 +98,7 @@ META tag values will be interpreted literally, and cannot contain `:`, `[` and Image files are in the same directory as the markdown files that embed them. Give files unique names to avoid overwrites in the generated website. -See [Style Guide - Charts and graphs](#Charts_and_graphs) for style +See [Style Guide - Charts and graphs](#charts-and-graphs) for style requirements for images. *Warning:* It is important for images and other binary files to be marked as a @@ -122,7 +124,7 @@ To links to sections within the current page, you can also skip the page title: [Go to section][#Section in Current Page] For standard URLs and locations to link to, see the -[mapping](https://github.com/cfengine/documentation-generator/blob/master/_references.md). +[mapping](https://github.com/cfengine/documentation/generator/blob/master/_references.md). #### Automatic linking @@ -427,13 +429,13 @@ As a general note, avoiding abbreviations provides better readability. ## Technical reference documentation -* follow the [Policy Style Guide](manuals/policy-style.markdown) +* follow the [Policy Style Guide](guide/writing-and-serving-policy/policy-style.markdown) in examples and code snippets * use the appropriate lexer for syntax highlighting via Pygments Most important are the `cf3` lexer, as well as `bash`, `console`, `diff`, `shell-session` and `sql`. But Jekyll supports - [many more lexers](http://pygments.org/docs/lexers/) + [many more lexers](https://pygments.org/docs/lexers/) * avoid custom color schemes and hand-coded HTML * document the example after the example code @@ -582,15 +584,15 @@ Special Variables are documented within the page of their context. ## Publishing Jekyll is used to generate the HTML pages. The toolchain is available at -https://github.com/cfengine/documentation-generator. After you compile pages, +https://github.com/cfengine/documentation/generator. After you compile pages, Jekyll will place all files into the _site folder, without subdirectories. Commits in this repository trigger the documentation generator to run, which -then updates the contents of http://cfengine.com/docs/ +then updates the contents of https://docs.cfengine.com/docs/ The documentation generation creates a log file that lists undocumented syntax elements, ambiguous link targets and other stuff that can be improved at -https://cfengine.com/docs/master/cfdoc_log.html +https://docs.cfengine.com/docs/master/cfdoc_log.html ## License diff --git a/cheatsheet.markdown b/cheatsheet.markdown index a632952bc..a2d3df42f 100644 --- a/cheatsheet.markdown +++ b/cheatsheet.markdown @@ -50,15 +50,32 @@ For example I can link to [collecting functions][Functions#collecting functions] [top of the page](markdown-cheatsheet.html), to the [Markdown Cheatsheet](markdown-cheatsheet.html), [inside page](markdown-cheatsheet.html#Links) and [inside current page](markdown-cheatsheet.html#Links) **Note:** For known pages, see the -[_references.md](https://github.com/cfengine/documentation-generator/blob/master/_references.md) +[_references.md](https://github.com/cfengine/documentation/generator/blob/master/_references.md) file. -**NOTE:** Anchors with underscores are problematic! +##### Special Characters in link targets + +_Most_ (`¯\_(ツ)_/¯`) special characters are _okay_. For example: + +* Link targets with `/` (forward slashes) work + * ```[Export/Import][Settings#Export/Import]``` == [Export/Import][Settings#Export/Import] + +Anchors with _underscores_ are problematic, *may* need to be escaped. For example ```services_autorun``` in the MPF documentation the underscore needs to be escaped with a ```\```. **See also:** [`services_autorun` in the Masterfiles Policy Framework][Masterfiles Policy Framework#services\_autorun] +But not always! For example + +``` +**See also:** [cf_lock.lmdb][CFEngine directory structure#state/cf_lock.lmdb] +``` + +**See also:** [cf_lock.lmdb][CFEngine directory structure#state/cf_lock.lmdb] + +Backticks are problematic. It seems impossible to link to anchors that contain backticks. + ### Link to CFEngine keyword The documentation pre-processor will create those automatically. @@ -272,7 +289,7 @@ SELECT +++ b/README.md @@ -377,8 +377,12 @@ As a general note, avoiding abbreviations provides better readability. - * follow the [Policy Style Guide](manuals/policy-style.markdown) + * follow the [Policy Style Guide](guide/writing-and-serving-policy/policy-style.markdown) in examples and code snippets -* always run it through Pygments plus the appropriate lexer (only cf3 - supported for now) @@ -293,7 +310,7 @@ index 92555a2..b49c0bb 100644 +++ b/README.md @@ -377,8 +377,12 @@ As a general note, avoiding abbreviations provides better readability. - * follow the [Policy Style Guide](manuals/policy-style.markdown) + * follow the [Policy Style Guide](guide/writing-and-serving-policy/policy-style.markdown) in examples and code snippets -* always run it through Pygments plus the appropriate lexer (only cf3 - supported for now) @@ -535,8 +552,8 @@ updated with each new version. Variables that are defined in the front matter (thats the content between the three dashes at the top) or in -[_config.yaml](https://github.com/cfengine/documentation-generator/blob/master/_config.yml) -in the documentation-generator repository can be used directly within markdown. +[_config.yaml](https://github.com/cfengine/documentation/generator/blob/master/_config.yml) +in the documentation/generator repository can be used directly within markdown. For example this is the '{{site.CFE_manuals_version}}' version of the documentation. That variable comes from _config.yaml. diff --git a/enterprise-cfengine-guide/Host-info-page.png b/enterprise-cfengine-guide/Host-info-page.png index 979618d9f..4fe5e9052 100644 Binary files a/enterprise-cfengine-guide/Host-info-page.png and b/enterprise-cfengine-guide/Host-info-page.png differ diff --git a/enterprise-cfengine-guide/Hosts-app-custom-tree-geographic-region.png b/enterprise-cfengine-guide/Hosts-app-custom-tree-geographic-region.png index af4523b33..6074eb1df 100644 Binary files a/enterprise-cfengine-guide/Hosts-app-custom-tree-geographic-region.png and b/enterprise-cfengine-guide/Hosts-app-custom-tree-geographic-region.png differ diff --git a/enterprise-cfengine-guide/Hosts-app-overview.png b/enterprise-cfengine-guide/Hosts-app-overview.png index 0b2c02fdb..b420c8585 100644 Binary files a/enterprise-cfengine-guide/Hosts-app-overview.png and b/enterprise-cfengine-guide/Hosts-app-overview.png differ diff --git a/enterprise-cfengine-guide/federated-reporting.markdown b/enterprise-cfengine-guide/federated-reporting.markdown index 61e34c290..ac1d67f00 100644 --- a/enterprise-cfengine-guide/federated-reporting.markdown +++ b/enterprise-cfengine-guide/federated-reporting.markdown @@ -33,6 +33,7 @@ are provided at each stage of installation and setup that follows. * [API Setup][Federated Reporting#API Setup] * [Disable Feeder][Federated Reporting#Disable Feeder] * [Uninstall][Federated Reporting#Uninstall] +* [Superhub Upgrade][Federated Reporting#Superhub Upgrade] ## Requirements ## @@ -127,7 +128,7 @@ install CFEngine Hub on a Superhub as well as Feeder hubs. ![Enable Hub Management](fr-hub-management-enabled.png) On the Superhub and all Feeders enable the Hub management -app by [Opening Settings][Settings#opening settings] then +app by [Opening Settings][Settings#opening settings] then selecting [Manage Apps][Settings#manage apps] and finally by clicking the `On` radio button for Hub management in the Status column. @@ -187,6 +188,74 @@ There are situations where feeder hubs may have hosts with duplicate hostkeys: - hosts may be cloned and not have their hostkey refreshed by running `cf-key` and refreshing `$(sys.workdir)/ppkeys/localhost.pub`. In the first case you will likely want to remove entries for hosts which are not the latest since the latest data will be most accurate. + +There are two options available for handling these situations depending on your environment: Distributed Cleanup or Handle Duplicate Hostkeys. + +### Distributed cleanup ### +This is the most thorough, performant and automated option. +This utility is a python script which runs on the superhub, searches for the most recent contact for each host, then communicates with the appropriate feeders to delete stale hosts. + +A few pre-requisites must be handled before enabling this utility: + +- gather the admin passwords for the superhub and all feeders +- ensure that the attached feeders resolve their hostnames properly + (you may need to add entries to your DNS or /etc/hosts) +- ensure python3 and urllib3 module for python3 are installed + +On Debian/Ubuntu: + +``` bash +# apt install -qy python3 python3-urllib3 +``` + +On RedHat/CentOS versions 7 and above: + +``` bash +# yum install -qy python3 python3-urllib3 +``` + +On RedHat/CentOS 6 you will have to install python3 manually and the install urllib3 with pip3. +Python 3 is actually quite easy to install with the standard [building python](https://docs.python.org/3.10/using/unix.html#building-python) instructions. + +After those steps, ensure `cfengine_mp_fr_enable_distributed_cleanup` is present in augments for your superhub and all feeders. + +```json +{ + "classes": { + "cfengine_mp_fr_enable_distributed_cleanup": ["any::"] + } +} +``` +(Note that this augment should be in addition to any others that you need such as `cfengine_mp_fr_dependencies_auto_install`) + +Let the policy run a few times on superhub and feeders. +This will distribute the needed certificates from feeders to superhub so that the script on the superhub may securely connect to the feeder API endpoints. + +When run manually for the first time the utility will create a limited privileges user to view and delete hosts on the feeders. +You will need to enter the following information at the prompts when running the utility manually: +- admin password for the superhub +- email address for the fr_distributed_cleanup limited privileges user +- admin password for each feeder + +After confirming all feeder certs and public keys are present on the superhub, run the distributed cleanup script manually. + +```bash +# ls /opt/cfengine/federation/cftransport/distributed_cleanup/ +superhub.pub feeder1.cert feeder1.pub feeder2.cert feeder2.pub + +# /opt/cfengine/federation/bin/distributed_cleanup.py +Enter admin credentials for superhub https://superhub.domain/api: +Enter email for fr_distributed_cleanup accounts: +Enter admin credentials for feeder1 at https://feeder1.domain/api: +Enter admin credentials for feeder2 at https://feeder2.domain/api: +``` + +The passwords are only kept for the duration of the script execution and are not saved. + +The policy will now run the distributed cleanup utility every agent run and cleanup any hosts which are stale on feeders leaving only the most recently contacts host for each unique hostkey. + +### Handle duplicate hostkeys ### +The other option removes duplicates during each import cycle. An augment is available to enable moving duplicated host data to a `dup` schema for analysis. The host data which has the most recent `hosts.lastreporttimestamp` will be kept in the `public` schema and all other data will be moved to the `dup` domain (schema). This feature is disabled by default. @@ -218,7 +287,7 @@ An API may be used instead of the UI. This could be used to automate the setup of infrastructure related to Federated Reporting and Feeder hubs. Command line examples follow using [curl](https://curl.haxx.se/) and -[cf-remote](https://github.com/cfengine/core/tree/master/contrib/cf-remote). +[cf-remote](https://github.com/cfengine/cf-remote). Some environment variables should be set according to your environment so that you can simply copy/paste steps as you go. @@ -231,6 +300,8 @@ $ export FEEDER=34.244.118.58 $ export FEEDER_BS=172.31.43.102 # _BS is bootstrap IP ``` +In these examples we use the `admin` account because the Admin Role which this user has contains the proper Role Based Access Control privileges to access the needed API endpoints. + ### Stop cf-execd on the superhub and feeder We don't want periodic agent runs to get in our ways so let's disable @@ -545,6 +616,10 @@ $ cat > target-state-off.json $ curl -k -i -s -X PUT -u admin:$PASSWORD https://$FEEDER/api/fr/hub-state -d @target-state-off.json --header "Content-Type: application/json" ``` +```console +$ curl -k -i -s -X POST -u admin:$PASSWORD https://$FEEDER/api/fr/federation-config +``` + ### Uninstall without API Edit ```/opt/cfengine/federation/cfapache/federation-config.json``` on the feeder @@ -570,52 +645,190 @@ you wish to disable and change the top-level `target_state` property value to `o ### Remove Feeder from Mission Portal Hub Management At this time it is not possible to remove a connected hub in the Mission Portal Hub -management app. The recommended way to remove a connected hub is with the API. +management app. + +* List all feeders to find the id value. Use of ```jq``` is optional for pretty printing the JSON. + + (Set approprivate values in your shell for `PASSWORD` and `SUPERHUB`) + ```console + $ curl -k -s -X GET -u admin:$PASSWORD https://$SUPERHUB/api/fr/remote-hub | jq '.' + ``` + + ```json + { + "id": 1, + "hostkey": "SHA=cd4be31f20f0c7d019a5d3bfe368415f2d34fec8af26ee28c4c123c6a0af49a2", + "api_url": "https://100.90.80.70", + "ui_name": "feeder1", + "role": "feeder", + "target_state": "on", + "transport": { + "mode": "pull_over_rsync", + "ssh_user": "cftransport", + "ssh_host": "172.32.1.20", + "ssh_pubkey": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVGoBB3zLKfVTzDNum/JWlmNJrSuDGrhTW1ZGtZEKjxFViFr4j0F8s6gIr5KOMcWtd91XvW6klpCPqKH3lfY767AI/RQa8JgVXgtvUG8rkD+gJ/wzGJm+VoGpxxs9dyBgSOtkaOSIDc574Om8dBR8enRcgxo1cNpvDVLVYKx9IzqhBwqp1gzEtGoIi+CDoGmoj1BT9XTlCRvGXYmSSBrgLARVO2mh5iqhP0XRVCp9Ki6OB9vMcs9rxIgQaPt8tVCt7/FK03IXrWPUsJC4M/kXiaKgHlE96H0CEvYl7GczaIU2NN5AHXZlviL79Zb8kOcUzsMdKv40G9YVa7/kyDOUX root@ip-172-32-1-20", + "ssh_fingerprint": "ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF18li5PyCyVy27+Lv09HDRxhyEnlL+zK++WaLc78W+Gji5i2VSRDg/jVV0xU2ZUmkohULZ66OmI5/sCOOIa3XU=\nssh-ed25519" + }, + "statistics": [] + } + { + "id": 2, + "hostkey": "SHA=30b6bb15fb94c9b7e386521bbe566934d266db2f6f63cd85f5e6fc406d11110b", + "api_url": "https://100.90.80.60", + "ui_name": "feeder2", + "role": "feeder", + "target_state": "on", + "transport": { + "mode": "pull_over_rsync", + "ssh_user": "cftransport", + "ssh_host": "172.32.1.21", + "ssh_pubkey": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDin59ffTXhtQxahrYkqNi3x36XIO08GnOvvVe3s+DmuT3kBn8Lh4P30kOVONSGKcfNZLnWVPrk2qqNWuEi6xg861G1kXqce02c26BW+4L/tnz86/kmTBGc2vb6d1NpEKA/1bg6bMf1da+EInxuMsS+yOWCe+s6DJ00bg6iCnmlLYtzAkMXmXK5QgVG6AImJXqG1Px5DlsRcKto00J8WJswfTpQXbZbuog4J6Ltm/J4DQW1/x7pEJby/r+/lKPJWp19t0gaGXfsxwHEPFK6YC8zmFzkBeqiVpAizhs7G8mZDgAAhMyY8d2eYIp+hDIFpfQA3aHHr0L7emsFeDa/rExt root@ip-172-32-1-21", + "ssh_fingerprint": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC7HE4qJfTLP9j02jZnkpTpUMCBiFzmAemgvIPcJjWJVNcawh1hpGSsWjw9EM1kwn7J6fWrjEkY8lTi2pNTnobL9qt+oQvwFqUvs5EZ8gAVIAyDjKE8GLckZRt8VGxLWMtOlBKaAmPBn0eFP6ToPqnPygJiiM05vKtxPui1xuCTrW+rXShtolUJLwwGH2APcDqjKAdZceQK4nybJzk4J1P77sJc+9IlHJCTpfj8AQEbh/Z3cHtNKauaz1mhDn5YT/QWwzKavGlqFSlSDwLXT2go6P6FoSaVYTV45V9l7q6ahEy3zEe7+7psMFVucS512qYFEKn5FoSIVQLgT3I8MfI1\necdsa-sha2-nistp256" + }, + "statistics": [] + } + ``` + +* Determine the id from the "id" property value and delete the remote hub with the API. In this case +we use the number "1". -First list all feeders to find the id value. Use of ```jq``` is optional for pretty printing the JSON. + ```console + root@superhub: ~# REMOTE_HUB_ID=1 + root@superhub: ~# curl -k -s -X DELETE -u admin:$PASSWORD https://$SUPERHUB/api/fr/remote-hub/$REMOTE_HUB_ID + ``` -```console -$ curl -k -s -X GET -u admin:$PASSWORD https://$SUPERHUB/api/fr/remote-hub | jq '.' -``` +* Remove the feeder from `/opt/cfengine/federation/cfapache/federation-config.json`. Replace "id-1" below with the appropriate id from the previous steps. -```json -{ - "id": 1, - "hostkey": "SHA=cd4be31f20f0c7d019a5d3bfe368415f2d34fec8af26ee28c4c123c6a0af49a2", - "api_url": "https://100.90.80.70", - "ui_name": "feeder1", - "role": "feeder", - "target_state": "on", - "transport": { - "mode": "pull_over_rsync", - "ssh_user": "cftransport", - "ssh_host": "172.32.1.20", - "ssh_pubkey": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVGoBB3zLKfVTzDNum/JWlmNJrSuDGrhTW1ZGtZEKjxFViFr4j0F8s6gIr5KOMcWtd91XvW6klpCPqKH3lfY767AI/RQa8JgVXgtvUG8rkD+gJ/wzGJm+VoGpxxs9dyBgSOtkaOSIDc574Om8dBR8enRcgxo1cNpvDVLVYKx9IzqhBwqp1gzEtGoIi+CDoGmoj1BT9XTlCRvGXYmSSBrgLARVO2mh5iqhP0XRVCp9Ki6OB9vMcs9rxIgQaPt8tVCt7/FK03IXrWPUsJC4M/kXiaKgHlE96H0CEvYl7GczaIU2NN5AHXZlviL79Zb8kOcUzsMdKv40G9YVa7/kyDOUX root@ip-172-32-1-20", - "ssh_fingerprint": "ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF18li5PyCyVy27+Lv09HDRxhyEnlL+zK++WaLc78W+Gji5i2VSRDg/jVV0xU2ZUmkohULZ66OmI5/sCOOIa3XU=\nssh-ed25519" - }, - "statistics": [] -} -{ - "id": 2, - "hostkey": "SHA=30b6bb15fb94c9b7e386521bbe566934d266db2f6f63cd85f5e6fc406d11110b", - "api_url": "https://100.90.80.60", - "ui_name": "feeder2", - "role": "feeder", - "target_state": "on", - "transport": { - "mode": "pull_over_rsync", - "ssh_user": "cftransport", - "ssh_host": "172.32.1.21", - "ssh_pubkey": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDin59ffTXhtQxahrYkqNi3x36XIO08GnOvvVe3s+DmuT3kBn8Lh4P30kOVONSGKcfNZLnWVPrk2qqNWuEi6xg861G1kXqce02c26BW+4L/tnz86/kmTBGc2vb6d1NpEKA/1bg6bMf1da+EInxuMsS+yOWCe+s6DJ00bg6iCnmlLYtzAkMXmXK5QgVG6AImJXqG1Px5DlsRcKto00J8WJswfTpQXbZbuog4J6Ltm/J4DQW1/x7pEJby/r+/lKPJWp19t0gaGXfsxwHEPFK6YC8zmFzkBeqiVpAizhs7G8mZDgAAhMyY8d2eYIp+hDIFpfQA3aHHr0L7emsFeDa/rExt root@ip-172-32-1-21", - "ssh_fingerprint": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC7HE4qJfTLP9j02jZnkpTpUMCBiFzmAemgvIPcJjWJVNcawh1hpGSsWjw9EM1kwn7J6fWrjEkY8lTi2pNTnobL9qt+oQvwFqUvs5EZ8gAVIAyDjKE8GLckZRt8VGxLWMtOlBKaAmPBn0eFP6ToPqnPygJiiM05vKtxPui1xuCTrW+rXShtolUJLwwGH2APcDqjKAdZceQK4nybJzk4J1P77sJc+9IlHJCTpfj8AQEbh/Z3cHtNKauaz1mhDn5YT/QWwzKavGlqFSlSDwLXT2go6P6FoSaVYTV45V9l7q6ahEy3zEe7+7psMFVucS512qYFEKn5FoSIVQLgT3I8MfI1\necdsa-sha2-nistp256" - }, - "statistics": [] -} -``` + ```console + root@superhub: ~# contents=$(jq 'del(.remote_hubs ."id-1")' /opt/cfengine/federation/cfapache/federation-config.json) && echo "${contents}" > /opt/cfengine/federation/cfapache/federation-config.json + ``` -Determine the id from the "id" property value and delete the remote hub. In this case -we use the number "1". +* Remove items associated with this feeder in the `cfdb` database. -```console -$ curl -k -s -X DELETE -u admin:$PASSWORD https://$SUPERHUB/api/fr/remote-hub/1 -``` + Determine the cfdb-specific `hub_id`. + + ```console + root@superhub: ~# /var/cfengine/bin/psql cfdb -c "select * from __hubs" + ``` + + Typical output would be like the following. + + ``` + hub_id | hostkey | last_import_ts + --------+----------------------------------------------------------------------+---------------- + 0 | SHA=50d370f41c81b3e119506befecc5deaa63c0f1d9039f674c68f9253a07f7ad84 | + 1 | SHA=bfd6f580f9d19cb190139452f068f38f843bf9227ca3515f7adfecfa39f68728 | + (2 rows) + ``` + + `hub_id` of `0` is the superhub. The others are the feeders. + In this case, it happens that the `hub_id` is also "1" so we will use that in the following queries. + +* Execute the following commands to remove the namespace for that feeder as well as the entry in the `__hubs` table. + + ```console + root@superhub: ~# /var/cfengine/bin/psql cfdb -c 'drop schema "hub_1" cascade;' + root@superhub: ~# /var/cfengine/bin/psql cfdb -c "delete from __hubs where hub_id = 1" + ``` + +* On the feeder, replace `/opt/cfengine/federation/cfapache/federation-config.json` with the following content. + + If you wish to re-add this feeder to a superhub, change "target_state" from "off" to "on". + Remember to trigger or wait for an agent run for the change from off to on to take effect. + + ```json + { + "hostname": null, + "role": "feeder", + "target_state": "off", + "remote_hubs": { } + } + ``` + + +* On 3.15.x and greater feeders, also run the following commands to truncate two tables: + + ```console + root@feeder: ~# /var/cfengine/bin/psql cfsettings -c 'TRUNCATE remote_hubs' + root@feeder: ~# /var/cfengine/bin/psql cfsettings -c 'TRUNCATE federated_reporting_settings' + ``` + +## Superhub Upgrade ## + +Starting with 3.15.6 and 3.18.2 superhubs can be directly upgraded by installing the new hub package. + +For versions 3.15.5, and 3.18.1 and older the superhub can not be directly upgraded by installing a new binary package and the hub software must be uninstalled and re-installed. + +### Uninstall/re-install ### + +Typically the superhub doesn't have unique information or serve policy. +This makes it reasonable and easy to upgrade the superhub with a fresh install. +If there are unique items like custom reports, dashboards, alerts or conditions on the superhub which need to be preserved +you may use the [Import & Export API] or Mission Portal Settings UI to export and then import after upgrading. + +Follow this procedure: + +* Download the new version from the [Enterprise Downloads Page][enterprise software download page] +* Export any items from Mission Portal you wish to migrate +* Stop all cfengine services on the superhub + + ```console + # systemctl stop cfengine3 + ``` + +* Uninstall cfengine hub + + ```console + # rpm -e cfengine-nova-hub + ``` + + or + + ```console + # apt-get remove cfengine-nova-hub + ``` + +* Cleanup directories + + ```console + # rm -rf /var/cfengine + # rm -rf /opt/cfengine + ``` +* Install new version of cfengine +* Confirm succesful installation + + ```console + # grep -i err /var/log/CFEngineInstall.log + ``` + +* Bootstrap the superhub to itself + + ```console + # cf-agent --bootstrap + ``` + +* Reconfigure all feeders (3.15 series and newer, skip for 3.12 series feeder hubs) + * edit `/opt/cfengine/federation/cfapache/federation-config.json` to remove all entries in the `remote_hubs` property. + similar to the following: + + ```json + { + "hostname": null, + "role": "feeder", + "target_state": "on", + "remote_hubs": { } + } + ``` + + * On 3.15.x and greater feeders, also truncate the `remote_hubs` table: + + ```console + # /var/cfengine/bin/psql cfsettings -c 'TRUNCATE remote_hubs' + ``` +* Reinstall and configure the superhub as described in [Installation][Federated Reporting#Installation] +* Import any saved information into Mission Portal via the [Import & Export API] or Mission Portal Settings UI +* Wait 20 minutes for federated reporting to be updated from feeders to superhub + + or + + * run `cf-agent -KI` on each feeder, and then `cf-agent -KI` on the superhub to manually force a Federated Reporting collection cycle. diff --git a/enterprise-cfengine-guide/fr-edit-hub-disable.png b/enterprise-cfengine-guide/fr-edit-hub-disable.png index fa0b224c9..894643f9f 100644 Binary files a/enterprise-cfengine-guide/fr-edit-hub-disable.png and b/enterprise-cfengine-guide/fr-edit-hub-disable.png differ diff --git a/enterprise-cfengine-guide/host-action-buttons.png b/enterprise-cfengine-guide/host-action-buttons.png index 521675538..53a73c7d4 100644 Binary files a/enterprise-cfengine-guide/host-action-buttons.png and b/enterprise-cfengine-guide/host-action-buttons.png differ diff --git a/enterprise-cfengine-guide/host-info-collect-reports.png b/enterprise-cfengine-guide/host-info-collect-reports.png index 4a9753f70..9cf08520f 100644 Binary files a/enterprise-cfengine-guide/host-info-collect-reports.png and b/enterprise-cfengine-guide/host-info-collect-reports.png differ diff --git a/enterprise-cfengine-guide/host-info-delete-host.png b/enterprise-cfengine-guide/host-info-delete-host.png index 762a00958..4f36bc754 100644 Binary files a/enterprise-cfengine-guide/host-info-delete-host.png and b/enterprise-cfengine-guide/host-info-delete-host.png differ diff --git a/enterprise-cfengine-guide/host-info-get-url.png b/enterprise-cfengine-guide/host-info-get-url.png index 8648c8744..32cbe9c4e 100644 Binary files a/enterprise-cfengine-guide/host-info-get-url.png and b/enterprise-cfengine-guide/host-info-get-url.png differ diff --git a/enterprise-cfengine-guide/host-info-run-agent.png b/enterprise-cfengine-guide/host-info-run-agent.png index bbc798288..dda9d5345 100644 Binary files a/enterprise-cfengine-guide/host-info-run-agent.png and b/enterprise-cfengine-guide/host-info-run-agent.png differ diff --git a/enterprise-cfengine-guide/hub_administration/custom-https-certificate.markdown b/enterprise-cfengine-guide/hub_administration/custom-https-certificate.markdown index d5bc82fb9..9d9b1788e 100644 --- a/enterprise-cfengine-guide/hub_administration/custom-https-certificate.markdown +++ b/enterprise-cfengine-guide/hub_administration/custom-https-certificate.markdown @@ -9,7 +9,7 @@ When first installed a self-signed ssl certificate is automatically generated and used to secure Mission Portal and API communications. You can change this certificate out with a custom one by replacing `/var/cfengine/httpd/ssl/certs/.cert` and -`/var/cfengine/httpd/ssl/private/.cert` where hostname is the fully +`/var/cfengine/httpd/ssl/private/.key` where hostname is the fully qualified domain name of the host. After installing the certificate please make sure that the certificate diff --git a/enterprise-cfengine-guide/hub_administration/lookup-license-info.markdown b/enterprise-cfengine-guide/hub_administration/lookup-license-info.markdown index 570f97c82..fbe0d2232 100644 --- a/enterprise-cfengine-guide/hub_administration/lookup-license-info.markdown +++ b/enterprise-cfengine-guide/hub_administration/lookup-license-info.markdown @@ -27,7 +27,13 @@ $ curl -u admin http://localhost/api/ Run as `root` from the hub itself. ```console -# cf-hub -Fvn | grep -i expiring -2016-07-11T15:54:23+0000 verbose: Found 25 CFEngine Enterprise licenses, expiring on 2222-12-25 for FREE ENTERPRISE - http://cfengine.com/terms for terms +[root@hub ~]# cf-hub --show-license +License file: /var/cfengine/licenses/hub-SHA=d13c14c3dc46ef1c5824eb70ffae3a1d1c67c7ce70a1e8e8634b1324d0041131.dat +License status: Valid +License count: 50 +Company name: CFEngine (hub.example.com) +License host key: SHA=2e5c7d9636c5644d023d71859f3296755f8d53d5d183af98efc1540655731fcc +Expiration date: 3018-01-01 +Utilization: 20/50 (Approximate) ``` diff --git a/enterprise-cfengine-guide/hub_administration/policy-deployment.markdown b/enterprise-cfengine-guide/hub_administration/policy-deployment.markdown index 095634066..c147e381a 100644 --- a/enterprise-cfengine-guide/hub_administration/policy-deployment.markdown +++ b/enterprise-cfengine-guide/hub_administration/policy-deployment.markdown @@ -26,12 +26,17 @@ the contents of `masterfiles` to it with the following commands (assuming you are already in your local repository checkout): ``` +echo cf_promises_validated >> .gitignore +echo cf_promises_release_id >> .gitignore cp -r /var/cfengine/masterfiles/* . +rm -f cf_promises_validated cf_promises_release_id git add * git commit -m 'Initial masterfiles check in' git push origin master ``` +**Note:** `cf_promises_validated` and `cf_promises_release_id` should be explicitly excluded from VCS as shown above. They are generated files and involved in controlling policy updates. If these files are checked into the repository it can create issues with policy distribution. + ## Requirements You must have the following: diff --git a/enterprise-cfengine-guide/hub_administration/reset-admin-creds.markdown b/enterprise-cfengine-guide/hub_administration/reset-admin-creds.markdown index 5e8ded34b..49d7eeaa9 100644 --- a/enterprise-cfengine-guide/hub_administration/reset-admin-creds.markdown +++ b/enterprise-cfengine-guide/hub_administration/reset-admin-creds.markdown @@ -23,3 +23,68 @@ To reset the CFEngine admin user run the following sql as root on your hub root@hub:~# psql cfsettings < cfsettings-setadminpassword.sql ``` +## Internal credentials + +Two internal credentials are present in an Enterprise Hub: Mission Portal API credentials and CFE Robot credentials. + +If these credentials are not synchronized between configuration files and database, errors can occur in the Mission Portal UI as well as in hub policy. + +Rotating these credentials is optional and can be performed at an interval based on your specific security policy. + +### Mission Portal API credentials + +These credentials enable Mission Portal to authenticate to the backend API. + +If these credentials are not synchronized properly you can get "Authentication failed" messages in the Mission Portal UI. + +To rotate these credentials execute the following shell script on the hub and then restart the system with `systemctl restart cfengine3` or similar. + +```bash +#!/usr/bin/env bash +# rotate_mp_credentials.sh +pwgen() { + dd if=/dev/urandom bs=1024 count=1 2>/dev/null | tr -dc 'a-zA-Z0-9' | fold -w $1 | head -n 1 +} +PREFIX=/var/cfengine # adjust if needed, this is the default +MP_PW=`pwgen 40` +SECRETS_FILE="$PREFIX/httpd/secrets.ini" # 3.16 version and newer +if [ -f "$SECRETS_FILE" ]; then + sed -i "/mp_client_secret/s/=.*/=$MP_PW/" $SECRETS_FILE +else + APPSETTINGS_FILE="$PREFIX/httpd/htdocs/application/config/appsettings.php" + sed -i "/MP_CLIENT_SECRET/c\$config['MP_CLIENT_SECRET'] = '$MP_PW';" $APPSETTINGS_FILE +fi +echo "UPDATE oauth_clients SET client_secret = '$MP_PW' where client_id = 'MP'" | $PREFIX/bin/psql cfsettings +``` + +### CFE Robot credentials + +These credentials are used by PHP CLI scripts to authenticate to the backend API. + +If these credentials are out of sync or incorrect you will see errors like "500 Internal Server Error" in `/var/cfengine/httpd/logs/application/` logs. + + +Execute the following shell script to rotate and synchronize the CFE Robot credentials and then restart the system with `systemctl restart cfengine3` or similar. + +```bash +#!/usr/bin/env bash +# rotate_cfrobot_credentials.sh +pwgen() { + dd if=/dev/urandom bs=1024 count=1 2>/dev/null | tr -dc 'a-zA-Z0-9' | fold -w $1 | head -n 1 +} +PREFIX=/var/cfengine # adjust if needed, this is the default +pwhash() { + echo -n "$1" | "$PREFIX/bin/openssl" dgst -sha256 | awk '{print $2}' +} +CFE_ROBOT_PW=`pwgen 40` +SECRETS_FILE="$PREFIX/httpd/secrets.ini" # 3.16 version and newer +if [ -f "$SECRETS_FILE" ]; then + sed -i "/cf_robot_password/s/=.*/=$CFE_ROBOT_PW/" $SECRETS_FILE +else + CFROBOT_FILE="$PREFIX/httpd/htdocs/application/config/cf_robot.php" + sed -i "/CFE_ROBOT_PASSWORD/c\$config['CFE_ROBOT_PASSWORD'] = \"$CFE_ROBOT_PW\";" $CFROBOT_FILE +fi +CFE_ROBOT_PW_SALT=`pwgen 10` +CFE_ROBOT_PW_HASH=`pwhash "$CFE_ROBOT_PW_SALT$CFE_ROBOT_PW"` +echo "UPDATE users SET password = 'SHA=$CFE_ROBOT_PW_HASH', salt = '$CFE_ROBOT_PW_SALT' WHERE username = 'CFE_ROBOT'" | "$PREFIX/bin/psql" cfsettings +``` diff --git a/enterprise-cfengine-guide/install-get-started.markdown b/enterprise-cfengine-guide/install-get-started.markdown index 6e5e47df3..6aedd416d 100644 --- a/enterprise-cfengine-guide/install-get-started.markdown +++ b/enterprise-cfengine-guide/install-get-started.markdown @@ -45,53 +45,7 @@ default. Metrics must match `monitoring_include` in the appropriate The [Masterfiles Policy Framework][Masterfiles Policy Framework] uses `body report_data_select default_data_select_policy_hub` to specify metrics that should be collected from policy hubs and `default_data_select_host` to specify -metrics that should be collected from non hubs. - -For example: - -To collect all metrics from hubs: - -```cf3 -body report_data_select default_data_select_policy_hub -# @brief Data to collect from policy servers by default -# -# By convention variables and classes known to be internal, (having no -# reporting value) should be prefixed with an underscore. By default the policy -# framework explicitly excludes these variables and classes from collection. -{ - metatags_include => { "inventory", "report" }; - metatags_exclude => { "noreport" }; - promise_handle_exclude => { "noreport_.*" }; - monitoring_include => { ".*" }; -} -``` - -To collect ```cpu```, ```loadavg``` , ```diskfree```, ```swap_page_in```, -```cpu_utilization```, ```swap_utilization```, and ```memory_utilization``` from -non hubs: - -```cf3 -body report_data_select default_data_select_host -# @brief Data to collect from remote hosts by default -# -# By convention variables and classes known to be internal, (having no -# reporting value) should be prefixed with an underscore. By default the policy -# framework explicitly excludes these variables and classes from collection. -{ - metatags_include => { "inventory", "report" }; - metatags_exclude => { "noreport" }; - promise_handle_exclude => { "noreport_.*" }; - monitoring_include => { - "cpu", - "loadavg", - "diskfree", - "swap_page_in", - "cpu_utilization", - "swap_utilization", - "memory_utilization", - }; -} -``` +metrics that should be collected from non hubs. Augments can be used to [configure which metrics should be collected][Masterfiles Policy Framework#Configure Enterprise Measurement/Monitoring Collection] for central reporting. ### Review settings diff --git a/enterprise-cfengine-guide/license-info.png b/enterprise-cfengine-guide/license-info.png new file mode 100644 index 000000000..bf9dd5273 Binary files /dev/null and b/enterprise-cfengine-guide/license-info.png differ diff --git a/enterprise-cfengine-guide/license-status-report.png b/enterprise-cfengine-guide/license-status-report.png new file mode 100644 index 000000000..5f7de78a1 Binary files /dev/null and b/enterprise-cfengine-guide/license-status-report.png differ diff --git a/enterprise-cfengine-guide/license.markdown b/enterprise-cfengine-guide/license.markdown new file mode 100644 index 000000000..0edab3df5 --- /dev/null +++ b/enterprise-cfengine-guide/license.markdown @@ -0,0 +1,17 @@ +--- +layout: default +title: License +published: true +--- + +## License information + +License information can be obtained from the About page under the user menu. + +Information about installed license. + +## License status report + +Click on the host count in the header to open the License Status Report. It contains information about the current license utilization. + +Report showing current license utilization. diff --git a/enterprise-cfengine-guide/measurements.markdown b/enterprise-cfengine-guide/measurements.markdown index 53983cc5d..cf65d213f 100644 --- a/enterprise-cfengine-guide/measurements.markdown +++ b/enterprise-cfengine-guide/measurements.markdown @@ -18,7 +18,7 @@ If multiple hosts are selected in the menu on the left, then you can select one * CPU(ALL) (in %) You can reduce the number of graphs by selecting a sub-set of hosts from the menu on the left. If only a -single host is selected, then a number of graphs for various measurements will be displayed for this host. Which exact measurements are reported depends on how [`cf-monitord`][cf-monitord] is configured and extended via [`measurements`][measurements] promises. +single host is selected, then a number of graphs for various measurements will be displayed for this host. Which exact measurements are reported depends on how [`cf-monitord`][component-cf-monitord] is configured and extended via [`measurements`][promise-type-measurements] promises. Clicking on an individual graph allows to select different time spans for which monitoring data will be displayed. @@ -26,5 +26,5 @@ Clicking on an individual graph allows to select different time spans for which If you don't see any data, make sure that: -* [`cf-monitord`][cf-monitord] is running on your hosts. -* [`cf-hub`][cf-hub] has access to collecting the monitoring data from your hosts. See [Configuring Enterprise Measurement and Monitoring Collection][Masterfiles Policy Framework#Configure Enterprise Measurement/Monitoring Collection] in the Masterfiles Policy Framework. +* [`cf-monitord`][component-cf-monitord] is running on your hosts. +* [`cf-hub`][component-cf-hub] has access to collecting the monitoring data from your hosts. See [Configuring Enterprise Measurement and Monitoring Collection][mpf-configure-measurement-collection] in the Masterfiles Policy Framework. diff --git a/enterprise-cfengine-guide/reporting.markdown b/enterprise-cfengine-guide/reporting.markdown index e5b7a20e8..632e53b9e 100644 --- a/enterprise-cfengine-guide/reporting.markdown +++ b/enterprise-cfengine-guide/reporting.markdown @@ -21,14 +21,14 @@ for a policy hub. Controlling which variables and classes should be collected by an Enterprise Hub is done primarily with a list of regular expressions matching promise meta tags for either [inclusion][access#metatags_include] or -[exclusion][access#metatags_exclude]. `cf-hub` collects `variables` that are not -prefixed with an underscore (`_`) or have meta tags matching -[`metatags_include`][access#metatags_include] and do not have any meta tags -matching [`metatags_exclude`][access#metatags_exclude] and does not have a -handle matching [`promise_handle_exclude`][access#promise_handle_exclude]. -`cf-hub` collects *namespace* scoped (global) `classes` having any meta tags -matching [`metatags_include`][access#metatags_include] that do not have any meta -tags matching [`metatags_exclude`][access#metatags_exclude]. +[exclusion][access#metatags_exclude]. `cf-hub` collects `variables` that have +meta tags matching [`metatags_include`][access#metatags_include] and do not have +any meta tags matching [`metatags_exclude`][access#metatags_exclude] and does +not have a handle matching +[`promise_handle_exclude`][access#promise_handle_exclude]. `cf-hub` collects +*namespace* scoped (global) `classes` having any meta tags matching +[`metatags_include`][access#metatags_include] that do not have any meta tags +matching [`metatags_exclude`][access#metatags_exclude]. Instead of modifying the list of regular expressions to control collection, we recommend that you leverage the defaults provided by the MPF (Masterfiles Policy diff --git a/enterprise-cfengine-guide/reporting/Report-Builder-3.18.0.png b/enterprise-cfengine-guide/reporting/Report-Builder-3.18.0.png new file mode 100644 index 000000000..db499abf0 Binary files /dev/null and b/enterprise-cfengine-guide/reporting/Report-Builder-3.18.0.png differ diff --git a/enterprise-cfengine-guide/reporting/reporting_ui.markdown b/enterprise-cfengine-guide/reporting/reporting_ui.markdown index cccb509eb..e53a121d8 100644 --- a/enterprise-cfengine-guide/reporting/reporting_ui.markdown +++ b/enterprise-cfengine-guide/reporting/reporting_ui.markdown @@ -51,6 +51,8 @@ Users not familiar with SQL syntax can easily create their own custom reports in * Limit - Limit the number of entries in your report. This is a recommended practice for testing your query, and even in production it may be helpful if you don't need to see every entry. * Show me the query - View and edit the SQL query directly. Please note, that editing the query directly here will invalidate your choices in the query builder interface, and changing your selections there will override your SQL query. +![Report Builder](Report-Builder-3.18.0.png) + ### Ensure the report collection is working ### * The reporting bundle must be called from `promises.cf`. For example, @@ -68,7 +70,7 @@ the following defines the attribute `Role` which is set to } ``` -* note the [`meta`][Promise Types and Attributes#meta] tag `inventory` +* note the [`meta`][Promise Types#meta] tag `inventory` * The hub must be able to collect the reports from the client. TCP port 5308 must be open and, because 3.6 uses TLS, should not be diff --git a/enterprise-cfengine-guide/settings-export-import-3.18.0.png b/enterprise-cfengine-guide/settings-export-import-3.18.0.png new file mode 100644 index 000000000..49bde4613 Binary files /dev/null and b/enterprise-cfengine-guide/settings-export-import-3.18.0.png differ diff --git a/enterprise-cfengine-guide/settings.markdown b/enterprise-cfengine-guide/settings.markdown index 3bd3b1fa7..73adc4a27 100644 --- a/enterprise-cfengine-guide/settings.markdown +++ b/enterprise-cfengine-guide/settings.markdown @@ -18,6 +18,8 @@ Settings view. * [Host Identifier][Settings#Host Identifier] * [Mail Settings][Settings#Mail settings] * [Authentication settings][Settings#Authentication settings] +* [Export/Import][Settings#Export/Import] +* [Role based access control][Settings#Role based access control] * [About CFEngine][Settings#About CFEngine] @@ -121,7 +123,7 @@ User three will only be able to see hosts that have reported the `ubuntu` class. * ```admin``` - The admin role can see everything and do anything. * ```cf_remoteagent``` - This role allows execution of `cf-runagent`. - + ### Default Role To set the default role, click Settings -> User management -> Roles. You can then select which role will be the default role for new users. @@ -184,7 +186,7 @@ Configure outbound mail settings: Mission portal can authenticate against an external directory. -**Special Notes:** +**Special Notes:** - LDAP API Url refers to the API cfengine uses internally for authentication. Most likely you will not alter the default value. @@ -200,27 +202,36 @@ Mission portal can authenticate against an external directory. ### LDAP groups syncing ### -- Ldap group syncing can be turned on by clicking the corresponding checkbox - - - User group attribute must be provided to obtain groups from an LDAP user entity. - The default value for Active Directory is `memberOf`. +- LDAP group syncing can be turned on by clicking the corresponding checkbox + + - User group attribute must be provided to obtain groups from an LDAP user entity. + The default value for Active Directory is `memberOf`. The group name will be taken from `cn` attribute - List of groups to sync, names must match in LDAP/MP. Each role should be added on a new line. - Click `Perform sync on every login` checkbox to synchronize user roles on every login, otherwise roles will be assigned to a user only on sign-up (first login). - -**See also:** [LDAP authentication REST API][LDAP authentication API] +**Note:** Roles *must* be created in Mission Portal. Enabling LDAP group sync +will not result in addition or removal of Mission Portal roles. + +**See also:** [LDAP authentication REST API][LDAP authentication API], [Role Management][Settings#Role Management] + +## Export/Import ## + +Mission Portal's configuration can be exported and imported. + +![Export/Import](settings-export-import-3.18.0.png) +**See also:** [Export/Import API][Import & Export API] ## Role based access control ## ![Role based access control](role_based_access_control_settings.png) -Roles in Mission portal can be restricted to perform only configured actions. +Roles in Mission portal can be restricted to perform only configured actions. Configure role-based access controls from settings. -**Special Notes:** +**Special Notes:** - Admin role has all permissions by default. @@ -228,7 +239,7 @@ Configure role-based access controls from settings. - Permissions granted by roles are additive, users with multiple roles are permitted to perform actions granted by each role. -**Restore admin role permissions:** +**Restore admin role permissions:** To restore the CFEngine admin role permissions run the following sql as root on your hub diff --git a/examples.markdown b/examples.markdown index 0cbae3ef6..7453a6a4c 100644 --- a/examples.markdown +++ b/examples.markdown @@ -70,7 +70,7 @@ Following these steps, you will login to your policy server via the SSH protocol In the policy file above, we have defined an **agent bundle** named `hello_world`. Agent bundles are only evaluated by **cf-agent**, the [agent component][cf-agent] of CFEngine. -This bundle [promises][Promise Types and Attributes] to [report][reports] on any [class of +This bundle [promises][Promise Types] to [report][reports] on any [class of hosts][Classes and Decisions]. @@ -109,7 +109,7 @@ Thus, CFEngine can function even if everything else fails ### Make the Example Stand Alone ### Instead of specifying the bundle sequence on the command line (as it was above), a [body common -control][Components and Common Control#Common Control] section can be added to +control][Components#Common Control] section can be added to the policy file. The **body common control** refers to those promises that are hard-coded into all CFEngine components and therefore affect the behavior of all components. Note that only one `body common control` is allowed per agent activation. @@ -252,7 +252,7 @@ from the example. }; ``` -4. Insert the policy file name in the [`inputs`][Components and Common Control#inputs] section of the main policy file +4. Insert the policy file name in the [`inputs`][Components#inputs] section of the main policy file `/var/cfengine/masterfiles/promises.cf`: ```cf3 diff --git a/examples/enterprise-api-examples/browsing-host-information.markdown b/examples/enterprise-api-examples/browsing-host-information.markdown index 01ab65865..9fc2ee0d0 100644 --- a/examples/enterprise-api-examples/browsing-host-information.markdown +++ b/examples/enterprise-api-examples/browsing-host-information.markdown @@ -121,6 +121,8 @@ SHA=1c8fafe478e05eec60fe08d2934415c81a51d2075aac27c9936e19012d625cb8 -X DELETE 204 No Content +**See also:** [Host REST API][Host REST API#remove host from the hub] + ## Example: Listing Available Vital Signs For A Host Each host record on the Hub has a set of vital signs collected by `cf-monitord` diff --git a/examples/enterprise-api-examples/changes-api-usage.markdown b/examples/enterprise-api-examples/changes-api-usage.markdown index ff7a45863..8aa09e955 100644 --- a/examples/enterprise-api-examples/changes-api-usage.markdown +++ b/examples/enterprise-api-examples/changes-api-usage.markdown @@ -17,7 +17,7 @@ Example is searching for changes that are performed by *linux* machines within * **Request** ``` -curl --user admin:admin 'https://test.cfengine.com/api/v2/changes/policy/count?include[]=linux&bundlename=generate_repairs' +curl --user admin:admin 'https://test.cfengine.com/api/v2/changes/policy/count?include\[\]=linux&bundlename=generate_repairs' ``` **Response** @@ -30,14 +30,14 @@ curl --user admin:admin 'https://test.cfengine.com/api/v2/changes/policy/count?i ## Example: Show vacuum command executions -Show all *vacuumdb* executions within last 24 hours executed on policy server. +Show all *vacuumdb* executions within last 24 hours executed on hosts reporting the `policy_server` or `test_cfengine_com` class. -Example is searching for changes that are performed by *policy_server* machines that execute *commands* promise with command */var/cfengine/bin/vacuumdb%* - there is '%' sign at the end which is a wildcard as vacuumdb is executed with different options across policy. +Example is searching for changes that are performed by *policy_server* machines that execute *commands* promise with command */var/cfengine/bin/vacuumdb%* - there is `%` sign at the end which is a wildcard as `vacuumdb` is executed with different options across policy. **Request** ``` -curl --user admin:admin 'https://test.cfengine.com/api/v2/changes/policy?include[]=policy_server&promisetype=commands&promiser=/var/cfengine/bin/vacuumdb%' +curl --user admin:admin 'https://test.cfengine.com/api/v2/changes/policy?include\[\]='policy_server\|test_cfengine_com'&promisetype=commands&promiser=/var/cfengine/bin/vacuumdb%' ``` **Response** diff --git a/examples/enterprise-api-examples/checking-status.markdown b/examples/enterprise-api-examples/checking-status.markdown index 1f49fa335..8a53ca790 100644 --- a/examples/enterprise-api-examples/checking-status.markdown +++ b/examples/enterprise-api-examples/checking-status.markdown @@ -12,7 +12,7 @@ diagnostics. **Request** - curl -k --user admin:admin https://test.cfengine.com/api/ + curl -k --user admin:admin --location https://test.cfengine.com/api/ **Response** diff --git a/examples/example-snippets.markdown b/examples/example-snippets.markdown index 37719a5e6..32b1bc85a 100644 --- a/examples/example-snippets.markdown +++ b/examples/example-snippets.markdown @@ -7,7 +7,7 @@ tags: [examples, policy, example snippets] --- * [General Examples][General Examples] -* [CFEngine Administration Examples][CFEngine Administration Examples] +* [Administration Examples][Administration Examples] * [Measuring Examples][Measuring Examples] * [Software Administration Examples][Software Administration Examples] * [Commands, Scripts, and Execution Examples][Commands, Scripts, and Execution Examples] diff --git a/examples/example-snippets/cfengine-administration.markdown b/examples/example-snippets/cfengine-administration.markdown index 4d39ab95c..ae940361b 100644 --- a/examples/example-snippets/cfengine-administration.markdown +++ b/examples/example-snippets/cfengine-administration.markdown @@ -1,13 +1,13 @@ --- layout: default -title: CFEngine Administration Examples +title: Administration Examples published: true sorting: 2 -tags: [Examples,CFEngine Administration] +tags: [Examples, CFEngine Administration] --- -* [Ordering promises][CFEngine Administration Examples#Ordering promises] -* [Aborting execution][CFEngine Administration Examples#Aborting execution] +* [Ordering promises][Administration Examples#Ordering promises] +* [Aborting execution][Administration Examples#Aborting execution] ## Ordering promises diff --git a/examples/example-snippets/set_up_hpc_clusters.cf b/examples/example-snippets/set_up_hpc_clusters.cf index 6b63b1d89..c4cac8ad0 100644 --- a/examples/example-snippets/set_up_hpc_clusters.cf +++ b/examples/example-snippets/set_up_hpc_clusters.cf @@ -8,7 +8,7 @@ body executor control mailmaxlines => "30"; # Once per hour, on the hour - schedule => { "Min00_05" }; + schedule => { "Min00" }; } ####################################################### diff --git a/examples/tutorials/custom_inventory.markdown b/examples/tutorials/custom_inventory.markdown index 685348951..9b18da29a 100644 --- a/examples/tutorials/custom_inventory.markdown +++ b/examples/tutorials/custom_inventory.markdown @@ -82,7 +82,7 @@ Git repository. If it is, please add the policy to your repository and ensure it gets to its final destination as needed. This policy will not be activated until it has been included in -[inputs][Components and Common Control#inputs]. For simplicity we will be +[inputs][Components#inputs]. For simplicity we will be adding it via [Augments][Augments] (`def.json`). Create `/var/cfengine/masterfiles/def.json` and populate it with the following content: diff --git a/examples/tutorials/distribute-files-from-a-central-location.markdown b/examples/tutorials/distribute-files-from-a-central-location.markdown index f68c2f84e..7f2568040 100644 --- a/examples/tutorials/distribute-files-from-a-central-location.markdown +++ b/examples/tutorials/distribute-files-from-a-central-location.markdown @@ -166,12 +166,12 @@ is involved in patch distribution, they can view that bundle for specifics. ## Integrate the policy Now that all the pieces of the policy are in place, they must be integrated -into the policy so they can be activated. Add each policy file to the [`inputs`][Components and Common Control#inputs] +into the policy so they can be activated. Add each policy file to the [`inputs`][Components#inputs] section which is found under `body common control`. Once the policy file is included in inputs, the bundle can be activated. Bundles can be activated by adding them to either the `bundlesequence` or they can be called as a `methods` type promise. -Add the following entries to `promises.cf` under `body common control` -> [`inputs`][Components and Common Control#inputs]: +Add the following entries to `promises.cf` under `body common control` -> [`inputs`][Components#inputs]: "lib/custom/files.cf", "services/patching.cf", diff --git a/examples/tutorials/installing-cfengine-enterprise-policy-server.markdown b/examples/tutorials/installing-cfengine-enterprise-policy-server.markdown deleted file mode 100644 index 1a09a161e..000000000 --- a/examples/tutorials/installing-cfengine-enterprise-policy-server.markdown +++ /dev/null @@ -1,15 +0,0 @@ ---- -layout: default -title: Installing CFEngine Enterprise Policy Server -published: true -sorting: 3 -tags: [getting started, tutorial] ---- - - - -A high level overview of CFEngine. How CFEngine increases stability and uptime while cutting costs. Definition of the most important CFEngine terms: convergence, classes, promises, promise attributes, bundles and bodies. Plus an interactive Q&A session. - -Presented by: [Aleksey Tsalolikhin](https://www.linkedin.com/in/atsaloli) of [Vertical SysAdmin](https://www.verticalsysadmin.com/). - -(Recorded April 23, 2013) diff --git a/examples/tutorials/integrating-alerts-with-pager-duty.markdown b/examples/tutorials/integrating-alerts-with-pager-duty.markdown new file mode 100644 index 000000000..3703765e7 --- /dev/null +++ b/examples/tutorials/integrating-alerts-with-pager-duty.markdown @@ -0,0 +1,131 @@ +--- +layout: default +title: Integrating alerts with PagerDuty +sorting: 15 +published: true +tags: [Examples, Tutorials, Alerts, Enterprise, Custom Actions, PagerDuty] +--- + +In this How To tutorial we will show you can integrate with [PagerDuty](http://www.pagerduty.com/) using the CFEngine notification dashboard. + +We will create a policy that ensures file integrity, and have CFEngine notify PagerDuty whenever there is a change in the file we manage. + +**System requirements:** + +* CFEngine Mission Portal +* Active PagerDuty Account + + + +## Create the file we want to manage + +Run the following command on your policy server to create the file we want to manage. + +```console +# touch /tmp/file-integrity +``` + +## Create a new policy to manage the file + +Insert the following policy into `/tmp/file_example.cf` + +```cf3 +bundle agent file_integrity +{ + files: + any:: + "/tmp/test-integrity" -> {"PCI-DSS-2", "SOX-nightmare"} + handle => "ensure-test-file-integrity", + changes => change_detection; +} + +body changes change_detection +{ + hash => "md5"; + update_hashes => "true"; + report_changes => "all"; + report_diffs => "true"; +} +``` + +## Ensure the policy always runs + +Normally, to ensure your policy file is put into action, you would need to follow these three steps: + +1. Move the policy file to your masterfiles directory (`/var/cfengine/masterfiles`): + + Normally, to ensure your policy file is put into action, you would need to follow these three steps: + + ```console + # mv /tmp/file_example.cf /var/cfengine/masterfiles/ + ``` + +2. Modify `promises.cf` to include your policy + + Unless you use version control system, or has a non-standard CFEngine setup, modify your `promises.cf` file by adding the new bundlename and policy-file so it will be picked up by CFEngine to be included in all future runs. + + ```console + # vi /var/cfengine/masterfiles/promises.cf + ``` + + a) Under the body common control, add `file_integrity` to your *bundlesequence* + + ![integrating-alerts-with-pagerduty_bundlesequence-800x357.png](integrating-alerts-with-pagerduty_bundlesequence-800x357.png) + + b) Under `body common control`, add `file_example.cf` to your inputs section. + + ![integrating-alerts-with-pagerduty_inputs-800x179.png](integrating-alerts-with-pagerduty_inputs-800x179.png) + + Now, any change you manually make to the `/tmp/file_integrity` file will be picked up by CFEngine! + + Next we need to a new service in PagerDuty which we will notify whenever a change is detected by CFEngine. + +## Create a new Service in PagerDuty + +1. Go to PagerDuty.com. In your account, under Services tab, click `Add New Service` + + ![integrating-alerts-with-pagerduty_Services_-_PagerDuty.png](integrating-alerts-with-pagerduty_Services_-_PagerDuty.png) + +2. Enter a name for the service and select an escalation policy. Select `Integrate via email.` Copy the integration email provided for use in CFEngine. + + ![integrating-alerts-with-pagerduty_CFEngine-Service-Setup-800x512.png](integrating-alerts-with-pagerduty_CFEngine-Service-Setup-800x512.png) + +3. Click `Add Service` button. Copy the integration email which we will use in CFEngine. + +## Create a new Alert in CFEngine Mission Portal + +1. Go to the the CFEngine Dashboard and click `Add` button to create a new alert. + + ![integrating-alerts-with-pagerduty_new_alert1.png](integrating-alerts-with-pagerduty_new_alert1.png) + +2. Fill out a new alert name `File integrity demo`, severity level `High` and name for the condition `File integrity demo`. + + ![integrating-alerts-with-pagerduty_new_alert_details.png](integrating-alerts-with-pagerduty_new_alert_details.png) + +3. Select `Policy` under type + + ![integrating-alerts-with-pagerduty_type_policy.png](integrating-alerts-with-pagerduty_type_policy.png) + +4. Select `Bundle`, type in the bundle name which is *file_integrity*, and finally select `Repaired` as the promise status. This means that whenever CFEngine needs to repair the bundle, it will create an alert notification. + + ![integrating-alerts-with-pagerduty_new_alert_bundle_repair.png](integrating-alerts-with-pagerduty_new_alert_bundle_repair.png) + +5. Type in the integration email defined above in the Notifications section. Press `Save` to active the alert. Choose any name you like for the New widget. In our demo we name the widget `PagerDuty`. + + **Integration complete!** + + ![integrating-alerts-with-pagerduty_notification.png](integrating-alerts-with-pagerduty_notification.png) + +## Test it! + +Now we have a made a policy to monitor the `/tmp/file-integrity` file. Whenever there is a change to this file, whether it be permissions or content, this will be detected by CFEngine which will send a notification to PagerDuty. + +1. Make a change to the `/tmp/file_integrity` file on your policy server: + + ```console + # echo "Hello World!!" > /tmp/file_integrity + ``` + + The next time CFEngine runs, it will detect the change and send an notification to PagerDuty. Go to PagerDuty and wait for an alert to be triggered. + + ![integrating-alerts-with-pagerduty_pagerduty_new_alert.png](integrating-alerts-with-pagerduty_pagerduty_new_alert.png) diff --git a/examples/tutorials/integrating-alerts-with-pagerduty_CFEngine-Service-Setup-800x512.png b/examples/tutorials/integrating-alerts-with-pagerduty_CFEngine-Service-Setup-800x512.png new file mode 100644 index 000000000..31c8004a4 Binary files /dev/null and b/examples/tutorials/integrating-alerts-with-pagerduty_CFEngine-Service-Setup-800x512.png differ diff --git a/examples/tutorials/integrating-alerts-with-pagerduty_Services_-_PagerDuty.png b/examples/tutorials/integrating-alerts-with-pagerduty_Services_-_PagerDuty.png new file mode 100644 index 000000000..61e3997c4 Binary files /dev/null and b/examples/tutorials/integrating-alerts-with-pagerduty_Services_-_PagerDuty.png differ diff --git a/examples/tutorials/integrating-alerts-with-pagerduty_bundlesequence-800x357.png b/examples/tutorials/integrating-alerts-with-pagerduty_bundlesequence-800x357.png new file mode 100644 index 000000000..572409f13 Binary files /dev/null and b/examples/tutorials/integrating-alerts-with-pagerduty_bundlesequence-800x357.png differ diff --git a/examples/tutorials/integrating-alerts-with-pagerduty_inputs-800x179.png b/examples/tutorials/integrating-alerts-with-pagerduty_inputs-800x179.png new file mode 100644 index 000000000..8af3039e3 Binary files /dev/null and b/examples/tutorials/integrating-alerts-with-pagerduty_inputs-800x179.png differ diff --git a/examples/tutorials/integrating-alerts-with-pagerduty_new_alert1.png b/examples/tutorials/integrating-alerts-with-pagerduty_new_alert1.png new file mode 100644 index 000000000..b4126836d Binary files /dev/null and b/examples/tutorials/integrating-alerts-with-pagerduty_new_alert1.png differ diff --git a/examples/tutorials/integrating-alerts-with-pagerduty_new_alert_bundle_repair.png b/examples/tutorials/integrating-alerts-with-pagerduty_new_alert_bundle_repair.png new file mode 100644 index 000000000..920a54443 Binary files /dev/null and b/examples/tutorials/integrating-alerts-with-pagerduty_new_alert_bundle_repair.png differ diff --git a/examples/tutorials/integrating-alerts-with-pagerduty_new_alert_details.png b/examples/tutorials/integrating-alerts-with-pagerduty_new_alert_details.png new file mode 100644 index 000000000..5b0183bf1 Binary files /dev/null and b/examples/tutorials/integrating-alerts-with-pagerduty_new_alert_details.png differ diff --git a/examples/tutorials/integrating-alerts-with-pagerduty_notification.png b/examples/tutorials/integrating-alerts-with-pagerduty_notification.png new file mode 100644 index 000000000..8469d1b6b Binary files /dev/null and b/examples/tutorials/integrating-alerts-with-pagerduty_notification.png differ diff --git a/examples/tutorials/integrating-alerts-with-pagerduty_pagerduty_new_alert.png b/examples/tutorials/integrating-alerts-with-pagerduty_pagerduty_new_alert.png new file mode 100644 index 000000000..f12544253 Binary files /dev/null and b/examples/tutorials/integrating-alerts-with-pagerduty_pagerduty_new_alert.png differ diff --git a/examples/tutorials/integrating-alerts-with-pagerduty_type_policy.png b/examples/tutorials/integrating-alerts-with-pagerduty_type_policy.png new file mode 100644 index 000000000..012e2d3e6 Binary files /dev/null and b/examples/tutorials/integrating-alerts-with-pagerduty_type_policy.png differ diff --git a/examples/tutorials/integrating-alerts-with-ticketing-systems.markdown b/examples/tutorials/integrating-alerts-with-ticketing-systems.markdown new file mode 100644 index 000000000..b6f206ed0 --- /dev/null +++ b/examples/tutorials/integrating-alerts-with-ticketing-systems.markdown @@ -0,0 +1,74 @@ +--- +layout: default +title: Integrating alerts with ticketing systems +sorting: 15 +published: true +tags: [Examples, Tutorials, Alerts, Enterprise, Custom Actions] +--- + +Custom actions can be used to integrate with external 3rd party systems. This tutorial shows how to use a custom action script to open a ticket in Jira when a condition is observed. + + + +## How it works + +We assume that there is already a CFEngine policy bundle in place called `web_service` that ensures an important service is working. + +As we are already using the JIRA ticketing system to get notified about issues with the infrastructure, we want CFEngine to open a ticket if our `web_service` bundle fails (is not kept). This is done centrally on our hub, because it knows the outcome of the policy on all the nodes. We will only open a ticket once the alert changes state to triggered, not while it remains in triggered, to avoid an unnecessary amount of tickets being automatically created. + +Note however that it is possible to expand on this by adjusting the Custom action script. For example, we could create reminder tickets, or even automatically close tickets when the alert clears. + +## Create a Custom action script that creates a new ticket + +1. Log in to the console of your CFEngine hub, and make sure you have python and the jira python package installed (normally by running `pip install jira`). + +2. On your workstation, unpack [cfengine\_custom\_action\_jira.py](integrating-alerts-with-ticketing-systems_cfengine_custom_action_jira.py.zip) to a working directory. + +3. Inside the script, fill in `MYJIRASERVER`, `MYUSERNAME` and `MYPASSWORD` with your information. + +4. Test the script by unpacking [alert\_parameters\_test](integrating-alerts-with-ticketing-systems_alert_parameters_test.zip) into the same directory and running `./cfengine_custom_action_jira.py alert_parameters`. + +5. Verify the previous step created a ticket in JIRA. If not, recheck the information to typed in, connectivity and any output generated when running the script. + +## Upload the Custom action script to the Mission Portal + +1. Log in to the Mission Portal of CFEngine, go to Settings (top right) followed by Custom notification scripts. + +2. Click on the button to Add a script, upload the script and fill in the information as shown in the screenshot. + + ![Upload custom action script](integrating-alerts-with-ticketing-systems_custom-action-script-upload-jira.png) + +3. Click save to allow the script to be used when creating alerts. + +## Create a new alert and associate the Custom action script + +1. Log into the Mission Portal of CFEngine, click the Dashboard tab. + +2. Click on the existing Policy compliance widget, followed by Add alert. + + ![Add alert to Policy Compliance widget](integrating-alerts-with-ticketing-systems_policy-compliance-add-alert.png) + +3. Name the alert "`Web service`" and set `Severity-level` at "`high`". + +4. Click create new condition and leave Name "`Web service`". + +5. Select Type to "`Policy`". + +6. Select Filter to "`Bundle`", and type "`web_service`". + +7. Type Promise Status to "`Not kept`". + + ![Set Type Promise Status to Not kept](integrating-alerts-with-ticketing-systems_web-service-condition.png) + +8. Associate the Custom action script we uploaded with the alert. + + ![Associate custom action script with alert](integrating-alerts-with-ticketing-systems_custom-action-alert-association-jira.png) + +## Conclusions + +In this tutorial, we have shown how easy it is to integrate with a ticketing system, with JIRA as an example, using CFEngine Custom actions scripts. + +Using this Custom action, you can choose to open JIRA tickets when some or all of your alerts are triggered. But this is just the beginning; using Custom actions, you can integrate with virtually *any* external system for notifying about- or handling triggered alerts. + +Read more in the [Custom action documentation][Custom actions for Alerts]. + diff --git a/examples/tutorials/integrating-alerts-with-ticketing-systems_alert_parameters_test.zip b/examples/tutorials/integrating-alerts-with-ticketing-systems_alert_parameters_test.zip new file mode 100644 index 000000000..b72a0d4d3 Binary files /dev/null and b/examples/tutorials/integrating-alerts-with-ticketing-systems_alert_parameters_test.zip differ diff --git a/examples/tutorials/integrating-alerts-with-ticketing-systems_cfengine_custom_action_jira.py.zip b/examples/tutorials/integrating-alerts-with-ticketing-systems_cfengine_custom_action_jira.py.zip new file mode 100644 index 000000000..df0663af1 Binary files /dev/null and b/examples/tutorials/integrating-alerts-with-ticketing-systems_cfengine_custom_action_jira.py.zip differ diff --git a/examples/tutorials/integrating-alerts-with-ticketing-systems_custom-action-alert-association-jira.png b/examples/tutorials/integrating-alerts-with-ticketing-systems_custom-action-alert-association-jira.png new file mode 100644 index 000000000..020111cc7 Binary files /dev/null and b/examples/tutorials/integrating-alerts-with-ticketing-systems_custom-action-alert-association-jira.png differ diff --git a/examples/tutorials/integrating-alerts-with-ticketing-systems_custom-action-script-upload-jira.png b/examples/tutorials/integrating-alerts-with-ticketing-systems_custom-action-script-upload-jira.png new file mode 100644 index 000000000..bdb575c4e Binary files /dev/null and b/examples/tutorials/integrating-alerts-with-ticketing-systems_custom-action-script-upload-jira.png differ diff --git a/examples/tutorials/integrating-alerts-with-ticketing-systems_policy-compliance-add-alert.png b/examples/tutorials/integrating-alerts-with-ticketing-systems_policy-compliance-add-alert.png new file mode 100644 index 000000000..0f46e0a77 Binary files /dev/null and b/examples/tutorials/integrating-alerts-with-ticketing-systems_policy-compliance-add-alert.png differ diff --git a/examples/tutorials/integrating-alerts-with-ticketing-systems_web-service-condition.png b/examples/tutorials/integrating-alerts-with-ticketing-systems_web-service-condition.png new file mode 100644 index 000000000..0d27f3656 Binary files /dev/null and b/examples/tutorials/integrating-alerts-with-ticketing-systems_web-service-condition.png differ diff --git a/examples/tutorials/integrating-with-sumo-logic.markdown b/examples/tutorials/integrating-with-sumo-logic.markdown new file mode 100644 index 000000000..937eaf6f4 --- /dev/null +++ b/examples/tutorials/integrating-with-sumo-logic.markdown @@ -0,0 +1,203 @@ +--- +layout: default +title: Integrating with Sumo Logic +sorting: 15 +published: true +tags: [Examples, Tutorials, Alerts, Enterprise, Custom Actions, Sumo Logic] +--- +In this How To we will show a simple integrate with [Sumo Logic](http://www.sumologic.com). Whenever there is a CFEngine policy update, that event will be exported to Sumo Logic. These events can become valuable traces when using Sumo Logic to analyze and detect unintendent system behavior. + +**Requirements:** + +- CFEngine Community/Enterprise +- Sumo Logic account (secret URL) + + + +# How it works + +Whenever there is a policy update or a new policy is detected by CFEngine, a special variable called "`sys.last_policy_update`" will be updated with current timestamp. + +We will store this timestamp in a file, and then via api upload the file to Sumo Logic. + +# Create the CFEngine Policy file + +In this section we will explain the most important parts of our policy file. + +First, we define a couple of variables. + +`policy_udpate_file` is the variable that contains the name of the file where we will store the timestamp and eventually upload to Sumo Logic. + +The two Sumo variables are used to access the service, while the `curl_args` is the actual curl command that will upload our timestamp file to Sumo Logic. + +```cf3 + vars: + "policy_update_file" + string => "/tmp/CFEngine_policy_updated"; + "sumo_url" + string => "https://collectors.sumologic.com/receiver/v1/http/"; + "sumo_secret" + string => "ZaVnC4dhaV1-MY_SECRET_KEY"; + "curl_args" + string => "-X POST -T $(policy_update_file) $(sumo_url)$(sumo_secret)"; +``` + +In this next section we tell CFEngine to ensure that the `/tmp/CFEngine_policy_updated` file, as defined by the variable `policy_update_file` always exists. + +We also ensures that the content of this file will be the value of the `sys.last_policy_update` variable which we now know is the timestamp. We further set a class called `new_policy_update` every time there is a change in the file. This class later becomes the trigger point for when to upload the file to Sumo Logic. + +Finally, below you will see a body defining how CFEngine is going to detect changes in policy files, this time using an md5 hash and only looking for change in the content (not permissions or ownership). + +```cf3 +files: + "$(policy_update_file)" + create => "true", + edit_line => insert("CFEngine_update: $(sys.last_policy_update)"), + edit_defaults => file; + + "$(policy_update_file)" + classes => if_repaired("new_policy_update"), + changes => change_detections; + +body changes change_detections +{ + hash => "md5"; + update_hashes => "true"; + report_changes => "content"; + report_diffs => "true"; +} +``` + +The final section in the CFEngine policy is where the command that uploads the file with the timestamp to Sumo Logic is defined. + +The command will only be issued whenever a class called `new_policy_update` is set, which we above defined to be set when there is a change detection. The handle argument is a useful way to document your intentions. + +```cf3 +commands: + new_policy_update:: + "/usr/bin/curl" + args => "$(curl_args)", + classes => if_repaired("new_policy_update_sent_to_sumo_logic"), + contain => shell_command, + handle => "New sumo logic event created"; +``` + +That's it! You can copy and paste the whole policy file at the bottom of this page. + +Save the policy file you make as `/tmp/sumologic_policy_update.cf` + +# Ensure the policy always runs + +Normally, to ensure your policy file is put into action, you would need to follow these two steps: + +1. Move the policy file to your masterfiles directory: + +```console +# mv /tmp/sumo.cf /var/cfengine/masterfiles/ +``` + +2. Modify `promises.cf` to include your policy + + Unless you use version control system, or has a non-standard CFEngine setup, modify your `promises.cf` file by adding the new bundle name and policy-file so it will be picked up by CFEngine and be part of all it future runs. + +```console +# vi /var/cfengine/masterfiles/promises.cf +``` + +Under the body common control, add `sumo_logic_policy_update` to your bundle sequence. + +![integrating-with-sumo-logic_bundle_sequence.png](integrating-with-sumo-logic_bundle_sequence.png) + +Under body common control, add /sumologic\_policy\_update.cf/ to your inputs section. + +![integrating-with-sumo-logic_inputs1.png](integrating-with-sumo-logic_inputs1.png) + +That's all. + + +# Test it! + +To test it, we need to make a change to any CFEngine policy, and then go to Sumo Logic to see if there is a new timestamp reported. + +* Make a change to any policy file, for examle `promises.cf`: + +```console +# vi /var/cfengine/masterfiles/promises.cf   +``` + +Add a comment and close the file. + +* Check if timestamp has been updated + +```console +# cat /tmp/CFEngine_policy_updated   +``` + +* Check with Sumo Logic + +![integrating-with-sumo-logic_sumo.png](integrating-with-sumo-logic_sumo.png) + +**Mission Accomplished!** + +As we can see above CFEngine detected a change on `Thursday Oct 2 at 01:16:42` and also at `01:13:45`. + +**Source-code:** + +The policy as found in `sumologic_policy_update.cf`. + + bundle agent sumo_logic_policy_update + { + vars: + "policy_update_file" + string => "/tmp/CFEngine_policy_updated"; + "sumo_url" + string => "https://collectors.sumologic.com/receiver/v1/http/"; + "sumo_secret" + string => "MY_SECRET_KEY"; + "curl_args" + string => "-X POST -T $(policy_update_file) $(sumo_url)$(sumo_secret)"; + + files: + "$(policy_update_file)" + create => "true", + edit_line => insert("CFEngine_update: $(sys.last_policy_update)"), + edit_defaults => file; + + "$(policy_update_file)" + classes => if_repaired("new_policy_update"), + changes => change_detections; + + commands: + new_policy_update:: + "/usr/bin/curl" + args => "$(curl_args)", + classes => if_repaired("new_policy_update_sent_to_sumo_logic"), + contain => shell_command, + handle => "New sumo logic event created"; + } + + body changes change_detections + { + hash => "md5"; + update_hashes => "true"; + report_changes => "content"; + report_diffs => "true"; + } + + body contain shell_command + { + useshell => "useshell"; + } + + bundle edit_line insert(str) + { + insert_lines: + "$(str)"; + } + + body edit_defaults file + { + empty_file_before_editing => "true"; + } + + diff --git a/examples/tutorials/integrating-with-sumo-logic_bundle_sequence.png b/examples/tutorials/integrating-with-sumo-logic_bundle_sequence.png new file mode 100644 index 000000000..6c1510d5c Binary files /dev/null and b/examples/tutorials/integrating-with-sumo-logic_bundle_sequence.png differ diff --git a/examples/tutorials/integrating-with-sumo-logic_inputs1.png b/examples/tutorials/integrating-with-sumo-logic_inputs1.png new file mode 100644 index 000000000..0b350c535 Binary files /dev/null and b/examples/tutorials/integrating-with-sumo-logic_inputs1.png differ diff --git a/examples/tutorials/integrating-with-sumo-logic_sumo.png b/examples/tutorials/integrating-with-sumo-logic_sumo.png new file mode 100644 index 000000000..9b09d6332 --- /dev/null +++ b/examples/tutorials/integrating-with-sumo-logic_sumo.png @@ -0,0 +1,7 @@ + +301 Moved Permanently + +

301 Moved Permanently

+
nginx
+ + diff --git a/examples/tutorials/line_editing.markdown b/examples/tutorials/line_editing.markdown deleted file mode 100644 index 337a650e1..000000000 --- a/examples/tutorials/line_editing.markdown +++ /dev/null @@ -1,7 +0,0 @@ ---- -layout: default -title: Line Editing -published: false -sorting: 3 -tags: [tutorial, json] ---- \ No newline at end of file diff --git a/examples/tutorials/manage-ntp.markdown b/examples/tutorials/manage-ntp.markdown index 229248be9..72d6eee49 100644 --- a/examples/tutorials/manage-ntp.markdown +++ b/examples/tutorials/manage-ntp.markdown @@ -447,7 +447,7 @@ The classes attribute here uses the [`results()`][lib/common.cf#results] classes template_method => "inline_mustache", ``` -CFEngine supports multiple templating engines, the `template_method` attribute specifies how the promised file content will be resolved. The value `inline_mustache` indicates that we will use the mustache templating engine and specify the template in-line, instead of in an external file. +CFEngine supports multiple templating engines, the [template_method][files#template_method] attribute specifies how the promised file content will be resolved. The value `inline_mustache` indicates that we will use the mustache templating engine and specify the template in-line, instead of in an external file. ##### edit_template_string diff --git a/examples/tutorials/manage-packages.markdown b/examples/tutorials/manage-packages.markdown index 8c78cfe8f..173b53cb8 100644 --- a/examples/tutorials/manage-packages.markdown +++ b/examples/tutorials/manage-packages.markdown @@ -6,8 +6,6 @@ sorting: 3 tags: [getting started, tutorial] --- - - Package management is a critical task for any system administrator. In this tutorial we will show you how easy it is to install, manage and remove packages using CFEngine. diff --git a/examples/tutorials/masterfiles_policy_framework_upgrade.markdown b/examples/tutorials/masterfiles_policy_framework_upgrade.markdown index 5af185752..4542de7be 100644 --- a/examples/tutorials/masterfiles_policy_framework_upgrade.markdown +++ b/examples/tutorials/masterfiles_policy_framework_upgrade.markdown @@ -6,163 +6,151 @@ sorting: 14 tags: [MPF, upgrade, masterfiles, tutorial] --- -# Introduction +Upgrading the Masterfiles Policy Framework (MPF) is the first step in upgrading CFEngine from one version to another. The MPF should always be the same version or newer than the binary versions running. + +Upgrading the MPF is not an exact process as the details highly depend on the specifics of the changes made to the default policy. This tutorial leverages `git` and shows an example of upgrading a simple policy set based on `3.18.0` to `3.18.5` and can be used as a reference for upgrading your own policy sets. + -Upgrading the Masterfiles Policy Framework (MPF) is an *optional* but **highly -recommended** first step when upgrading CFEngine. -Upgrading the MPF is not an exact process as the details highly depend on the -specifics of the changes made to the default policy. This tutorial leverages -`git` and shows an example of upgrading a simple policy set based on 3.6.7 to -3.7.4 and can be used as a reference for upgrading your own policy sets. # Prepare a Git clone of your working masterfiles -If you are not using Git and instead editing directly in -`$(sys.workdir/masterfiles)` you can simply copy your masterfiles into a new -directory and initalize a new Git repository. - -If you're using Git already simply clone your repository and skip to the next -step. - -```console -[root@hub MPF_upgrade]# rsync -a /var/cfengine/masterfiles/ MPF_upgrade/ - -``` - -Then initialize the new Git repository and add all the files to it. - -```console -[root@hub ~]# cd MPF_upgrade/ -[root@hub MPF_upgrade]# git init -Initialized empty Git repository in /root/MPF_upgrade/.git/ -[root@hub MPF_upgrade]# git add -A -[root@hub MPF_upgrade]# git commit -m "Before Upgrade" -[master (root-commit) 108c210] Before Upgrade - 78 files changed, 19980 insertions(+) - create mode 100644 CUSTOM/policy1.cf - create mode 100644 cf_promises_release_id - create mode 100644 cf_promises_validated - create mode 100644 cfe_internal/CFE_cfengine.cf - create mode 100644 cfe_internal/CFE_hub_specific.cf - create mode 100644 cfe_internal/CFE_knowledge.cf - create mode 100644 cfe_internal/cfengine_processes.cf - create mode 100644 cfe_internal/ha/ha.cf - create mode 100644 cfe_internal/ha/ha_def.cf - create mode 100644 cfe_internal/host_info_report.cf - create mode 100644 controls/3.4/cf_serverd.cf - create mode 100644 controls/cf_agent.cf - create mode 100644 controls/cf_execd.cf - create mode 100644 controls/cf_hub.cf - create mode 100644 controls/cf_monitord.cf - create mode 100644 controls/cf_runagent.cf - create mode 100644 controls/cf_serverd.cf - create mode 100644 def.cf - create mode 100644 inventory/any.cf - create mode 100644 inventory/debian.cf - create mode 100644 inventory/generic.cf - create mode 100644 inventory/linux.cf - create mode 100644 inventory/lsb.cf - create mode 100644 inventory/macos.cf - create mode 100644 inventory/os.cf - create mode 100644 inventory/redhat.cf - create mode 100644 inventory/suse.cf - create mode 100644 inventory/windows.cf - create mode 100644 lib/3.5/bundles.cf - create mode 100644 lib/3.5/cfe_internal.cf - create mode 100644 lib/3.5/commands.cf - create mode 100644 lib/3.5/common.cf - create mode 100644 lib/3.5/databases.cf - create mode 100644 lib/3.5/feature.cf - create mode 100644 lib/3.5/files.cf - create mode 100644 lib/3.5/guest_environments.cf - create mode 100644 lib/3.5/monitor.cf - create mode 100644 lib/3.5/packages.cf - create mode 100644 lib/3.5/paths.cf - create mode 100644 lib/3.5/processes.cf - create mode 100644 lib/3.5/reports.cf - create mode 100644 lib/3.5/services.cf - create mode 100644 lib/3.5/storage.cf - create mode 100644 lib/3.6/bundles.cf - create mode 100644 lib/3.6/cfe_internal.cf - create mode 100644 lib/3.6/cfengine_enterprise_hub_ha.cf - create mode 100644 lib/3.6/commands.cf - create mode 100644 lib/3.6/common.cf - create mode 100644 lib/3.6/databases.cf - create mode 100644 lib/3.6/edit_xml.cf - create mode 100644 lib/3.6/examples.cf - create mode 100644 lib/3.6/feature.cf - create mode 100644 lib/3.6/files.cf - create mode 100644 lib/3.6/guest_environments.cf - create mode 100644 lib/3.6/monitor.cf - create mode 100644 lib/3.6/packages.cf - create mode 100644 lib/3.6/paths.cf - create mode 100644 lib/3.6/processes.cf - create mode 100644 lib/3.6/reports.cf - create mode 100644 lib/3.6/services.cf - create mode 100644 lib/3.6/stdlib.cf - create mode 100644 lib/3.6/storage.cf - create mode 100644 lib/3.6/users.cf - create mode 100644 lib/3.6/vcs.cf - create mode 100644 promises.cf - create mode 100644 services/autorun.cf - create mode 100644 services/autorun/custom_policy2.cf - create mode 100644 services/autorun/hello.cf - create mode 100644 services/file_change.cf - create mode 100644 sketches/meta/api-runfile.cf - create mode 100644 templates/host_info_report.mustache - create mode 100644 update.cf - create mode 100644 update/cfe_internal_dc_workflow.cf - create mode 100644 update/cfe_internal_local_git_remote.cf - create mode 100644 update/cfe_internal_update_from_repository.cf - create mode 100644 update/update_bins.cf - create mode 100644 update/update_policy.cf - create mode 100644 update/update_processes.cf -[root@hub MPF_upgrade]# git status -# On branch master -nothing to commit, working directory clean -``` - -Now we have a Git repository that we can start merging in the changes from -upstream. - -# Merge the upstream changes to the MPF into your policy - -## Remove everything except the .git directory. - -By first removing everything we will easily be able so see which files are -*new*, *changed*, *moved* or *removed* upstream. - -```console -[root@hub MPF_upgrade]# rm -rf * - - -[root@hub MPF_upgrade]# git status +We will perform the integration work in `/tmp/MPF-upgrade/integration`. `masterfiles` should exist in the integration directory and is expected to be both the root of your policy set and a `git` repository. + + + + +## Validating expectations + +From `/tmp/MPF-upgrade/integration/masterfiles`. Let's inspect what we expect. + +Is it the root of a policy set? `promises.cf` will be present if so. + +```bash + export INTEGRATION_ROOT="/tmp/MPF-upgrade/integration" + cd $INTEGRATION_ROOT/masterfiles +if [ -e "promises.cf" ]; then + echo "promise.cf exists, it's likely the root of a policy set" +else + echo "promises.cf is missing, $INTEGRATION_ROOT/masterfiles does not seem like the root of a policy set" +fi +``` + +Output: + +``` +promise.cf exists, it's likely the root of a policy set +``` + +Let's see what version of the MPF we are starting from by looking at `version` in `body common control` of `promises.cf`. + +```bash +grep -P "\s+version\s+=>" $INTEGRATION_ROOT/masterfiles/promises.cf 2>&1 \ + || echo "promises.cf is missing, $INTEGRATION_ROOT/masterfiles does not seem to be the root of a policy set" +``` + +Output: + +``` +version => "CFEngine Promises.cf 3.18.0"; +``` + +And finally, is it a git repository, what is the last commit? + +```bash +git status \ + || echo "$INTEGRATION_ROOT/masterfiles does not appear to be a git repository!" \ + && git log -1 +``` + +Output: + +``` +On branch master +nothing to commit, working tree clean +commit a1d0b726bb8894848bc877d037c546062228881f +Author: Nick Anderson +Date: Wed Jul 26 17:37:31 2023 -0500 + + CFEngine Policy set prior to upgrade +``` + + + + +# Merge upstream changes from the MPF into your policy + + + + +## Remove everything except the `.git` directory + +By first removing everything we will easily be able so see which files are **new**, **changed**, **moved** or **removed** upstream. + +```bash +rm -rf * +``` + +Check `git status` to see that all the files have been deleted and are not staged for commit. + +```bash +git status +``` + +Output: + +``` On branch master Changes not staged for commit: (use "git add/rm ..." to update what will be committed) - (use "git checkout -- ..." to discard changes in working directory) - - deleted: CUSTOM/policy1.cf - deleted: cf_promises_release_id - deleted: cf_promises_validated + (use "git restore ..." to discard changes in working directory) deleted: cfe_internal/CFE_cfengine.cf - deleted: cfe_internal/CFE_hub_specific.cf - deleted: cfe_internal/CFE_knowledge.cf - deleted: cfe_internal/cfengine_processes.cf - deleted: cfe_internal/ha/ha.cf - deleted: cfe_internal/ha/ha_def.cf - deleted: cfe_internal/host_info_report.cf - deleted: controls/3.4/cf_serverd.cf + deleted: cfe_internal/core/deprecated/cfengine_processes.cf + deleted: cfe_internal/core/host_info_report.cf + deleted: cfe_internal/core/limit_robot_agents.cf + deleted: cfe_internal/core/log_rotation.cf + deleted: cfe_internal/core/main.cf + deleted: cfe_internal/core/watchdog/templates/watchdog-windows.ps1.mustache + deleted: cfe_internal/core/watchdog/templates/watchdog.mustache + deleted: cfe_internal/core/watchdog/watchdog.cf + deleted: cfe_internal/enterprise/CFE_hub_specific.cf + deleted: cfe_internal/enterprise/CFE_knowledge.cf + deleted: cfe_internal/enterprise/federation/federation.cf + deleted: cfe_internal/enterprise/file_change.cf + deleted: cfe_internal/enterprise/ha/ha.cf + deleted: cfe_internal/enterprise/ha/ha_def.cf + deleted: cfe_internal/enterprise/ha/ha_update.cf + deleted: cfe_internal/enterprise/main.cf + deleted: cfe_internal/enterprise/mission_portal.cf + deleted: cfe_internal/enterprise/templates/httpd.conf.mustache + deleted: cfe_internal/enterprise/templates/runalerts.php.mustache + deleted: cfe_internal/enterprise/templates/runalerts.sh.mustache + deleted: cfe_internal/recommendations.cf + deleted: cfe_internal/update/cfe_internal_dc_workflow.cf + deleted: cfe_internal/update/cfe_internal_update_from_repository.cf + deleted: cfe_internal/update/lib.cf + deleted: cfe_internal/update/systemd_units.cf + deleted: cfe_internal/update/update_bins.cf + deleted: cfe_internal/update/update_policy.cf + deleted: cfe_internal/update/update_processes.cf + deleted: cfe_internal/update/windows_unattended_upgrade.cf deleted: controls/cf_agent.cf deleted: controls/cf_execd.cf deleted: controls/cf_hub.cf deleted: controls/cf_monitord.cf deleted: controls/cf_runagent.cf deleted: controls/cf_serverd.cf - deleted: def.cf + deleted: controls/def.cf + deleted: controls/def_inputs.cf + deleted: controls/reports.cf + deleted: controls/update_def.cf + deleted: controls/update_def_inputs.cf + deleted: custom-2.cf + deleted: def.json + deleted: inventory/aix.cf deleted: inventory/any.cf deleted: inventory/debian.cf + deleted: inventory/freebsd.cf deleted: inventory/generic.cf deleted: inventory/linux.cf deleted: inventory/lsb.cf @@ -171,724 +159,875 @@ Changes not staged for commit: deleted: inventory/redhat.cf deleted: inventory/suse.cf deleted: inventory/windows.cf - deleted: lib/3.5/bundles.cf - deleted: lib/3.5/cfe_internal.cf - deleted: lib/3.5/commands.cf - deleted: lib/3.5/common.cf - deleted: lib/3.5/databases.cf - deleted: lib/3.5/feature.cf - deleted: lib/3.5/files.cf - deleted: lib/3.5/guest_environments.cf - deleted: lib/3.5/monitor.cf - deleted: lib/3.5/packages.cf - deleted: lib/3.5/paths.cf - deleted: lib/3.5/processes.cf - deleted: lib/3.5/reports.cf - deleted: lib/3.5/services.cf - deleted: lib/3.5/storage.cf - deleted: lib/3.6/bundles.cf - deleted: lib/3.6/cfe_internal.cf - deleted: lib/3.6/cfengine_enterprise_hub_ha.cf - deleted: lib/3.6/commands.cf - deleted: lib/3.6/common.cf - deleted: lib/3.6/databases.cf - deleted: lib/3.6/edit_xml.cf - deleted: lib/3.6/examples.cf - deleted: lib/3.6/feature.cf - deleted: lib/3.6/files.cf - deleted: lib/3.6/guest_environments.cf - deleted: lib/3.6/monitor.cf - deleted: lib/3.6/packages.cf - deleted: lib/3.6/paths.cf - deleted: lib/3.6/processes.cf - deleted: lib/3.6/reports.cf - deleted: lib/3.6/services.cf - deleted: lib/3.6/stdlib.cf - deleted: lib/3.6/storage.cf - deleted: lib/3.6/users.cf - deleted: lib/3.6/vcs.cf + deleted: lib/autorun.cf + deleted: lib/bundles.cf + deleted: lib/cfe_internal.cf + deleted: lib/cfe_internal_hub.cf + deleted: lib/cfengine_enterprise_hub_ha.cf + deleted: lib/commands.cf + deleted: lib/common.cf + deleted: lib/databases.cf + deleted: lib/deprecated-upstream.cf + deleted: lib/edit_xml.cf + deleted: lib/event.cf + deleted: lib/examples.cf + deleted: lib/feature.cf + deleted: lib/files.cf + deleted: lib/guest_environments.cf + deleted: lib/monitor.cf + deleted: lib/packages-ENT-3719.cf + deleted: lib/packages.cf + deleted: lib/paths.cf + deleted: lib/processes.cf + deleted: lib/reports.cf + deleted: lib/services.cf + deleted: lib/stdlib.cf + deleted: lib/storage.cf + deleted: lib/testing.cf + deleted: lib/users.cf + deleted: lib/vcs.cf + deleted: modules/packages/vendored/WiRunSQL.vbs.mustache + deleted: modules/packages/vendored/apk.mustache + deleted: modules/packages/vendored/apt_get.mustache + deleted: modules/packages/vendored/freebsd_ports.mustache + deleted: modules/packages/vendored/msiexec-list.vbs.mustache + deleted: modules/packages/vendored/msiexec.bat.mustache + deleted: modules/packages/vendored/nimclient.mustache + deleted: modules/packages/vendored/pkg.mustache + deleted: modules/packages/vendored/pkgsrc.mustache + deleted: modules/packages/vendored/slackpkg.mustache + deleted: modules/packages/vendored/snap.mustache + deleted: modules/packages/vendored/yum.mustache + deleted: modules/packages/vendored/zypper.mustache deleted: promises.cf - deleted: services/autorun.cf - deleted: services/autorun/custom_policy2.cf + deleted: services/autorun/custom-1.cf deleted: services/autorun/hello.cf - deleted: services/file_change.cf - deleted: sketches/meta/api-runfile.cf + deleted: services/custom-3.cf + deleted: services/init.cf + deleted: services/main.cf + deleted: standalone_self_upgrade.cf + deleted: templates/cf-apache.service.mustache + deleted: templates/cf-execd.service.mustache + deleted: templates/cf-hub.service.mustache + deleted: templates/cf-monitord.service.mustache + deleted: templates/cf-postgres.service.mustache + deleted: templates/cf-runalerts.service.mustache + deleted: templates/cf-serverd.service.mustache + deleted: templates/cfengine3.service.mustache + deleted: templates/cfengine_watchdog.mustache + deleted: templates/federated_reporting/10-base_filter.sed + deleted: templates/federated_reporting/50-merge_inserts.awk + deleted: templates/federated_reporting/config.sh.mustache + deleted: templates/federated_reporting/dump.sh + deleted: templates/federated_reporting/import.sh + deleted: templates/federated_reporting/import_file.sh + deleted: templates/federated_reporting/log.sh.mustache + deleted: templates/federated_reporting/parallel.sh + deleted: templates/federated_reporting/psql_wrapper.sh.mustache + deleted: templates/federated_reporting/pull_dumps_from.sh + deleted: templates/federated_reporting/transport.sh deleted: templates/host_info_report.mustache + deleted: templates/json_multiline.mustache + deleted: templates/json_serial.mustache + deleted: templates/vercmp.ps1 deleted: update.cf - deleted: update/cfe_internal_dc_workflow.cf - deleted: update/cfe_internal_local_git_remote.cf - deleted: update/cfe_internal_update_from_repository.cf - deleted: update/update_bins.cf - deleted: update/update_policy.cf - deleted: update/update_processes.cf no changes added to commit (use "git add" and/or "git commit -a") ``` -## Install the new MPF -The MPF can be obtained from -any [community package](https://cfengine.com/product/community/) (in -```$(sys.workdir)/share/CoreBase/```), -[enterprise hub package](https://cfengine.com/product/free-download/) (in -```$(sys.workdir)/share/NovaBase/```), -[masterfiles source tarball](https://cfengine-package-repos.s3.amazonaws.com/tarballs/cfengine-masterfiles-{{site.cfengine.branch}}.{{site.cfengine.latest_patch_release}}.tar.gz) (requires -```./configure``` and ```make install``` -), -[installed masterfiles tarball](https://cfengine-package-repos.s3.amazonaws.com/tarballs/cfengine-masterfiles-{{site.cfengine.branch}}.{{site.cfengine.latest_patch_release}}.pkg.tar.gz) (ready -for extraction), -or -[directly from github](https://github.com/cfengine/masterfiles/tree/{{site.cfengine.masterfiles_branch}}). -We will install the MPF from source obtained -directly from github. -**Note:** You will need ```automake``` to install from source. +## Install the new version of the MPF + + -First clone the masterfiles repository for the version you are installing. And -verify you have the correct tag checked out. -**Note:** Directly checking out a tag as in the example below is only -supported in Git versions 1.7.9.5 and newer. +### Installing from Git -```console -[root@hub MPF_upgrade]# cd .. -[root@hub ~]# git clone -b 3.7.4 https://github.com/cfengine/masterfiles -[root@hub ~]# cd masterfiles -[root@hub ~]# git describe -3.7.4 +First, clone the desired version of the MPF source. + +```bash +export MPF_VERSION="3.18.5" +git clone -b $MPF_VERSION https://github.com/cfengine/masterfiles $INTEGRATION_ROOT/masterfiles-source-$MPF_VERSION ``` -**Note:** For systems without python 3 easily available (such as centos 6) you can use the following to get around problems in autogen with 3rdparty/core/determine-version.py, which requires python3. +Output: ``` -export EXPLICIT_VERSION=$(git describe) +Cloning into '/tmp/MPF-upgrade/integration/masterfiles-source-3.18.5'... +Note: switching to 'ea81b7b790f1bf98cbf145d8700a06b37d74fb84'. + +You are in 'detached HEAD' state. You can look around, make experimental +changes and commit them, and you can discard any commits you make in this +state without impacting any branches by switching back to a branch. + +If you want to create a new branch to retain commits you create, you may +do so (now or later) by using -c with the switch command. Example: + + git switch -c + +Or undo this operation with: + + git switch - + +Turn off this advice by setting config variable advice.detachedHead to false ``` -Now we will install the masterfiles from upstream into the directory where we -are doing the integration. +Then build and install targeting the integration root directory. When installed from source masterfiles installs into the `masterfiles` directory. -First we build and install masterfiles to a temporary location. +```bash +cd $INTEGRATION_ROOT/masterfiles-source-$MPF_VERSION +export EXPLICIT_VERSION=$MPF_VERSION -```console ./autogen.sh -[root@hub masterfiles]# ./autogen.sh -configure.ac:31: installing `./config.guess' -configure.ac:31: installing `./config.sub' -configure.ac:34: installing `./install-sh' -configure.ac:34: installing `./missing' -checking build system type... x86_64-unknown-linux-gnu -checking host system type... x86_64-unknown-linux-gnu -checking target system type... x86_64-unknown-linux-gnu +make +make install prefix=$INTEGRATION_ROOT/ +``` + +Output: + +``` +./autogen.sh: Running determine-version.sh ... +./autogen.sh: Running determine-release.sh ... +All tags pointing to current commit: +3.18.5 +3.18.5-build5 +Latest version: 3.18.5 +Could not parse it, using default release number 1 +./autogen.sh: Running autoreconf ... +configure.ac:40: installing './config.guess' +configure.ac:40: installing './config.sub' +configure.ac:43: installing './install-sh' +configure.ac:43: installing './missing' +parallel-tests: installing './test-driver' +/tmp/MPF-upgrade/integration/masterfiles-source-3.18.5 +checking build system type... x86_64-pc-linux-gnu +checking host system type... x86_64-pc-linux-gnu +checking target system type... x86_64-pc-linux-gnu checking for a BSD-compatible install... /usr/bin/install -c checking whether build environment is sane... yes -checking for a thread-safe mkdir -p... /bin/mkdir -p +checking for a race-free mkdir -p... /usr/bin/mkdir -p checking for gawk... gawk checking whether make sets $(MAKE)... yes +checking whether make supports nested variables... yes +checking whether UID '1000' is supported by ustar format... yes +checking whether GID '1000' is supported by ustar format... yes checking how to create a ustar tar archive... gnutar -checking whether to disable maintainer-specific portions of Makefiles... yes -checking for a thread-safe mkdir -p... /bin/mkdir -p -checking for a BSD-compatible install... /usr/bin/install -c - -Summary of options: +checking if GNU tar supports --hard-dereference... yes +checking whether to enable maintainer-specific portions of Makefiles... yes +checking whether make supports nested variables... (cached) yes +checking for pkg_install... no +checking for shunit2... no + +Summary: +Version -> 3.18.5 +Release -> 1 Core directory -> not set - tests are disabled Enterprise directory -> not set - some tests are disabled Install prefix -> /var/cfengine +bindir -> /var/cfengine/bin configure: generating makefile targets +checking that generated files are newer than configure... done configure: creating ./config.status config.status: creating Makefile -config.status: creating controls/3.5/update_def.cf -config.status: creating controls/3.6/update_def.cf -config.status: creating controls/3.7/update_def.cf -config.status: creating modules/packages/Makefile +config.status: creating controls/update_def.cf config.status: creating promises.cf +config.status: creating standalone_self_upgrade.cf +config.status: creating tests/Makefile config.status: creating tests/acceptance/Makefile config.status: creating tests/unit/Makefile DONE: Configuration done. Run "make install" to install CFEngine Masterfiles. -[root@hub masterfiles]# ./configure --prefix /tmp/masterfiles-3.7.4 -checking build system type... x86_64-unknown-linux-gnu -checking host system type... x86_64-unknown-linux-gnu -checking target system type... x86_64-unknown-linux-gnu -checking for a BSD-compatible install... /usr/bin/install -c -checking whether build environment is sane... yes -checking for a thread-safe mkdir -p... /bin/mkdir -p -checking for gawk... gawk -checking whether make sets $(MAKE)... yes -checking how to create a ustar tar archive... gnutar -checking whether to disable maintainer-specific portions of Makefiles... yes -checking for a thread-safe mkdir -p... /bin/mkdir -p -checking for a BSD-compatible install... /usr/bin/install -c -Summary of options: -Core directory -> not set - tests are disabled -Enterprise directory -> not set - some tests are disabled -Install prefix -> /tmp/masterfiles-3.7.4 +Making all in tests/ +make[1]: Entering directory '/tmp/MPF-upgrade/integration/masterfiles-source-3.18.5/tests' +Making all in . +make[2]: Entering directory '/tmp/MPF-upgrade/integration/masterfiles-source-3.18.5/tests' +make[2]: Nothing to be done for 'all-am'. +make[2]: Leaving directory '/tmp/MPF-upgrade/integration/masterfiles-source-3.18.5/tests' +Making all in unit +make[2]: Entering directory '/tmp/MPF-upgrade/integration/masterfiles-source-3.18.5/tests/unit' +make[2]: Nothing to be done for 'all'. +make[2]: Leaving directory '/tmp/MPF-upgrade/integration/masterfiles-source-3.18.5/tests/unit' +make[1]: Leaving directory '/tmp/MPF-upgrade/integration/masterfiles-source-3.18.5/tests' +make[1]: Entering directory '/tmp/MPF-upgrade/integration/masterfiles-source-3.18.5' +make[1]: Nothing to be done for 'all-am'. +make[1]: Leaving directory '/tmp/MPF-upgrade/integration/masterfiles-source-3.18.5' +Making install in tests/ +make[1]: Entering directory '/tmp/MPF-upgrade/integration/masterfiles-source-3.18.5/tests' +Making install in . +make[2]: Entering directory '/tmp/MPF-upgrade/integration/masterfiles-source-3.18.5/tests' +make[3]: Entering directory '/tmp/MPF-upgrade/integration/masterfiles-source-3.18.5/tests' +make[3]: Nothing to be done for 'install-exec-am'. +make[3]: Nothing to be done for 'install-data-am'. +make[3]: Leaving directory '/tmp/MPF-upgrade/integration/masterfiles-source-3.18.5/tests' +make[2]: Leaving directory '/tmp/MPF-upgrade/integration/masterfiles-source-3.18.5/tests' +Making install in unit +make[2]: Entering directory '/tmp/MPF-upgrade/integration/masterfiles-source-3.18.5/tests/unit' +make[3]: Entering directory '/tmp/MPF-upgrade/integration/masterfiles-source-3.18.5/tests/unit' +make[3]: Nothing to be done for 'install-exec-am'. +make[3]: Nothing to be done for 'install-data-am'. +make[3]: Leaving directory '/tmp/MPF-upgrade/integration/masterfiles-source-3.18.5/tests/unit' +make[2]: Leaving directory '/tmp/MPF-upgrade/integration/masterfiles-source-3.18.5/tests/unit' +make[1]: Leaving directory '/tmp/MPF-upgrade/integration/masterfiles-source-3.18.5/tests' +make[1]: Entering directory '/tmp/MPF-upgrade/integration/masterfiles-source-3.18.5' +make[2]: Entering directory '/tmp/MPF-upgrade/integration/masterfiles-source-3.18.5' +make[2]: Nothing to be done for 'install-exec-am'. + /usr/bin/mkdir -p '/tmp/MPF-upgrade/integration//masterfiles' + /usr/bin/mkdir -p '/tmp/MPF-upgrade/integration//masterfiles/cfe_internal/core' + /usr/bin/install -c -m 644 ./cfe_internal/core/host_info_report.cf ./cfe_internal/core/log_rotation.cf ./cfe_internal/core/main.cf ./cfe_internal/core/limit_robot_agents.cf '/tmp/MPF-upgrade/integration//masterfiles/cfe_internal/core' + /usr/bin/mkdir -p '/tmp/MPF-upgrade/integration//masterfiles/cfe_internal/enterprise/templates' + /usr/bin/install -c -m 644 ./cfe_internal/enterprise/templates/runalerts.sh.mustache ./cfe_internal/enterprise/templates/httpd.conf.mustache ./cfe_internal/enterprise/templates/apachectl.mustache ./cfe_internal/enterprise/templates/runalerts.php.mustache '/tmp/MPF-upgrade/integration//masterfiles/cfe_internal/enterprise/templates' + /usr/bin/mkdir -p '/tmp/MPF-upgrade/integration//masterfiles/inventory' + /usr/bin/install -c -m 644 ./inventory/windows.cf ./inventory/suse.cf ./inventory/macos.cf ./inventory/lsb.cf ./inventory/any.cf ./inventory/os.cf ./inventory/freebsd.cf ./inventory/generic.cf ./inventory/debian.cf ./inventory/linux.cf ./inventory/redhat.cf ./inventory/aix.cf '/tmp/MPF-upgrade/integration//masterfiles/inventory' + /usr/bin/mkdir -p '/tmp/MPF-upgrade/integration//masterfiles/cfe_internal/enterprise/federation' + /usr/bin/install -c -m 644 ./cfe_internal/enterprise/federation/federation.cf '/tmp/MPF-upgrade/integration//masterfiles/cfe_internal/enterprise/federation' + /usr/bin/mkdir -p '/tmp/MPF-upgrade/integration//masterfiles/cfe_internal/core/deprecated' + /usr/bin/install -c -m 644 ./cfe_internal/core/deprecated/cfengine_processes.cf '/tmp/MPF-upgrade/integration//masterfiles/cfe_internal/core/deprecated' + /usr/bin/mkdir -p '/tmp/MPF-upgrade/integration//masterfiles/lib/templates' + /usr/bin/install -c -m 644 ./lib/templates/tap.mustache ./lib/templates/junit.mustache '/tmp/MPF-upgrade/integration//masterfiles/lib/templates' + /usr/bin/mkdir -p '/tmp/MPF-upgrade/integration//masterfiles/services/autorun' + /usr/bin/install -c -m 644 ./services/autorun/hello.cf '/tmp/MPF-upgrade/integration//masterfiles/services/autorun' + /usr/bin/mkdir -p '/tmp/MPF-upgrade/integration//masterfiles/lib' + /usr/bin/install -c -m 644 ./lib/testing.cf ./lib/examples.cf ./lib/packages.cf ./lib/common.cf ./lib/users.cf ./lib/guest_environments.cf ./lib/cfengine_enterprise_hub_ha.cf ./lib/edit_xml.cf ./lib/files.cf ./lib/bundles.cf ./lib/reports.cf ./lib/event.cf ./lib/storage.cf ./lib/paths.cf ./lib/vcs.cf ./lib/stdlib.cf ./lib/autorun.cf ./lib/databases.cf ./lib/feature.cf ./lib/cfe_internal_hub.cf ./lib/monitor.cf ./lib/services.cf ./lib/packages-ENT-3719.cf ./lib/commands.cf ./lib/processes.cf ./lib/cfe_internal.cf '/tmp/MPF-upgrade/integration//masterfiles/lib' + /usr/bin/mkdir -p '/tmp/MPF-upgrade/integration//masterfiles/cfe_internal/update' + /usr/bin/install -c -m 644 ./cfe_internal/update/cfe_internal_dc_workflow.cf ./cfe_internal/update/lib.cf ./cfe_internal/update/update_processes.cf ./cfe_internal/update/windows_unattended_upgrade.cf ./cfe_internal/update/systemd_units.cf ./cfe_internal/update/update_policy.cf ./cfe_internal/update/update_bins.cf ./cfe_internal/update/cfe_internal_update_from_repository.cf '/tmp/MPF-upgrade/integration//masterfiles/cfe_internal/update' + /usr/bin/mkdir -p '/tmp/MPF-upgrade/integration//masterfiles/controls' + /usr/bin/install -c -m 644 ./controls/cf_agent.cf ./controls/cf_runagent.cf ./controls/cf_execd.cf ./controls/def_inputs.cf ./controls/cf_monitord.cf ./controls/def.cf ./controls/reports.cf ./controls/update_def_inputs.cf ./controls/cf_serverd.cf ./controls/cf_hub.cf ./controls/update_def.cf '/tmp/MPF-upgrade/integration//masterfiles/controls' + /usr/bin/mkdir -p '/tmp/MPF-upgrade/integration//masterfiles/cfe_internal/enterprise/ha' + /usr/bin/install -c -m 644 ./cfe_internal/enterprise/ha/ha_def.cf ./cfe_internal/enterprise/ha/ha.cf ./cfe_internal/enterprise/ha/ha_update.cf '/tmp/MPF-upgrade/integration//masterfiles/cfe_internal/enterprise/ha' + /usr/bin/mkdir -p '/tmp/MPF-upgrade/integration//masterfiles/modules/packages/vendored' + /usr/bin/install -c -m 644 ./modules/packages/vendored/apk.mustache ./modules/packages/vendored/msiexec.bat.mustache ./modules/packages/vendored/nimclient.mustache ./modules/packages/vendored/snap.mustache ./modules/packages/vendored/yum.mustache ./modules/packages/vendored/msiexec-list.vbs.mustache ./modules/packages/vendored/apt_get.mustache ./modules/packages/vendored/slackpkg.mustache ./modules/packages/vendored/pkgsrc.mustache ./modules/packages/vendored/pkg.mustache ./modules/packages/vendored/freebsd_ports.mustache ./modules/packages/vendored/zypper.mustache ./modules/packages/vendored/WiRunSQL.vbs.mustache '/tmp/MPF-upgrade/integration//masterfiles/modules/packages/vendored' + /usr/bin/mkdir -p '/tmp/MPF-upgrade/integration//masterfiles/cfe_internal' + /usr/bin/install -c -m 644 ./cfe_internal/recommendations.cf ./cfe_internal/CFE_cfengine.cf '/tmp/MPF-upgrade/integration//masterfiles/cfe_internal' + /usr/bin/install -c -m 644 ./update.cf ./promises.cf ./standalone_self_upgrade.cf '/tmp/MPF-upgrade/integration//masterfiles/.' + /usr/bin/mkdir -p '/tmp/MPF-upgrade/integration//masterfiles/cfe_internal/core/watchdog' + /usr/bin/install -c -m 644 ./cfe_internal/core/watchdog/watchdog.cf '/tmp/MPF-upgrade/integration//masterfiles/cfe_internal/core/watchdog' + /usr/bin/mkdir -p '/tmp/MPF-upgrade/integration//masterfiles/cfe_internal/core/watchdog/templates' + /usr/bin/install -c -m 644 ./cfe_internal/core/watchdog/templates/watchdog-windows.ps1.mustache ./cfe_internal/core/watchdog/templates/watchdog.mustache '/tmp/MPF-upgrade/integration//masterfiles/cfe_internal/core/watchdog/templates' + /usr/bin/mkdir -p '/tmp/MPF-upgrade/integration//masterfiles/templates' + /usr/bin/install -c -m 644 ./templates/cf-execd.service.mustache ./templates/cf-apache.service.mustache ./templates/host_info_report.mustache ./templates/cf-monitord.service.mustache ./templates/json_serial.mustache ./templates/json_multiline.mustache ./templates/cf-hub.service.mustache ./templates/cfengine3.service.mustache ./templates/cf-postgres.service.mustache ./templates/cfengine_watchdog.mustache ./templates/vercmp.ps1 ./templates/cf-runalerts.service.mustache ./templates/cf-serverd.service.mustache ./templates/cf-reactor.service.mustache '/tmp/MPF-upgrade/integration//masterfiles/templates' + /usr/bin/mkdir -p '/tmp/MPF-upgrade/integration//masterfiles/cfe_internal/enterprise' + /usr/bin/install -c -m 644 ./cfe_internal/enterprise/CFE_knowledge.cf ./cfe_internal/enterprise/file_change.cf ./cfe_internal/enterprise/CFE_hub_specific.cf ./cfe_internal/enterprise/mission_portal.cf ./cfe_internal/enterprise/main.cf '/tmp/MPF-upgrade/integration//masterfiles/cfe_internal/enterprise' + /usr/bin/mkdir -p '/tmp/MPF-upgrade/integration//masterfiles/templates/federated_reporting' + /usr/bin/install -c -m 644 ./templates/federated_reporting/cfsecret.py ./templates/federated_reporting/import_file.sh ./templates/federated_reporting/psql_wrapper.sh.mustache ./templates/federated_reporting/import.sh ./templates/federated_reporting/transfer_distributed_cleanup_items.sh ./templates/federated_reporting/config.sh.mustache ./templates/federated_reporting/distributed_cleanup.py ./templates/federated_reporting/transport.sh ./templates/federated_reporting/log.sh.mustache ./templates/federated_reporting/dump.sh ./templates/federated_reporting/10-base_filter.sed ./templates/federated_reporting/nova_api.py ./templates/federated_reporting/pull_dumps_from.sh ./templates/federated_reporting/50-merge_inserts.awk ./templates/federated_reporting/parallel.sh '/tmp/MPF-upgrade/integration//masterfiles/templates/federated_reporting' + /usr/bin/mkdir -p '/tmp/MPF-upgrade/integration//masterfiles/services' + /usr/bin/install -c -m 644 ./services/init.cf ./services/main.cf '/tmp/MPF-upgrade/integration//masterfiles/services' +make[2]: Leaving directory '/tmp/MPF-upgrade/integration/masterfiles-source-3.18.5' +make[1]: Leaving directory '/tmp/MPF-upgrade/integration/masterfiles-source-3.18.5' +``` -configure: generating makefile targets -configure: creating ./config.status -config.status: creating Makefile -config.status: creating controls/3.5/update_def.cf -config.status: creating controls/3.6/update_def.cf -config.status: creating controls/3.7/update_def.cf -config.status: creating modules/packages/Makefile -config.status: creating promises.cf -config.status: creating tests/acceptance/Makefile -config.status: creating tests/unit/Makefile +We no longer need the source, we can clean it up. -DONE: Configuration done. Run "make install" to install CFEngine Masterfiles. +```bash +cd $INTEGRATION_ROOT/ +rm -rf $INTEGRATION_ROOT/masterfiles-source-$MPF_VERSION ``` -Then after running `make install` we move the installed masterfiles into our -integration directory. -```console -[root@hub masterfiles]# mv /tmp/masterfiles-3.7.4/masterfiles/* ../MPF_upgrade -[root@hub masterfiles]# cd ../MPF_upgrade/ -``` + ## Merge differences -Now we can use `git status` to see an overview of the changes to the -repository between our starting point and the new MPF. +Now we can use `git status` to see an overview of the changes to the repository between our starting point and the new MPF. + +```bash +cd $INTEGRATION_ROOT/masterfiles +git status +``` + +Output: -```console -[root@hub MPF_upgrade]# git status +``` On branch master Changes not staged for commit: (use "git add/rm ..." to update what will be committed) - (use "git checkout -- ..." to discard changes in working directory) - - deleted: CUSTOM/policy1.cf - deleted: cf_promises_release_id - deleted: cf_promises_validated - modified: cfe_internal/CFE_cfengine.cf - deleted: cfe_internal/CFE_hub_specific.cf - deleted: cfe_internal/CFE_knowledge.cf - deleted: cfe_internal/cfengine_processes.cf - deleted: cfe_internal/ha/ha.cf - deleted: cfe_internal/ha/ha_def.cf - deleted: cfe_internal/host_info_report.cf - deleted: controls/3.4/cf_serverd.cf - deleted: controls/cf_agent.cf - deleted: controls/cf_execd.cf - deleted: controls/cf_hub.cf - deleted: controls/cf_monitord.cf - deleted: controls/cf_runagent.cf - deleted: controls/cf_serverd.cf - deleted: def.cf + (use "git restore ..." to discard changes in working directory) + modified: cfe_internal/core/watchdog/templates/watchdog.mustache + modified: cfe_internal/enterprise/CFE_hub_specific.cf + modified: cfe_internal/enterprise/CFE_knowledge.cf + modified: cfe_internal/enterprise/federation/federation.cf + modified: cfe_internal/enterprise/file_change.cf + modified: cfe_internal/enterprise/mission_portal.cf + modified: cfe_internal/enterprise/templates/httpd.conf.mustache + modified: cfe_internal/update/lib.cf + modified: cfe_internal/update/update_bins.cf + modified: cfe_internal/update/update_policy.cf + modified: cfe_internal/update/update_processes.cf + modified: cfe_internal/update/windows_unattended_upgrade.cf + modified: controls/cf_agent.cf + modified: controls/cf_execd.cf + modified: controls/cf_serverd.cf + modified: controls/def.cf + modified: controls/reports.cf + modified: controls/update_def.cf + deleted: custom-2.cf + deleted: def.json modified: inventory/any.cf modified: inventory/linux.cf - modified: inventory/lsb.cf - modified: lib/3.5/cfe_internal.cf - modified: lib/3.5/common.cf - modified: lib/3.5/files.cf - modified: lib/3.5/packages.cf - deleted: lib/3.5/reports.cf - modified: lib/3.6/cfe_internal.cf - modified: lib/3.6/common.cf - modified: lib/3.6/files.cf - modified: lib/3.6/packages.cf - deleted: lib/3.6/reports.cf - modified: lib/3.6/services.cf - modified: lib/3.6/stdlib.cf + modified: inventory/os.cf + modified: inventory/redhat.cf + modified: lib/autorun.cf + modified: lib/bundles.cf + modified: lib/cfe_internal_hub.cf + deleted: lib/deprecated-upstream.cf + modified: lib/files.cf + modified: lib/packages.cf + modified: lib/paths.cf + modified: lib/services.cf + modified: modules/packages/vendored/apt_get.mustache + modified: modules/packages/vendored/msiexec-list.vbs.mustache + modified: modules/packages/vendored/pkg.mustache + modified: modules/packages/vendored/zypper.mustache modified: promises.cf - deleted: services/autorun.cf - deleted: services/autorun/custom_policy2.cf - deleted: services/file_change.cf - modified: sketches/meta/api-runfile.cf + deleted: services/autorun/custom-1.cf + deleted: services/custom-3.cf + modified: standalone_self_upgrade.cf + modified: templates/cf-apache.service.mustache + modified: templates/cf-execd.service.mustache + modified: templates/cf-hub.service.mustache + modified: templates/cf-monitord.service.mustache + modified: templates/cf-postgres.service.mustache + modified: templates/cf-runalerts.service.mustache + modified: templates/cf-serverd.service.mustache + modified: templates/federated_reporting/config.sh.mustache + modified: templates/federated_reporting/dump.sh + modified: templates/federated_reporting/import.sh + modified: templates/federated_reporting/psql_wrapper.sh.mustache + modified: templates/federated_reporting/pull_dumps_from.sh modified: update.cf - deleted: update/cfe_internal_dc_workflow.cf - deleted: update/cfe_internal_local_git_remote.cf - deleted: update/cfe_internal_update_from_repository.cf - deleted: update/update_bins.cf - deleted: update/update_policy.cf - deleted: update/update_processes.cf Untracked files: (use "git add ..." to include in what will be committed) - - cfe_internal/core/ - cfe_internal/enterprise/ - cfe_internal/update/ - controls/3.5/ - controls/3.6/ - controls/3.7/ - inventory/freebsd.cf - lib/3.6/autorun.cf - lib/3.6/cfe_internal_hub.cf - lib/3.7/ - services/main.cf + cfe_internal/enterprise/templates/apachectl.mustache + lib/templates/ + templates/cf-reactor.service.mustache + templates/federated_reporting/cfsecret.py + templates/federated_reporting/distributed_cleanup.py + templates/federated_reporting/nova_api.py + templates/federated_reporting/transfer_distributed_cleanup_items.sh no changes added to commit (use "git add" and/or "git commit -a") ``` -All of the *Untracked files* are new additions from upstream so they should be -safe to take. - -```console -[root@hub MPF_upgrade]# git add cfe_internal/core/ \ -cfe_internal/enterprise/ \ -cfe_internal/update/ \ -controls/3.5/ \ -controls/3.6/ \ -controls/3.7/ \ -inventory/freebsd.cf \ -lib/3.6/autorun.cf \ -lib/3.6/cfe_internal_hub.cf \ -lib/3.7/ \ -services/main.cf +All of the **Untracked files** are new additions from upstream so they should be safe to take. + +```bash +git add cfe_internal/enterprise/templates/apachectl.mustache +git add lib/templates/junit.mustache +git add lib/templates/tap.mustache +git add templates/cf-reactor.service.mustache +git add templates/federated_reporting/cfsecret.py +git add templates/federated_reporting/distributed_cleanup.py +git add templates/federated_reporting/nova_api.py +git add templates/federated_reporting/transfer_distributed_cleanup_items.sh ``` We can run git status again to see the current overview: -```console -[root@hub MPF_upgrade]# git status +```bash +git status +``` + +Output: + +``` On branch master Changes to be committed: - (use "git reset HEAD ..." to unstage) - - new file: cfe_internal/core/deprecated/cfengine_processes.cf - new file: cfe_internal/core/host_info_report.cf - new file: cfe_internal/core/limit_robot_agents.cf - new file: cfe_internal/core/log_rotation.cf - new file: cfe_internal/core/main.cf - new file: cfe_internal/enterprise/CFE_hub_specific.cf - new file: cfe_internal/enterprise/CFE_knowledge.cf - new file: cfe_internal/enterprise/file_change.cf - new file: cfe_internal/enterprise/ha/ha.cf - new file: cfe_internal/enterprise/ha/ha_def.cf - new file: cfe_internal/enterprise/ha/ha_update.cf - new file: cfe_internal/enterprise/main.cf - new file: cfe_internal/update/cfe_internal_dc_workflow.cf - new file: cfe_internal/update/cfe_internal_local_git_remote.cf - new file: cfe_internal/update/cfe_internal_update_from_repository.cf - new file: cfe_internal/update/update_bins.cf - new file: cfe_internal/update/update_policy.cf - new file: cfe_internal/update/update_processes.cf - new file: controls/3.5/cf_agent.cf - new file: controls/3.5/cf_execd.cf - new file: controls/3.5/cf_hub.cf - new file: controls/3.5/cf_monitord.cf - new file: controls/3.5/cf_runagent.cf - new file: controls/3.5/cf_serverd.cf - new file: controls/3.5/def.cf - new file: controls/3.5/def_inputs.cf - new file: controls/3.5/reports.cf - new file: controls/3.5/update_def.cf - new file: controls/3.5/update_def_inputs.cf - new file: controls/3.6/cf_agent.cf - new file: controls/3.6/cf_execd.cf - new file: controls/3.6/cf_hub.cf - new file: controls/3.6/cf_monitord.cf - new file: controls/3.6/cf_runagent.cf - new file: controls/3.6/cf_serverd.cf - new file: controls/3.6/def.cf - new file: controls/3.6/def_inputs.cf - new file: controls/3.6/reports.cf - new file: controls/3.6/update_def.cf - new file: controls/3.6/update_def_inputs.cf - new file: controls/3.7/cf_agent.cf - new file: controls/3.7/cf_execd.cf - new file: controls/3.7/cf_hub.cf - new file: controls/3.7/cf_monitord.cf - new file: controls/3.7/cf_runagent.cf - new file: controls/3.7/cf_serverd.cf - new file: controls/3.7/def.cf - new file: controls/3.7/def_inputs.cf - new file: controls/3.7/reports.cf - new file: controls/3.7/update_def.cf - new file: controls/3.7/update_def_inputs.cf - new file: inventory/freebsd.cf - new file: lib/3.6/autorun.cf - new file: lib/3.6/cfe_internal_hub.cf - new file: lib/3.7/autorun.cf - new file: lib/3.7/bundles.cf - new file: lib/3.7/cfe_internal.cf - new file: lib/3.7/cfe_internal_hub.cf - new file: lib/3.7/cfengine_enterprise_hub_ha.cf - new file: lib/3.7/commands.cf - new file: lib/3.7/common.cf - new file: lib/3.7/databases.cf - new file: lib/3.7/edit_xml.cf - new file: lib/3.7/examples.cf - new file: lib/3.7/feature.cf - new file: lib/3.7/files.cf - new file: lib/3.7/guest_environments.cf - new file: lib/3.7/monitor.cf - new file: lib/3.7/packages.cf - new file: lib/3.7/paths.cf - new file: lib/3.7/processes.cf - new file: lib/3.7/services.cf - new file: lib/3.7/stdlib.cf - new file: lib/3.7/storage.cf - new file: lib/3.7/users.cf - new file: lib/3.7/vcs.cf - new file: services/main.cf + (use "git restore --staged ..." to unstage) + new file: cfe_internal/enterprise/templates/apachectl.mustache + new file: lib/templates/junit.mustache + new file: lib/templates/tap.mustache + new file: templates/cf-reactor.service.mustache + new file: templates/federated_reporting/cfsecret.py + new file: templates/federated_reporting/distributed_cleanup.py + new file: templates/federated_reporting/nova_api.py + new file: templates/federated_reporting/transfer_distributed_cleanup_items.sh Changes not staged for commit: (use "git add/rm ..." to update what will be committed) - (use "git checkout -- ..." to discard changes in working directory) - - deleted: CUSTOM/policy1.cf - deleted: cf_promises_release_id - deleted: cf_promises_validated - modified: cfe_internal/CFE_cfengine.cf - deleted: cfe_internal/CFE_hub_specific.cf - deleted: cfe_internal/CFE_knowledge.cf - deleted: cfe_internal/cfengine_processes.cf - deleted: cfe_internal/ha/ha.cf - deleted: cfe_internal/ha/ha_def.cf - deleted: cfe_internal/host_info_report.cf - deleted: controls/3.4/cf_serverd.cf - deleted: controls/cf_agent.cf - deleted: controls/cf_execd.cf - deleted: controls/cf_hub.cf - deleted: controls/cf_monitord.cf - deleted: controls/cf_runagent.cf - deleted: controls/cf_serverd.cf - deleted: def.cf + (use "git restore ..." to discard changes in working directory) + modified: cfe_internal/core/watchdog/templates/watchdog.mustache + modified: cfe_internal/enterprise/CFE_hub_specific.cf + modified: cfe_internal/enterprise/CFE_knowledge.cf + modified: cfe_internal/enterprise/federation/federation.cf + modified: cfe_internal/enterprise/file_change.cf + modified: cfe_internal/enterprise/mission_portal.cf + modified: cfe_internal/enterprise/templates/httpd.conf.mustache + modified: cfe_internal/update/lib.cf + modified: cfe_internal/update/update_bins.cf + modified: cfe_internal/update/update_policy.cf + modified: cfe_internal/update/update_processes.cf + modified: cfe_internal/update/windows_unattended_upgrade.cf + modified: controls/cf_agent.cf + modified: controls/cf_execd.cf + modified: controls/cf_serverd.cf + modified: controls/def.cf + modified: controls/reports.cf + modified: controls/update_def.cf + deleted: custom-2.cf + deleted: def.json modified: inventory/any.cf modified: inventory/linux.cf - modified: inventory/lsb.cf - modified: lib/3.5/cfe_internal.cf - modified: lib/3.5/common.cf - modified: lib/3.5/files.cf - modified: lib/3.5/packages.cf - deleted: lib/3.5/reports.cf - modified: lib/3.6/cfe_internal.cf - modified: lib/3.6/common.cf - modified: lib/3.6/files.cf - modified: lib/3.6/packages.cf - deleted: lib/3.6/reports.cf - modified: lib/3.6/services.cf - modified: lib/3.6/stdlib.cf + modified: inventory/os.cf + modified: inventory/redhat.cf + modified: lib/autorun.cf + modified: lib/bundles.cf + modified: lib/cfe_internal_hub.cf + deleted: lib/deprecated-upstream.cf + modified: lib/files.cf + modified: lib/packages.cf + modified: lib/paths.cf + modified: lib/services.cf + modified: modules/packages/vendored/apt_get.mustache + modified: modules/packages/vendored/msiexec-list.vbs.mustache + modified: modules/packages/vendored/pkg.mustache + modified: modules/packages/vendored/zypper.mustache modified: promises.cf - deleted: services/autorun.cf - deleted: services/autorun/custom_policy2.cf - deleted: services/file_change.cf - modified: sketches/meta/api-runfile.cf + deleted: services/autorun/custom-1.cf + deleted: services/custom-3.cf + modified: standalone_self_upgrade.cf + modified: templates/cf-apache.service.mustache + modified: templates/cf-execd.service.mustache + modified: templates/cf-hub.service.mustache + modified: templates/cf-monitord.service.mustache + modified: templates/cf-postgres.service.mustache + modified: templates/cf-runalerts.service.mustache + modified: templates/cf-serverd.service.mustache + modified: templates/federated_reporting/config.sh.mustache + modified: templates/federated_reporting/dump.sh + modified: templates/federated_reporting/import.sh + modified: templates/federated_reporting/psql_wrapper.sh.mustache + modified: templates/federated_reporting/pull_dumps_from.sh modified: update.cf - deleted: update/cfe_internal_dc_workflow.cf - deleted: update/cfe_internal_local_git_remote.cf - deleted: update/cfe_internal_update_from_repository.cf - deleted: update/update_bins.cf - deleted: update/update_policy.cf - deleted: update/update_processes.cf ``` -Next we want to bring back any of our custom policy files. Keeping your -polices organized together helps to make this process easy. The custom policy -files in the example policy set are `CUSTOM/policy1.cf` and -`services/autorun/custom_policy2.cf`. Restore them with `git checkout`. +Next we want to bring back any of our custom files. Look through the **deleted** files, identify your custom files and restore them with `git checkout`. + +```bash +git ls-files --deleted +``` + +Output: + + custom-2.cf + def.json + lib/deprecated-upstream.cf + services/autorun/custom-1.cf + services/custom-3.cf + +Keeping your polices organized together helps to make this process easy. The custom policy files in the example policy set are `def.json`, `services/autorun/custom-1.cf`, `custom-2.cf`, and `services/custom-3.cf`. + +```bash +git checkout custom-2.cf +git checkout def.json +git checkout services/autorun/custom-1.cf +git checkout services/custom-3.cf +``` + +Output: + + Updated 1 path from the index + Updated 1 path from the index + Updated 1 path from the index + Updated 1 path from the index + +Other deleted files from the upstream framework like `lib/deprecated-upstream.cf` should be deleted with `git rm`. + +**Note:** It is uncommon for any files to be moved or deleted between patch releases (e.g. `3.18.0` -> `3.18.5`) like `lib/deprecated-upstream.cf` in this example. + +```bash +git rm lib/deprecated-upstream.cf +``` + +Output: -```console -[root@hub MPF_upgrade] git checkout CUSTOM/policy1.cf services/autorun/custom_policy2.cf +``` +rm 'lib/deprecated-upstream.cf' ``` -The files marked as *modified* in the `git status` output are files that have -changed upstream. +The files marked as **modified** in the `git status` output are files that have changed upstream. -```console -[root@hub MPF_upgrade]# git status | grep modified - modified: cfe_internal/CFE_cfengine.cf +```bash +git status +``` + +Output: + +``` +On branch master +Changes to be committed: + (use "git restore --staged ..." to unstage) + new file: cfe_internal/enterprise/templates/apachectl.mustache + deleted: lib/deprecated-upstream.cf + new file: lib/templates/junit.mustache + new file: lib/templates/tap.mustache + new file: templates/cf-reactor.service.mustache + new file: templates/federated_reporting/cfsecret.py + new file: templates/federated_reporting/distributed_cleanup.py + new file: templates/federated_reporting/nova_api.py + new file: templates/federated_reporting/transfer_distributed_cleanup_items.sh + +Changes not staged for commit: + (use "git add ..." to update what will be committed) + (use "git restore ..." to discard changes in working directory) + modified: cfe_internal/core/watchdog/templates/watchdog.mustache + modified: cfe_internal/enterprise/CFE_hub_specific.cf + modified: cfe_internal/enterprise/CFE_knowledge.cf + modified: cfe_internal/enterprise/federation/federation.cf + modified: cfe_internal/enterprise/file_change.cf + modified: cfe_internal/enterprise/mission_portal.cf + modified: cfe_internal/enterprise/templates/httpd.conf.mustache + modified: cfe_internal/update/lib.cf + modified: cfe_internal/update/update_bins.cf + modified: cfe_internal/update/update_policy.cf + modified: cfe_internal/update/update_processes.cf + modified: cfe_internal/update/windows_unattended_upgrade.cf + modified: controls/cf_agent.cf + modified: controls/cf_execd.cf + modified: controls/cf_serverd.cf + modified: controls/def.cf + modified: controls/reports.cf + modified: controls/update_def.cf modified: inventory/any.cf modified: inventory/linux.cf - modified: inventory/lsb.cf - modified: lib/3.5/cfe_internal.cf - modified: lib/3.5/common.cf - modified: lib/3.5/files.cf - modified: lib/3.5/packages.cf - modified: lib/3.6/cfe_internal.cf - modified: lib/3.6/common.cf - modified: lib/3.6/files.cf - modified: lib/3.6/packages.cf - modified: lib/3.6/services.cf - modified: lib/3.6/stdlib.cf + modified: inventory/os.cf + modified: inventory/redhat.cf + modified: lib/autorun.cf + modified: lib/bundles.cf + modified: lib/cfe_internal_hub.cf + modified: lib/files.cf + modified: lib/packages.cf + modified: lib/paths.cf + modified: lib/services.cf + modified: modules/packages/vendored/apt_get.mustache + modified: modules/packages/vendored/msiexec-list.vbs.mustache + modified: modules/packages/vendored/pkg.mustache + modified: modules/packages/vendored/zypper.mustache modified: promises.cf - modified: sketches/meta/api-runfile.cf + modified: standalone_self_upgrade.cf + modified: templates/cf-apache.service.mustache + modified: templates/cf-execd.service.mustache + modified: templates/cf-hub.service.mustache + modified: templates/cf-monitord.service.mustache + modified: templates/cf-postgres.service.mustache + modified: templates/cf-runalerts.service.mustache + modified: templates/cf-serverd.service.mustache + modified: templates/federated_reporting/config.sh.mustache + modified: templates/federated_reporting/dump.sh + modified: templates/federated_reporting/import.sh + modified: templates/federated_reporting/psql_wrapper.sh.mustache + modified: templates/federated_reporting/pull_dumps_from.sh modified: update.cf ``` -For any files that you have not modified (like those in lib) simply add them -to gits staging area with `git add`. Carefully review and merge or -re-integrate your custom changes on top of the upstream files. +It's best to review the diff of **each** modified to understand the upstream changes as well as identify any local modifications that need to be retained. You should always keep a good record of any modifications made to vendored files to ensure that nothing is lost during future framework upgrades. -The remaining files in `git status` marked as *deleted* are files that have -been moved or removed from upstream. +For example, here the diff for `promises.cf` shows upstream changes but also highlights where the vendored policy had been customized to integrate a custom policy. -**NOTE:** It is uncommon for any files to be moved or deleted between patch -releases (e.g. 3.7.1 -> 3.7.2). +```bash +git diff promises.cf +``` -```console -[root@hub MPF_upgrade]# git status | grep deleted - deleted: cf_promises_release_id - deleted: cf_promises_validated - deleted: cfe_internal/CFE_hub_specific.cf - deleted: cfe_internal/CFE_knowledge.cf - deleted: cfe_internal/cfengine_processes.cf - deleted: cfe_internal/ha/ha.cf - deleted: cfe_internal/ha/ha_def.cf - deleted: cfe_internal/host_info_report.cf - deleted: controls/3.4/cf_serverd.cf - deleted: controls/cf_agent.cf - deleted: controls/cf_execd.cf - deleted: controls/cf_hub.cf - deleted: controls/cf_monitord.cf - deleted: controls/cf_runagent.cf - deleted: controls/cf_serverd.cf - deleted: def.cf - deleted: lib/3.5/reports.cf - deleted: lib/3.6/reports.cf - deleted: services/autorun.cf - deleted: services/file_change.cf - deleted: update/cfe_internal_dc_workflow.cf - deleted: update/cfe_internal_local_git_remote.cf - deleted: update/cfe_internal_update_from_repository.cf - deleted: update/update_bins.cf - deleted: update/update_policy.cf - deleted: update/update_processes.cf -``` - -It's a good idea to review these files as some of them might have contained -modifications, especially `def.cf` and any files under `controls`. Always keep -track of the modifications you make to any of the files that ship with the -MPF. Make sure that any necessary customization's to the deleted files are -carried through to their new locations. - -Once the files are no longer needed you can `git rm` them. - -```console -[root@hub MPF_upgrade]# git rm def.cf cf_promises_release_id cf_promises_validated cfe_internal/CFE_hub_specific.cf cfe_internal/CFE_knowledge.cf cfe_internal/cfengine_processes.cf cfe_internal/ha/ha.cf cfe_internal/ha/ha_def.cf cfe_internal/host_info_report.cf controls/3.4/cf_serverd.cf controls/cf_agent.cf controls/cf_execd.cf controls/cf_hub.cf controls/cf_monitord.cf controls/cf_runagent.cf controls/cf_serverd.cf lib/3.5/reports.cf lib/3.6/reports.cf services/autorun.cf services/file_change.cf update/cfe_internal_dc_workflow.cf update/cfe_internal_local_git_remote.cf update/cfe_internal_update_from_repository.cf update/update_bins.cf update/update_policy.cf update/update_processes.cf -rm 'def.cf' -rm 'cf_promises_release_id' -rm 'cf_promises_validated' -rm 'cfe_internal/CFE_hub_specific.cf' -rm 'cfe_internal/CFE_knowledge.cf' -rm 'cfe_internal/cfengine_processes.cf' -rm 'cfe_internal/ha/ha.cf' -rm 'cfe_internal/ha/ha_def.cf' -rm 'cfe_internal/host_info_report.cf' -rm 'controls/3.4/cf_serverd.cf' -rm 'controls/cf_agent.cf' -rm 'controls/cf_execd.cf' -rm 'controls/cf_hub.cf' -rm 'controls/cf_monitord.cf' -rm 'controls/cf_runagent.cf' -rm 'controls/cf_serverd.cf' -rm 'lib/3.5/reports.cf' -rm 'lib/3.6/reports.cf' -rm 'services/autorun.cf' -rm 'services/file_change.cf' -rm 'update/cfe_internal_dc_workflow.cf' -rm 'update/cfe_internal_local_git_remote.cf' -rm 'update/cfe_internal_update_from_repository.cf' -rm 'update/update_bins.cf' -rm 'update/update_policy.cf' -rm 'update/update_processes.cf' -``` - -Review `git status` and make sure that the policy validates then commit your -changes. - -```console -[root@hub MPF_upgrade]# git status +Output: + +```diff +diff --git a/promises.cf b/promises.cf +index 15c0c40..1fd27e5 100644 +--- a/promises.cf ++++ b/promises.cf +@@ -59,7 +59,6 @@ body common control + main, + @(cfengine_enterprise_hub_ha.management_bundles), + @(def.bundlesequence_end), +-custom_2, + + }; + +@@ -86,23 +85,17 @@ custom_2, + @(services_autorun.inputs), + + "services/main.cf", +-"custom-2.cf", + }; + +- version => "CFEngine Promises.cf 3.18.0"; ++ version => "CFEngine Promises.cf 3.18.5"; + + # From 3.7 onwards there is a new package promise implementation using package + # modules in which you MUST provide package modules used to generate + # software inventory reports. You can also provide global default package module + # instead of specifying it in all package promises. +- (debian).!disable_inventory_package_refresh:: ++ (debian|redhat|centos|suse|sles|opensuse|amazon_linux).cfe_python_for_package_modules_supported.!disable_inventory_package_refresh:: + package_inventory => { $(package_module_knowledge.platform_default) }; + +- # We only define pacakge_invetory on redhat like systems that have a +- # python version that works with the package module. +- (redhat|centos|suse|sles|opensuse|amazon_linux).cfe_yum_package_module_supported.!disable_inventory_package_refresh:: +- package_inventory => { $(package_module_knowledge.platform_default) }; +- + (debian|redhat|suse|sles|opensuse|amazon_linux):: + package_module => $(package_module_knowledge.platform_default); + +@@ -127,6 +120,12 @@ custom_2, + ignore_missing_inputs => "$(def.control_common_ignore_missing_inputs)"; + + ++ control_common_tls_min_version_defined:: ++ tls_min_version => "$(default:def.control_common_tls_min_version)"; # See also: allowtlsversion in body server control ++ ++ control_common_tls_ciphers_defined:: ++ tls_ciphers => "$(default:def.control_common_tls_ciphers)"; # See also: allowciphers in body server control ++ + } + + bundle common inventory +@@ -136,8 +135,6 @@ bundle common inventory + # + # Inventory bundles are simply common bundles loaded before anything + # else in promises.cf +-# +-# Tested to work properly against 3.5.x + { + classes: + "other_unix_os" expression => "!(windows|macos|linux|freebsd|aix)"; +@@ -341,9 +338,7 @@ bundle common services_autorun + # added to inputs automatically. + { + vars: +- services_autorun:: +- "inputs" slist => { "$(sys.local_libdir)/autorun.cf" }; +- ++ services_autorun|services_autorun_inputs:: + "_default_autorun_input_dir" + string => "$(this.promise_dirname)/services/autorun"; + "_default_autorun_inputs" +@@ -360,23 +355,34 @@ bundle common services_autorun + "found_inputs" slist => { @(_default_autorun_inputs), + sort( getvalues(_extra_autorun_inputs), "lex") }; + +- "bundles" slist => { "autorun" }; # run loaded bundles +- +- !services_autorun:: ++ !(services_autorun|services_autorun_inputs|services_autorun_bundles):: + # If services_autorun is not enabled, then we should not extend inputs + # automatically. + "inputs" slist => { }; + "found_inputs" slist => {}; + "bundles" slist => { "services_autorun" }; # run self + ++ services_autorun|services_autorun_inputs|services_autorun_bundles:: ++ "inputs" slist => { "$(sys.local_libdir)/autorun.cf" }; ++ "bundles" slist => { "autorun" }; # run loaded bundles ++ + reports: + DEBUG|DEBUG_services_autorun:: + "DEBUG $(this.bundle): Services Autorun Disabled" +- if => "!services_autorun"; ++ if => "!(services_autorun|services_autorun_bundles|services_autorun_inputs)"; + + "DEBUG $(this.bundle): Services Autorun Enabled" + if => "services_autorun"; + ++ "DEBUG $(this.bundle): Services Autorun Bundles Enabled" ++ if => "services_autorun_bundles"; ++ ++ "DEBUG $(this.bundle): Services Autorun Inputs Enabled" ++ if => "services_autorun_inputs"; ++ ++ "DEBUG $(this.bundle): Services Autorun (Bundles & Inputs) Enabled" ++ if => "services_autorun_inputs.services_autorun_bundles"; ++ + "DEBUG $(this.bundle): adding input='$(inputs)'" + if => isvariable("inputs"); +``` + +Carefully review the diffs and merge or re-integrate your custom changes on top of the upstream files. If you identify changes to the vendored files consider re-integrating those changes in a way that does not modify vendored files, here for example we have migrated the integration of the custom policy to Augments (`def.json`). + +```bash +git diff def.json +``` + +Output: + +```diff +diff --git a/def.json b/def.json +index a7b98e6..60a0ce1 100644 +--- a/def.json ++++ b/def.json +@@ -1,8 +1,11 @@ + { +- "inputs": [ "services/custom-3.cf" ], ++ "inputs": [ "custom-2.cf", "services/custom-3.cf" ], + "classes": { + "default:services_autorun": { + "class_expressions": [ "any::" ], + "comment": "We want to use the autorun functionality because it is convenient." +- } ++ }, ++ "vars":{ ++ "control_common_bundlesequence_end": [ "custom_2" ] ++ } + } +\ No newline at end of file +``` + +So, we now want to accept all the changes to `promises.cf` and `def.json`. + +```bash +git add promises.cf def.json +``` + +If you are unsure if or how to integrate customizations without modifying vendored policy reach out to support for help. For any modified files that you have not customized simply stage them for commit with `git add`. + +```bash +git add cfe_internal/core/watchdog/templates/watchdog.mustache +git add cfe_internal/enterprise/CFE_hub_specific.cf +git add cfe_internal/enterprise/CFE_knowledge.cf +git add cfe_internal/enterprise/federation/federation.cf +git add cfe_internal/enterprise/file_change.cf +git add cfe_internal/enterprise/mission_portal.cf +git add cfe_internal/enterprise/templates/httpd.conf.mustache +git add cfe_internal/update/lib.cf +git add cfe_internal/update/update_bins.cf +git add cfe_internal/update/update_policy.cf +git add cfe_internal/update/update_processes.cf +git add cfe_internal/update/windows_unattended_upgrade.cf +git add controls/cf_agent.cf +git add controls/cf_execd.cf +git add controls/cf_serverd.cf +git add controls/def.cf +git add controls/reports.cf +git add controls/update_def.cf +git add def.json +git add inventory/any.cf +git add inventory/linux.cf +git add inventory/os.cf +git add inventory/redhat.cf +git add lib/autorun.cf +git add lib/bundles.cf +git add lib/cfe_internal_hub.cf +git add lib/files.cf +git add lib/packages.cf +git add lib/paths.cf +git add lib/services.cf +git add modules/packages/vendored/apt_get.mustache +git add modules/packages/vendored/msiexec-list.vbs.mustache +git add modules/packages/vendored/pkg.mustache +git add modules/packages/vendored/zypper.mustache +git add promises.cf +git add standalone_self_upgrade.cf +git add templates/cf-apache.service.mustache +git add templates/cf-execd.service.mustache +git add templates/cf-hub.service.mustache +git add templates/cf-monitord.service.mustache +git add templates/cf-postgres.service.mustache +git add templates/cf-runalerts.service.mustache +git add templates/cf-serverd.service.mustache +git add templates/federated_reporting/config.sh.mustache +git add templates/federated_reporting/dump.sh +git add templates/federated_reporting/import.sh +git add templates/federated_reporting/psql_wrapper.sh.mustache +git add templates/federated_reporting/pull_dumps_from.sh +git add update.cf +``` + +Review `git status` one more time to make sure the changes are as expected. + +```bash +git status +``` + +Output: + +``` On branch master Changes to be committed: - (use "git reset HEAD ..." to unstage) - - deleted: cf_promises_release_id - deleted: cf_promises_validated - modified: cfe_internal/CFE_cfengine.cf - renamed: cfe_internal/cfengine_processes.cf -> cfe_internal/core/deprecated/cfengine_processes.cf - renamed: cfe_internal/host_info_report.cf -> cfe_internal/core/host_info_report.cf - new file: cfe_internal/core/limit_robot_agents.cf - new file: cfe_internal/core/log_rotation.cf - new file: cfe_internal/core/main.cf - renamed: cfe_internal/CFE_hub_specific.cf -> cfe_internal/enterprise/CFE_hub_specific.cf - renamed: cfe_internal/CFE_knowledge.cf -> cfe_internal/enterprise/CFE_knowledge.cf - renamed: services/file_change.cf -> cfe_internal/enterprise/file_change.cf - new file: cfe_internal/enterprise/ha/ha.cf - renamed: cfe_internal/ha/ha_def.cf -> cfe_internal/enterprise/ha/ha_def.cf - new file: cfe_internal/enterprise/ha/ha_update.cf - new file: cfe_internal/enterprise/main.cf - deleted: cfe_internal/ha/ha.cf - renamed: update/cfe_internal_dc_workflow.cf -> cfe_internal/update/cfe_internal_dc_workflow.cf - renamed: update/cfe_internal_local_git_remote.cf -> cfe_internal/update/cfe_internal_local_git_remote.cf - new file: cfe_internal/update/cfe_internal_update_from_repository.cf - renamed: update/update_bins.cf -> cfe_internal/update/update_bins.cf - renamed: update/update_policy.cf -> cfe_internal/update/update_policy.cf - renamed: update/update_processes.cf -> cfe_internal/update/update_processes.cf - deleted: controls/3.4/cf_serverd.cf - renamed: controls/cf_agent.cf -> controls/3.5/cf_agent.cf - new file: controls/3.5/cf_execd.cf - renamed: controls/cf_hub.cf -> controls/3.5/cf_hub.cf - renamed: controls/cf_monitord.cf -> controls/3.5/cf_monitord.cf - renamed: controls/cf_runagent.cf -> controls/3.5/cf_runagent.cf - renamed: controls/cf_serverd.cf -> controls/3.5/cf_serverd.cf - renamed: def.cf -> controls/3.5/def.cf - new file: controls/3.5/def_inputs.cf - renamed: lib/3.5/reports.cf -> controls/3.5/reports.cf - renamed: update.cf -> controls/3.5/update_def.cf - new file: controls/3.5/update_def_inputs.cf - new file: controls/3.6/cf_agent.cf - new file: controls/3.6/cf_execd.cf - new file: controls/3.6/cf_hub.cf - new file: controls/3.6/cf_monitord.cf - new file: controls/3.6/cf_runagent.cf - new file: controls/3.6/cf_serverd.cf - new file: controls/3.6/def.cf - new file: controls/3.6/def_inputs.cf - renamed: lib/3.6/reports.cf -> controls/3.6/reports.cf - new file: controls/3.6/update_def.cf - new file: controls/3.6/update_def_inputs.cf - new file: controls/3.7/cf_agent.cf - new file: controls/3.7/cf_execd.cf - new file: controls/3.7/cf_hub.cf - new file: controls/3.7/cf_monitord.cf - new file: controls/3.7/cf_runagent.cf - new file: controls/3.7/cf_serverd.cf - new file: controls/3.7/def.cf - new file: controls/3.7/def_inputs.cf - new file: controls/3.7/reports.cf - new file: controls/3.7/update_def.cf - new file: controls/3.7/update_def_inputs.cf - deleted: controls/cf_execd.cf + (use "git restore --staged ..." to unstage) + modified: cfe_internal/core/watchdog/templates/watchdog.mustache + modified: cfe_internal/enterprise/CFE_hub_specific.cf + modified: cfe_internal/enterprise/CFE_knowledge.cf + modified: cfe_internal/enterprise/federation/federation.cf + modified: cfe_internal/enterprise/file_change.cf + modified: cfe_internal/enterprise/mission_portal.cf + new file: cfe_internal/enterprise/templates/apachectl.mustache + modified: cfe_internal/enterprise/templates/httpd.conf.mustache + modified: cfe_internal/update/lib.cf + modified: cfe_internal/update/update_bins.cf + modified: cfe_internal/update/update_policy.cf + modified: cfe_internal/update/update_processes.cf + modified: cfe_internal/update/windows_unattended_upgrade.cf + modified: controls/cf_agent.cf + modified: controls/cf_execd.cf + modified: controls/cf_serverd.cf + modified: controls/def.cf + modified: controls/reports.cf + modified: controls/update_def.cf + modified: def.json modified: inventory/any.cf - new file: inventory/freebsd.cf modified: inventory/linux.cf - modified: inventory/lsb.cf - modified: lib/3.5/cfe_internal.cf - modified: lib/3.5/common.cf - modified: lib/3.5/files.cf - modified: lib/3.5/packages.cf - renamed: services/autorun.cf -> lib/3.6/autorun.cf - modified: lib/3.6/cfe_internal.cf - renamed: lib/3.6/cfe_internal.cf -> lib/3.6/cfe_internal_hub.cf - modified: lib/3.6/common.cf - modified: lib/3.6/files.cf - modified: lib/3.6/packages.cf - modified: lib/3.6/services.cf - modified: lib/3.6/stdlib.cf - new file: lib/3.7/autorun.cf - new file: lib/3.7/bundles.cf - new file: lib/3.7/cfe_internal.cf - new file: lib/3.7/cfe_internal_hub.cf - new file: lib/3.7/cfengine_enterprise_hub_ha.cf - new file: lib/3.7/commands.cf - new file: lib/3.7/common.cf - new file: lib/3.7/databases.cf - new file: lib/3.7/edit_xml.cf - new file: lib/3.7/examples.cf - new file: lib/3.7/feature.cf - new file: lib/3.7/files.cf - new file: lib/3.7/guest_environments.cf - new file: lib/3.7/monitor.cf - new file: lib/3.7/packages.cf - new file: lib/3.7/paths.cf - new file: lib/3.7/processes.cf - new file: lib/3.7/services.cf - new file: lib/3.7/stdlib.cf - new file: lib/3.7/storage.cf - new file: lib/3.7/users.cf - new file: lib/3.7/vcs.cf + modified: inventory/os.cf + modified: inventory/redhat.cf + modified: lib/autorun.cf + modified: lib/bundles.cf + modified: lib/cfe_internal_hub.cf + deleted: lib/deprecated-upstream.cf + modified: lib/files.cf + modified: lib/packages.cf + modified: lib/paths.cf + modified: lib/services.cf + new file: lib/templates/junit.mustache + new file: lib/templates/tap.mustache + modified: modules/packages/vendored/apt_get.mustache + modified: modules/packages/vendored/msiexec-list.vbs.mustache + modified: modules/packages/vendored/pkg.mustache + modified: modules/packages/vendored/zypper.mustache modified: promises.cf - new file: services/main.cf - modified: sketches/meta/api-runfile.cf + modified: standalone_self_upgrade.cf + modified: templates/cf-apache.service.mustache + modified: templates/cf-execd.service.mustache + modified: templates/cf-hub.service.mustache + modified: templates/cf-monitord.service.mustache + modified: templates/cf-postgres.service.mustache + new file: templates/cf-reactor.service.mustache + modified: templates/cf-runalerts.service.mustache + modified: templates/cf-serverd.service.mustache + new file: templates/federated_reporting/cfsecret.py + modified: templates/federated_reporting/config.sh.mustache + new file: templates/federated_reporting/distributed_cleanup.py + modified: templates/federated_reporting/dump.sh + modified: templates/federated_reporting/import.sh + new file: templates/federated_reporting/nova_api.py + modified: templates/federated_reporting/psql_wrapper.sh.mustache + modified: templates/federated_reporting/pull_dumps_from.sh + new file: templates/federated_reporting/transfer_distributed_cleanup_items.sh modified: update.cf - deleted: update/cfe_internal_update_from_repository.cf - -[root@hub MPF_upgrade]# cf-promises -cf ./promises.cf -[root@hub MPF_upgrade]# cf-promises -cf ./update.cf -[root@hub MPF_upgrade]# git commit -m "After Policy Upgrade" - 100 files changed, 12521 insertions(+), 1493 deletions(-) - delete mode 100644 cf_promises_release_id - delete mode 100644 cf_promises_validated - rewrite cfe_internal/CFE_cfengine.cf (88%) - rename cfe_internal/{ => core/deprecated}/cfengine_processes.cf (95%) - rename cfe_internal/{ => core}/host_info_report.cf (98%) - create mode 100644 cfe_internal/core/limit_robot_agents.cf - create mode 100644 cfe_internal/core/log_rotation.cf - create mode 100644 cfe_internal/core/main.cf - rename cfe_internal/{ => enterprise}/CFE_hub_specific.cf (85%) - rename cfe_internal/{ => enterprise}/CFE_knowledge.cf (100%) - rename {services => cfe_internal/enterprise}/file_change.cf (58%) - create mode 100644 cfe_internal/enterprise/ha/ha.cf - rename cfe_internal/{ => enterprise}/ha/ha_def.cf (54%) - create mode 100644 cfe_internal/enterprise/ha/ha_update.cf - create mode 100644 cfe_internal/enterprise/main.cf - delete mode 100644 cfe_internal/ha/ha.cf - rename {update => cfe_internal/update}/cfe_internal_dc_workflow.cf (100%) - rename {update => cfe_internal/update}/cfe_internal_local_git_remote.cf (100%) - create mode 100644 cfe_internal/update/cfe_internal_update_from_repository.cf - rename {update => cfe_internal/update}/update_bins.cf (97%) - rename {update => cfe_internal/update}/update_policy.cf (92%) - rename {update => cfe_internal/update}/update_processes.cf (92%) - delete mode 100644 controls/3.4/cf_serverd.cf - rename controls/{ => 3.5}/cf_agent.cf (80%) - create mode 100644 controls/3.5/cf_execd.cf - rename controls/{ => 3.5}/cf_hub.cf (100%) - rename controls/{ => 3.5}/cf_monitord.cf (100%) - rename controls/{ => 3.5}/cf_runagent.cf (100%) - rename controls/{ => 3.5}/cf_serverd.cf (87%) - rename def.cf => controls/3.5/def.cf (74%) - create mode 100644 controls/3.5/def_inputs.cf - rename {lib => controls}/3.5/reports.cf (80%) - rename update.cf => controls/3.5/update_def.cf (59%) - create mode 100644 controls/3.5/update_def_inputs.cf - create mode 100644 controls/3.6/cf_agent.cf - create mode 100644 controls/3.6/cf_execd.cf - create mode 100644 controls/3.6/cf_hub.cf - create mode 100644 controls/3.6/cf_monitord.cf - create mode 100644 controls/3.6/cf_runagent.cf - create mode 100644 controls/3.6/cf_serverd.cf - create mode 100644 controls/3.6/def.cf - create mode 100644 controls/3.6/def_inputs.cf - rename {lib => controls}/3.6/reports.cf (78%) - create mode 100644 controls/3.6/update_def.cf - create mode 100644 controls/3.6/update_def_inputs.cf - create mode 100644 controls/3.7/cf_agent.cf - create mode 100644 controls/3.7/cf_execd.cf - create mode 100644 controls/3.7/cf_hub.cf - create mode 100644 controls/3.7/cf_monitord.cf - create mode 100644 controls/3.7/cf_runagent.cf - create mode 100644 controls/3.7/cf_serverd.cf - create mode 100644 controls/3.7/def.cf - create mode 100644 controls/3.7/def_inputs.cf - create mode 100644 controls/3.7/reports.cf - create mode 100644 controls/3.7/update_def.cf - create mode 100644 controls/3.7/update_def_inputs.cf - delete mode 100644 controls/cf_execd.cf - create mode 100644 inventory/freebsd.cf - rename {services => lib/3.6}/autorun.cf (50%) - rewrite lib/3.6/cfe_internal.cf (67%) - rename lib/3.6/{cfe_internal.cf => cfe_internal_hub.cf} (77%) - create mode 100644 lib/3.7/autorun.cf - create mode 100644 lib/3.7/bundles.cf - create mode 100644 lib/3.7/cfe_internal.cf - create mode 100644 lib/3.7/cfe_internal_hub.cf - create mode 100644 lib/3.7/cfengine_enterprise_hub_ha.cf - create mode 100644 lib/3.7/commands.cf - create mode 100644 lib/3.7/common.cf - create mode 100644 lib/3.7/databases.cf - create mode 100644 lib/3.7/edit_xml.cf - create mode 100644 lib/3.7/examples.cf - create mode 100644 lib/3.7/feature.cf - create mode 100644 lib/3.7/files.cf - create mode 100644 lib/3.7/guest_environments.cf - create mode 100644 lib/3.7/monitor.cf - create mode 100644 lib/3.7/packages.cf - create mode 100644 lib/3.7/paths.cf - create mode 100644 lib/3.7/processes.cf - create mode 100644 lib/3.7/services.cf - create mode 100644 lib/3.7/stdlib.cf - create mode 100644 lib/3.7/storage.cf - create mode 100644 lib/3.7/users.cf - create mode 100644 lib/3.7/vcs.cf - create mode 100644 services/main.cf - rewrite update.cf (78%) - delete mode 100644 update/cfe_internal_update_from_repository.cf +``` + +Make sure the policy validates and commit your changes. + +```bash +git commit -m "Upgraded MPF from 3.18.0 to 3.18.5" +``` + +Output: + +``` +[master 6e0cbd2] Upgraded MPF from 3.18.0 to 3.18.5 + 58 files changed, 2476 insertions(+), 526 deletions(-) + create mode 100644 cfe_internal/enterprise/templates/apachectl.mustache + rewrite inventory/redhat.cf (63%) + delete mode 100644 lib/deprecated-upstream.cf + create mode 100644 lib/templates/junit.mustache + create mode 100644 lib/templates/tap.mustache + create mode 100644 templates/cf-reactor.service.mustache + create mode 100644 templates/federated_reporting/cfsecret.py + create mode 100644 templates/federated_reporting/distributed_cleanup.py + create mode 100644 templates/federated_reporting/nova_api.py + create mode 100644 templates/federated_reporting/transfer_distributed_cleanup_items.sh ``` Now your Masterfiles Policy Framework is upgraded and ready to be tested. diff --git a/examples/tutorials/nfs_and_containers.cf b/examples/tutorials/nfs_and_containers.cf deleted file mode 100644 index 6e5861a8b..000000000 --- a/examples/tutorials/nfs_and_containers.cf +++ /dev/null @@ -1,383 +0,0 @@ -body common control { - - inputs => { - "/var/cfengine/masterfiles/lib/3.6/stdlib.cf", - }; - -} - -bundle common global_vars -{ - vars: - - "host_ip" string => "10.100.100.129"; - "lxc_network" string => "192.168.122"; - "gateway_ip" string => "$(lxc_network).1"; - "container_ip" string => "$(lxc_network).101"; - "container_name" string => "cfe-centos-2"; - -} - -bundle agent remove_existing_items -{ - - commands: - "/usr/bin/lxc-stop -n $(globar_vars.container_name)"; - "/bin/rm -fr /root/3514296"; - "/bin/rm -fr /root/epel-release-6-8.noarch.rpm"; - "/bin/rm -fr /root/remi-release-6.rpm"; - "/bin/rm -fr /root/quick-install-cfengine-enterprise.sh"; - "/bin/rm -fr /var/lib/lxc/$(globar_vars.container_name)"; - "/bin/rm -fr /usr/share/lxc/templates/lxc-centos"; - - reports: - "This bundle ensures we are starting fresh. The command lines could be replaced by native CFEngine functionality."; - -} - -bundle agent change_service_state(service, state) -{ - - commands: - "/sbin/service $(service) $(state)"; - -} - - -bundle agent install_wget -{ - - # Shouldn't be necessary, but just in case - - vars: - - "match_package" slist => { - "wget" - }; - - packages: - "$(match_package)" - package_policy => "add", - package_method => yum; - - reports: - "Installing the wget package is done here using CFEngine's 'packages' promise type. The installation of wget may not be necessary in many cases, as it might already be installed."; - -} - -bundle agent get_cfe_script -{ - commands: - "/usr/bin/wget -nc -P /root https://s3.amazonaws.com/cfengine.packages/quick-install-cfengine-enterprise.sh"; -} - -bundle agent install_ssh_server_and_client -{ - - vars: - - "match_package" slist => { - "openssh-server", - "openssh-clients" - }; - - packages: - "$(match_package)" - package_policy => "add", - package_method => yum; - -} - -bundle agent install_lxc -{ - - commands: - "/usr/bin/wget -nc -P /root http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm"; - "/usr/bin/wget -nc -P /root http://rpms.famillecollet.com/enterprise/remi-release-6.rpm"; - "/bin/rpm -Uvh /root/remi-release-6*.rpm /root/epel-release-6*.rpm"; - - vars: - - "match_package" slist => { - "lxc" - }; - - packages: - "$(match_package)" - package_policy => "add", - package_method => yum; - -} - -bundle agent install_git{ - - vars: - - "match_package" slist => { - "git" - }; - - packages: - "$(match_package)" - package_policy => "add", - package_method => yum; - -} - -bundle agent get_lxc_centos_template -{ - - commands: - "/usr/bin/git clone https://gist.github.com/3514296.git /root/3514296"; - "/bin/cp /root/3514296/lxc-centos /usr/share/lxc/templates/"; #This could be done with CFEngine native functionality as well - "/bin/chmod a+rx /usr/share/lxc/templates/lxc-centos"; - - reports: - "Clone, copy, and change the permissions on the centos template that lxc will use."; - -} - -bundle agent edit_lxc_centos_template -{ - files: - - "/usr/share/lxc/templates/lxc-centos" - create => "false", - edit_line => replace_release_url; - - reports: - "The release url probably needs modification, to change 10 to 11.1 (at the time of writing). This bundle makes the appropriate change, with some help from other bundles (replace_release_url and new_release_url)."; - -} - - -bundle edit_line replace_release_url -{ - replace_patterns: - - "RELEASE_URL=\"$MIRROR_URL/Packages/centos-release-$release-$releaseminor.el6.centos.10.$arch.rpm\".*" - replace_with => new_release_url; - reports: - "Trying to find the current release url line"; - -} - -body replace_with new_release_url -{ - replace_value => "RELEASE_URL=\"$MIRROR_URL/Packages/centos-release-$release-$releaseminor.el6.centos.11.1.$arch.rpm\""; - occurrences => "all"; - -} - -bundle agent create_centos_container -{ - - commands: - "/usr/bin/lxc-create -n $(global_vars.container_name) -t centos -- m 5"; - - reports: - "This bundle uses a command to create the container using the centos template that was downloaded earlier"; - -} - - - -bundle agent configure_centos_container -{ - - files: - - "/var/lib/lxc/$(global_vars.container_name)/rootfs/etc/sysconfig/network-scripts/ifcfg-eth0" - create => "false", - edit_line => modify_ifcfg; - - "/var/lib/lxc/$(global_vars.container_name)/rootfs/etc/sysconfig/network" - create => "false", - edit_line => add_gateway_line; - - "/var/lib/lxc/$(global_vars.container_name)/rootfs/etc/resolv.conf" - create => "true", - edit_line => add_nameserver; - - reports: - "The default configuration of the container requires some modification to define a static IP address. This bundle modifies the appropriate files and also adds some information for DNS to resolv.conf."; - -} - -bundle edit_line modify_ifcfg -{ - replace_patterns: - - "BOOTPROTO=dhcp" - replace_with => change_bootproto; - - insert_lines: - "IPADDR=$(global_vars.container_ip)"; - "NETMASK=255.255.255.0"; - -} - -body replace_with change_bootproto -{ - replace_value => "BOOTPROTO=static"; - occurrences => "all"; -} - -bundle edit_line add_gateway_line -{ - - insert_lines: - "GATEWAY=$(global_vars.gateway_ip)"; - -} - -bundle edit_line add_nameserver -{ - - insert_lines: - "nameserver $(global_vars.gateway_ip)"; - -} - -bundle agent start_centos_container -{ - - commands: - "/bin/rm /var/lib/lxc/cfe-centos-2/rootfs/etc/sysconfig/network-scripts/ifcfg-eth0.cf-before-edit"; - "/usr/bin/lxc-start -d -n $(global_vars.container_name)"; - reports: - "This bundle starts the container that was defined earlier, using a command."; - -} - -bundle agent install_nfs -{ - - vars: - - "match_package" slist => { - "nfs-utils" - }; - - packages: - "$(match_package)" - package_policy => "add", - package_method => yum; - -} - -bundle agent configure_nfs -{ - - files: - - "/var/lib/lxc/$(global_vars.container_name)/rootfs/etc/exports" - create => "true", - edit_line => add_export_line; - "/var/cfengine/masterfiles/def.cf" - create => "false", - edit_line => modify_def; - - reports: - "NFS and CFEngine require some modifications to support the network bridge and container."; - -} - -bundle edit_line add_export_line -{ - - insert_lines: - "/home $(global_vars.host_ip)(rw,sync,no_root_squash,no_subtree_check)"; - -} - -bundle edit_line modify_def -{ - replace_patterns: - - "\"$(sys.policy_hub)/16\".*" - replace_with => change_acl; - -} - -body replace_with change_acl -{ - replace_value => "\"$(sys.policy_hub)/16\",\"192.168.122.*\","; - occurrences => "all"; -} - - - - -bundle agent start_nfs -{ - - methods: - - "any" usebundle => change_service_state("rpcbind","restart"); - "any" usebundle => change_service_state("nfs","restart"); - "any" usebundle => change_service_state("nfslock","restart"); - - reports: - "This bundle starts NFS and related services. It uses restart rather than start just in case the services were already running and modifications made earlier need to be recognized."; - -} - -bundle agent setup_ssh_connection_to_container -{ - - commands: - - "/usr/bin/sshpass -p \"password\" /usr/bin/ssh -o StrictHostKeyChecking=no root@$(global_vars.container_ip) /usr/bin/yum -y install openssh-server openssh-clients"; - "/usr/bin/sshpass -p \"password\" /usr/bin/ssh-copy-id -i /root/.ssh/id_rsa.pub root@$(global_vars.container_ip)"; - - reports: - "Setting up an ssh connection to the container will be used to pass commands from the host machine."; - -} - -bundle agent setup_nfs_on_container -{ - - commands: - "/usr/bin/ssh $(global_vars.container_ip) /usr/bin/yum -y install nfs-utils"; - reports: - "NFS needs to be installed onto the container. This bundle does this using a command call to ssh, and in turn to the yum installer on the container."; - -} - -bundle agent mount_shared_folder_on_container -{ - - commands: - "/usr/bin/ssh $(global_vars.container_ip) /bin/mount $(global_vars.container_ip):/root /root/mount"; - reports: - "After NFS is installed on the container, the shared directory between the two machines can be mounted."; - -} - -bundle agent install_cfengine_on_container -{ - - commands: - "/usr/bin/ssh $(global_vars.container_ip) /bin/chmod a+x /root/mount/quick-install-cfengine-cfengine-enterprise.sh"; - "/usr/bin/ssh $(global_vars.container_ip) /root/mount/quick-install-cfengine-cfengine-enterprise.sh agent"; - reports: - "After the shared folder is mounted, a call to the CFEngine script install can be made from the host via ssh to the container"; - -} - -bundle agent bootstrap_cfengine_on_container -{ - - commands: - "/usr/bin/ssh $(global_vars.container_ip) /var/cfengine/bin/cf-agent --bootstrap $(global_vars.gateway_ip)"; - - reports: - "CFEngine should now be installed, and this bundle will boostrap it to the host machine that is running hub."; - "Note: changes may need to be made to /var/cfengine/masterfiles/controls/cf_serverd.cf on the hub to properly use the container as a CFEngine host across a network bridge (in this tutorial via the 192.168.122.1 gateway). Look for the section '!am_policy_hub.enterprise::' and change all three lines that state 'admit => { \"$(sys.policy_hub)\"};' to read 'admit => { \"$(sys.policy_hub)\",\"192.168.122.1\" };'."; - -} - - - diff --git a/examples/tutorials/nfs_and_containers.markdown b/examples/tutorials/nfs_and_containers.markdown deleted file mode 100644 index 798fc9ce4..000000000 --- a/examples/tutorials/nfs_and_containers.markdown +++ /dev/null @@ -1,22 +0,0 @@ ---- -layout: default -title: NFS and LXC -published: false -sorting: 110 -tags: [examples, tutorials, nfs, lxc, containers] ---- - -1. Downloaded nfs_and_containers.cf). Place it in /var/cfengine/masterfiles/nfs_and_containers.cf. -2. Run the following commands as root on the command line: - - ```console - - ``` -The policy has some notes in reports for several of the bundles used in the bundle sequence. The information will also be shown as output when running the policy using cf-agent as described above. - - - -## Full Policy ## - -[%CFEngine_include_snippet(documentation/examples/tutorials/nfs_and_containers.cf, .* )%] - diff --git a/generator/.gitattributes b/generator/.gitattributes new file mode 100644 index 000000000..409e030f4 --- /dev/null +++ b/generator/.gitattributes @@ -0,0 +1,24 @@ +# +## These files are binary and should be left untouched +# + +# (binary is a macro for -text -diff) +*.png binary +*.jpg binary +*.jpeg binary +*.gif binary +*.ico binary +*.mov binary +*.mp4 binary +*.mp3 binary +*.flv binary +*.fla binary +*.swf binary +*.gz binary +*.zip binary +*.7z binary +*.ttf binary +*.eot binary +*.woff binary +*.pyc binary +*.pdf binary diff --git a/generator/.gitignore b/generator/.gitignore new file mode 100644 index 000000000..1c41f96d3 --- /dev/null +++ b/generator/.gitignore @@ -0,0 +1,9 @@ +# Build Artifacts +_cache/* +_site/* +_generated/* +/nbproject/private/ +/pages/* +*.pyc +.DS_Store + diff --git a/generator/LICENSE b/generator/LICENSE new file mode 100644 index 000000000..b1bc51e02 --- /dev/null +++ b/generator/LICENSE @@ -0,0 +1,22 @@ +Copyright (C) 2013 CFEngine AS + + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/generator/README.org b/generator/README.org new file mode 100644 index 000000000..0d17e4025 --- /dev/null +++ b/generator/README.org @@ -0,0 +1,187 @@ +#+Title: CFEngine Documentation System + +* Writing Documentation +:PROPERTIES: +:ID: ce6c1ae3-7302-4e56-a409-1b4440dc8aad +:END: + +Documentation is written in Markdown. Our [[https://github.com/cfengine/documentation/blob/master/cheatsheet.markdown][Markdown Cheatsheet]] shows useful +formatting examples including usage of our custom extensions. + + +** Site-wide Variables +:PROPERTIES: +:ID: 0121d94c-7e2e-4efb-a66f-b6ceaac54128 +:END: + +For a given branch it can be useful to define variables that can be expanded +anywhere in the documentation. Variables can be defined in [[./_config.yml][=_config.yml=]] and +then referenced in markdown. Jekyll uses the [[https://github.com/Shopify/liquid/wiki][Liquid templating language]] to +process templates. + +[[https://github.com/cfengine/documentation/commit/1793bf6fe8a632e5bb290290cd6044695728f2a6][This commit]] shows liquid variables in use. + +* Building Documentation +:PROPERTIES: +:ID: 4e51b3c2-0970-438d-9587-6a54369533a6 +:END: + +The documentation is rendered using [[https://jekyllrb.com/][jekyll]]. Jekyll is a ruby based static site +generator and is the engine used by GitHub Pages. + +** Build Environment Setup +:PROPERTIES: +:ID: 6b24605a-c9f6-481a-9af5-97aafb68115e +:END: + +*Note:* The documentation build environment has very particular requirements for +*old* libraries and environment. + +*** Installation +:PROPERTIES: +:ID: 8a488c3a-6870-463d-bb6a-eba032727aba +:END: + +This installation was tested on ubuntu 10.04 with vagrant, ruby 1.9.3 and +Java 7. Make sure your machine has everything according to jekyll requirements: +https://github.com/mojombo/jekyll. + +=git= is required to checkout documentation. + +You also need: +- =java= - for closure compiler +- =python-pygments= 1.5 or later for the code highlighting + +Gems: +- =jekyll-asset-pipeline= +- =closure-compiler= +- =yui-compressor= +- =redcarpet= +- =albino= +- =uglifier= +- =execjs= +- =sanitize= + +If =pandoc= is used: +- =pandoc-ruby= +- =rdiscount= + + +- [[./_setup/start.sh][=_setup/start.sh=]] is a shell script to install these dependencies. It needs to +be reviewed and adjusted to your environment accordingly. Clean setup + +- Review steps described in file [[./_setup/step2.txt][=_setup/step2.txt=]] + +- Create pages folder and checkout [[https://github.com/cfengine/documentation.git][CFEngine documentation]] inside. + +- All *content* files should be inside [[./pages][=pages=]] folder. +- All files must have *=meta=* data on top. See (Page meta data section). +- Templates are saved in the [[./_layouts][=_layouts=]] folder. +- Images and other materials saved in *media* folder. +- JS and css saved in _assets folder. To include new files open + [[./_includes/head.html][=_includes/head.html=]] file. +- Make sure you set correct settings in =_config.yml= for =CFE_OUTPUT= and + =CFE_DIR= + +To build the project change into to the project root folder and run =jekyll=. +Your can find the finished rendered files inside [[./pages][=pages=]] folder + +Run local server and automatically build changes to pages: +Change into the project root folder and run =jekyll --auth --server 4000= +where =4000= is the local port number to run the webserver on. + +*Note:* you must restart server after changes in =_config.yml=. + +To configure jekyll edit =_config.yml=. See the [[https://github.com/mojombo/jekyll/wiki/Configuration][jekyll Configuration Wiki]] for a +description of all available options. + +** CI Pipelines for Documentation +:PROPERTIES: +:ID: 63d4b942-d0f1-49a2-a043-4b53691e1d87 +:END: + +The documentation is built automatically by Jenkins. For details see the =Poll +SCM= setting in each =bootstrap-documentation-*= job configuration. + +You can trigger a documentation build for a branch by starting the appropriate +=bootstrap-documentation-*= job manually. + +*** Build Preparation +:PROPERTIES: +:ID: 3ee78a67-cfee-469f-8c59-05c75c90063f +:END: + +The official documentation uses multiple sources in its build. + +- [[https://github.com/cfengine/buildscripts.git][cfengine/buildscripts.git]] +- [[https://github.com/cfengine/core.git][cfengine/core.git]] +- [[https://github.com/cfengine/masterfiles.git][cfengine/masterfiles.git]] +- [[https://github.com/cfengine/nova.git][cfengine/nova.git]] +- [[https://github.com/cfengine/enterprise.git][cfengine/enterprise.git]] +- [[https://github.com/cfengine/mission-portal.git][cfengine/mission-portal.git]] +- [[https://github.com/cfengine/documentation.git][cfengine/documentation.git]] + +After all repositories are updated and available the following is executed. + +#+BEGIN_SRC sh +export WRKDIR=`pwd` +$WRKDIR/documentation/generator/_scripts/cfdoc_bootstrap.py master +cd $WRKDIR/core +NO_CONFIGURE=1 ./autogen.sh +#+END_SRC + +*** Building +:PROPERTIES: +:ID: ca019978-b49c-4f51-8e7f-f1693cfbb375 +:END: + +Documentation builds are performed by the =build-documentation-*= jobs. See +=Execute shell= in one of the =build-documentation-*= jobs. + +For example: + +#+BEGIN_SRC sh + #!/bin/bash + + set -x + + export WRKDIR=`pwd` + cd $WRKDIR/core + + ./configure --with-lmdb=/usr/local --without-pam || exit 1 + + make || exit 2 + + cd $WRKDIR/documentation/generator + + bash -x ./_regenerate_json.sh || exit 3 + + ./_scripts/cfdoc_preprocess.py master || exit 4 + + bash -x ./_scripts/_run_jekyll.sh master || exit 5 +#+END_SRC + + +Parts of the documentation rely on agent output (syntax description from +=cf-promises --syntax-description json=) so compiling the agent is the first +step in building the documentation. + +After the agent has been built the syntax map is regenerated by the +[[https://github.com/cfengine/documentation/blob/master/generator/_regenerate_json.sh][_regenerate_json.sh]] script. + +Once the syntax map has been regenerated [[https://github.com/cfengine/documentation/blob/master/generator/_scripts/cfdoc_preprocess.py][cfdoc_preprocess.py]] is executed. The +=cfdoc_preprocess.py= script appears to be responsible for resolving metadata +(whatever that means), resolving links, expanding our custom macros and creating +printable (dead trees) versions of the documentation. + +After pre-processing is complete [[https://github.com/cfengine/documentation/blob/master/generator/_scripts/_run_jekyll.sh][_run_jekyll.sh]] performs the =jekyll= build, +post processes the build with [[https://github.com/cfengine/documentation/blob/master/generator/_scripts/cfdoc_postprocess.py][cfdoc_postprocess.py]] + +*** Publishing +:PROPERTIES: +:ID: b42ac865-1aae-4862-8695-2d4d88ba0470 +:END: + +At the end of a successful documentation build the artifacts are published to a +public server with the [[https://github.com/cfengine/documentation/blob/master/generator/_scripts/_publish.sh][_publish.sh]] script. See =Post build task= for one of the +=build-documentation-*= jobs. diff --git a/generator/_assets/css/hacks.css b/generator/_assets/css/hacks.css new file mode 100644 index 000000000..861b90cae --- /dev/null +++ b/generator/_assets/css/hacks.css @@ -0,0 +1,108 @@ +.highlight .err { + border: none !important; +} + +header { + border-bottom-color: #052569 !important; + border-bottom-width: 1px !important; +} + +a { + color: #052569 !important; +} + +strong { + font-weight: bold !important; +} + +footer { + background-color: #052569 !important; +} + +footer, +footer a, +footer p { + color: #FFFFFF !important; +} + +footer { + border-top-color: #f5f9fb !important; +} + +#TOCbox { + color: #052569 !important; +} + + +#tags li { + border-color: #052569 !important; +} + +#moveTop a { + border-color: #888888 !important; + color: #888888 !important; +} + +.level1 a { + color: #606770 !important; + font-size: 14px !important; + line-height: 19px !important; +} + +.level2 a { + color: #606770 !important; + font-size: 14px !important; + line-height: 32px !important; +} + +article h1 { + color:#0B132A !important; + font-weight: bold !important; + font-size: 40px !important; +} + +article h2 { + color:#0B132A !important; + font-size: 24px !important; +} + +#left_col .level1>ul>li { + background-color: #F7F7F7 !important; +} + +#left_col .level1>ul>li>ul.level2>li { + background-color: #ffffff !important; +} + + #left_col .level1>ul>li { + border-top: 1px solid #A7AAB2 !important; +} + +#nav_level1_wrapper>ul { + border-radius: 8px !important; +} + +body { + font-family: "Red Hat Display", "Helvetica", "Arial" !important; +} + +html { + color: #4F5665 !important; +} + +.wrapper { + width: 98% !important; + max-width: 1300px !important; +} + +.article_title { + width: 80% !important; +} + +#nav_level1_wrapper>ul { + border: 1px solid #A7AAB2 !important; +} + +#left_col .level1>ul>:nth-child(1) { + border-top: none !important; +} diff --git a/generator/_assets/css/navigation.css b/generator/_assets/css/navigation.css new file mode 100644 index 000000000..8e09cc895 --- /dev/null +++ b/generator/_assets/css/navigation.css @@ -0,0 +1,494 @@ + + +/*main navigation*/ +#nav_container { + line-height: 0; + position: relative; + margin-top:12px; +} + +#nav_wrapper ul { + margin: 0; + padding: 0; + position:absolute; + bottom:12px; + right:0; +} + + #nav_wrapper ul li { + display:inline; + list-style: none; + margin-left:30px; + } + + #nav_wrapper ul li:first-child { + margin-left:0; + } + +#nav_wrapper li a, +#nav_wrapper li span { + font-family: 'DINRoundOT-Bold', 'Helvetica Neue', Helvetica, Arial, Tahoma, sans-serif; + font-size: 16px; + line-height: 16px; + text-decoration: none; + text-transform: uppercase; +} + +#nav_wrapper a:hover, +#nav_wrapper a.hover, +#nav_wrapper a.navselected { + text-decoration: none; + color: #F5821F; +} + +#nav_wrapper a:active, +#nav_wrapper a.navselected { + color: #4A4A4A; + text-decoration: underline; +} + +/*left and breadcrumbs*/ +#nav_level1_wrapper > ul { + border: 1px solid #A4BFC8; + border-radius: 5px; + background-color: #E1EBED; + width: 262px; + overflow: hidden; +} + +#breadcrumbs ul, +#left_col ul, +.sidr-inner ul { + margin: 0 !important; + padding: 0 !important; +} + + #left_col ul li, + .sidr-inner ul li { + clear: both; + font-size: 13px; + line-height: 28px; + text-align: left; + text-decoration: none; + white-space: normal; + } + + #left_col ul li a, + #left_col ul li span, + .sidr-inner ul li a, + .sidr-inner ul li span { + color: #4386A5; + text-decoration: none; + padding: 0 14px; + display:inline-block; + } + + #left_col ul li a:hover, + .sidr-inner ul li a:hover { + text-decoration: underline; + } + +#breadcrumbs li, +#left_col li, +.sidr-inner li { + list-style: none +} + +#left_col .level1 > ul > li { + background: #E1EBED; + border-top: 1px solid #A4BFC8; + font-weight: 700; + color: #F5821F; + line-height: 18px; +} + +#left_col .level1 > ul > li:first-child { + border-top: none; +} + +#left_col .level1 > ul > li a, +#left_col .level1 > ul > li span { + color: #F5821F; + text-transform: uppercase; + padding-top: 6px; + padding-bottom: 6px; +} + +.sidr-inner .sidr-class-level1 li a, +.sidr-inner .sidr-class-level1 li span { + +} + +#left_col ul.level2, +.sidr-inner ul.level2 { + border-top: 1px solid #A4BFC8; +} + +#left_col .level1 > ul > li > ul.level2 li.selected span { + color: #222; +} + +#left_col .level1 > ul > li > ul.level2 > li { + line-height: 26px; + font-weight: 500; + background-color: #f6f6f6; +} + +#left_col .level1 > ul > li > ul.level2 > li:first-child { + padding-top: 6px; +} +#left_col .level1 > ul > li > ul.level2 > li:last-child { + padding-bottom: 6px; +} + + +#left_col .level1 > ul > li > ul.level2 > li a, +#left_col .level1 > ul > li > ul.level2 > li span, +.sidr-inner .level2 li a { + color: #4386A5; + text-transform: none; + padding-top: 0; + padding-bottom: 0; +} + +#left_col ul.level3 { + padding: 0 14px !important; +} + +#left_col .level3 li { + margin-left: 28px; +} + +#left_col .level3 li:before { + margin-left: -13px; + margin-right: 5px; + font-family: FontAwesome; + content: "\f101"; + color: #4386A5; +} + +#left_col ul.level3 > li a, +#left_col ul.level3 > li span { + display: inline; + padding: 0; +} + +#left_col li.selected span, +.sidr-inner li.selected span { + color: #F5821F; +} + +#breadcrumbs_wrapper { + display:block; + width:100%; + position:relative; + min-height: 25px; +} + +#breadcrumbs { + display: inline-block; + width: 477px; +} + +#breadcrumbs li { + float: left; + margin: 0 6px 0 0; +} + +#breadcrumbs li:after { + font-size: 11px; + margin: 0 0 0 3px; + font-family: FontAwesome; + content: "\f101"; + color: #4386A5; +} + +#breadcrumbs li:last-child:after { + display: none; +} + +#breadcrumbs a { + font-weight: 300; + font-size: 14px; + color: #4386A5; + line-height: 17px; + text-decoration: none; +} + +#breadcrumbs a:hover { + text-decoration: underline; +} + +#breadcrumbs span { + border-bottom: 1px solid #f5821f; + color: #222; + font-size: 1em; + padding-bottom: 10px; +} + +#breadcrumbs_wrapper a i { + font-size: 18px; + color: #A4BFC8; + line-height: 18px; +} + +#breadcrumbs_wrapper a i:hover { + color: #4386A5; +} + +#tags { + color: #15357F; + padding-bottom: 3.2rem; + margin: 4rem 0; + border-bottom: 1px solid #E2E2E2; +} + +#tags ul { + list-style: none; + padding-left: 0; +} + +#tags li { + display: inline-block; + border: 1px solid #15357F; + border-radius: 4px; + padding: 4px 8px; + margin-right: 8px; + font-weight: 400; + font-size: 12px; + line-height: 16px; +} + +#tags li:hover, #tags li:focus { + background: #E6E9F3; +} + +#tags a { + color: inherit; + text-decoration: none; +} + +#TOCbox_wrapper { + float: right; + position: relative; + padding-top:4px; +} + +#TOCbox, .TOCheader { + font-weight: 500; + font-size: 15px; + color: #4386A5; + line-height: 17px; + cursor: pointer; +} + +#TOCbox:hover, +.TOCheader:hover { + text-decoration: underline; +} + +#TOCbox i.fa { + font-size: 11px; + margin-right: 4px; + vertical-align: middle; + line-height: 16px; +} + +.TOCheader { + padding: 0 14px 10px; + border-bottom: 1px solid #A4BFC8; +} + + #TOCbox_wrapper #TOCbox .TOCheader { + color: #555; + } + +#TOCbox_list { + display: none; + max-height: 420px; + overflow: auto; + padding: 10px 0; + position: absolute; + right: -15px; + top: -2px; + min-width: 250px; + border-radius: 5px; + background: rgba(246,246,246,0.90); + border: 1px solid #A4BFC8; + -moz-box-shadow: 1px 1px 5px 0px rgba(151,151,151,0.50); + box-shadow: 1px 1px 5px 0px rgba(151,151,151,0.50); + text-align: right; +} + + #TOCbox_list nav { + text-align: left; + padding: 10px 15px 0; + } + + #TOCbox_list .link_h1 a, + #TOCbox_list_mobile .sidr-class-link_h1 a { + font-weight: 500; + font-size: 15px; + color: #f5821f; + line-height: 17px; + text-transform: uppercase; + } + + #TOCbox_list .link_h3, + #TOCbox_list_mobile .sidr-class-link_h3 { + margin-left: 1em; + } + + #TOCbox_list .link_h3:before, + #TOCbox_list_mobile .sidr-class-link_h3:before, + #TOCbox_list .link_h4:before, + #TOCbox_list_mobile .sidr-class-link_h4:before { + font-size: 11px; + margin: 0 6px 0 0; + font-family: FontAwesome; + content: "\f101"; + color: #4386A5; + } + + #TOCbox_list .link_h4, + #TOCbox_list_mobile .sidr-class-link_h4 { + margin-left: 2em; + } + + #TOCbox_list li, + #TOCbox_list_mobile li { + line-height: 1.6; + list-style: none !important; + } + + #TOCbox_list li a, + #TOCbox_list_mobile li a { + background: none; + text-decoration: none; + font-weight: 500; + font-size: 13px; + color: #F5821F; + line-height: 25px; + } + + #TOCbox_list li a { + color: #4386A5; + } + + #TOCbox_list ul, + #TOCbox_list_mobile ul { + margin: 0; + padding: 0; + } + +#moveTop { + bottom: 20px; + display: none; + right: 20px; + position: fixed; + width: 45px; + z-index:999999; +} + + #moveTop a { + display: block; + text-align: center; + text-decoration: none; + width: 39px; + border-radius: 50%; + border: 3px solid #0c668f; + color: #0c668f; + } + + #moveTop a:hover { + border-color: #f5821f; + } + + #moveTop a:before { + content: '\f077'; + font-family: FontAwesome; + font-style: normal; + font-weight: normal; + speak: none; + display: inline-block; + text-decoration: inherit; + width: 40px; + font-size: 22px; + line-height: 36px; + text-align: center; + font-variant: normal; + text-transform: none; + margin-bottom: 3px; + } + + #moveTop a:hover:before { + color: #f5821f; + } + + +#mobile_naw_wrapper { + background: #036990; + bottom: 0; + opacity: 0.8; + padding: 0.5em 0; + position: fixed; + width: 100%; +} + + #mobile_naw_wrapper a { + display: inline-block; + text-decoration: none; + } + + #mobile_naw_wrapper a:hover { + background: #000; + border: 2px solid #000; + } + +#mobile_links table { + width: 100% +} + +#mobile_links table, +#mobile_links td { + border: none; + padding: 0; +} + +#mobile_left { + text-align: left +} + +#mobile_center { + text-align: center +} + +#mobile_right { + text-align: right +} + +#mobile_left_menu, +#mobile_go_top, +#mobile_right_menu { + background: #F6F6F6; + border: 2px solid #AAC4D0; + border-radius: 5px 5px 5px 5px; + color: #0071A6; + font-size: 0.7em; + height: 1.2em; + line-height: 1.2em; + padding: 5px; + text-align: center; +} + +#mobile_left_menu { + width: 60px +} + +#mobile_go_top { + width: 60px +} + +#mobile_right_menu { + width: 92px +} \ No newline at end of file diff --git a/generator/_assets/css/print.css b/generator/_assets/css/print.css new file mode 100644 index 000000000..07666b318 --- /dev/null +++ b/generator/_assets/css/print.css @@ -0,0 +1,101 @@ +@media print { + .wrapper { + padding: 0 10px + } + + #left_col, + #nav_wrapper, + header, + #TOCbox_wrapper, + #pageControls_block, + #mobile_naw_wrapper { + display: none; + visibility: hidden; + } + + #content_col { + margin: 0 0 0 0 + } + + footer { + font-size: 8px + } + + footer .download_info, + footer .contact_info { + display: none + } + + footer .license_info { + border-top: 1px solid #000; + height: 20px; + margin: 40px 0 0 0; + padding-top: 10px; + } + + footer .license_info, + footer .license_info a { + background: none; + color: #000; + line-height: 1; + } + + a[href]:after { + content: " (" attr(href) ") " + } + + #breadcrumbs a[href]:after, + #tags a[href]:after, + #print_TOC_wrapper a[href]:after { + content: "" + } + + #breadcrumbs i.arrow { + background: url('../media/images/single_arrow.png') no-repeat 0 7px + } + + #tags li a { + background: none !important; + margin: 0 0 0 10px; + padding: 0; + } + + #tags li span { + background: none; + margin: 0 0 0 0px; + padding: 0; + } + + #print_TOC a { + text-decoration: none + } + + + #print_TOC .link_h3 { + margin-left: 1em + } + + #print_TOC .link_h4 { + margin-left: 2em + } + + body:before { + background: url('../media/images/cfengine-logo.svg') no-repeat; + color: transparent; + content: "CFEngine"; + height: 26px; + left: 0px; + position: absolute; + top: -55px; + width: 100px; + } + + @page + { + margin: 3cm 2cm; + @bottom-right { + font-size: 80%; + content: "Page " counter(page) " of " counter(pages); + } + } +} diff --git a/generator/_assets/css/pygments.css b/generator/_assets/css/pygments.css new file mode 100644 index 000000000..67e6ea397 --- /dev/null +++ b/generator/_assets/css/pygments.css @@ -0,0 +1,61 @@ +.hll { background-color: #ffffcc } +.c { color: #408080; font-style: italic } /* Comment */ +.err { border: 1px solid #FF0000 } /* Error */ +.k { color: #008000; font-weight: bold } /* Keyword */ +.o { color: #666666 } /* Operator */ +.cm { color: #408080; font-style: italic } /* Comment.Multiline */ +.cp { color: #BC7A00 } /* Comment.Preproc */ +.c1 { color: #408080; font-style: italic } /* Comment.Single */ +.cs { color: #408080; font-style: italic } /* Comment.Special */ +.gd { color: #A00000 } /* Generic.Deleted */ +.ge { font-style: italic } /* Generic.Emph */ +.gr { color: #FF0000 } /* Generic.Error */ +.gh { color: #000080; font-weight: bold } /* Generic.Heading */ +.gi { color: #00A000 } /* Generic.Inserted */ +.go { color: #888888 } /* Generic.Output */ +.gp { color: #000080; font-weight: bold } /* Generic.Prompt */ +.gs { font-weight: bold } /* Generic.Strong */ +.gu { color: #800080; font-weight: bold } /* Generic.Subheading */ +.gt { color: #0044DD } /* Generic.Traceback */ +.kc { color: #008000; font-weight: bold } /* Keyword.Constant */ +.kd { color: #008000; font-weight: bold } /* Keyword.Declaration */ +.kn { color: #008000; font-weight: bold } /* Keyword.Namespace */ +.kp { color: #008000 } /* Keyword.Pseudo */ +.kr { color: #008000; font-weight: bold } /* Keyword.Reserved */ +.kt { color: #B00040 } /* Keyword.Type */ +.m { color: #666666 } /* Literal.Number */ +.s { color: #BA2121 } /* Literal.String */ +.na { color: #7D9029 } /* Name.Attribute */ +.nb { color: #008000 } /* Name.Builtin */ +.nc { color: #0000FF; font-weight: bold } /* Name.Class */ +.no { color: #880000 } /* Name.Constant */ +.nd { color: #AA22FF } /* Name.Decorator */ +.ni { color: #999999; font-weight: bold } /* Name.Entity */ +.ne { color: #D2413A; font-weight: bold } /* Name.Exception */ +.nf { color: #0000FF } /* Name.Function */ +.nl { color: #A0A000 } /* Name.Label */ +.nn { color: #0000FF; font-weight: bold } /* Name.Namespace */ +.nt { color: #008000; font-weight: bold } /* Name.Tag */ +.nv { color: #19177C } /* Name.Variable */ +.ow { color: #AA22FF; font-weight: bold } /* Operator.Word */ +.w { color: #bbbbbb } /* Text.Whitespace */ +.mf { color: #666666 } /* Literal.Number.Float */ +.mh { color: #666666 } /* Literal.Number.Hex */ +.mi { color: #666666 } /* Literal.Number.Integer */ +.mo { color: #666666 } /* Literal.Number.Oct */ +.sb { color: #BA2121 } /* Literal.String.Backtick */ +.sc { color: #BA2121 } /* Literal.String.Char */ +.sd { color: #BA2121; font-style: italic } /* Literal.String.Doc */ +.s2 { color: #BA2121 } /* Literal.String.Double */ +.se { color: #BB6622; font-weight: bold } /* Literal.String.Escape */ +.sh { color: #BA2121 } /* Literal.String.Heredoc */ +.si { color: #BB6688; font-weight: bold } /* Literal.String.Interpol */ +.sx { color: #008000 } /* Literal.String.Other */ +.sr { color: #BB6688 } /* Literal.String.Regex */ +.s1 { color: #BA2121 } /* Literal.String.Single */ +.ss { color: #19177C } /* Literal.String.Symbol */ +.bp { color: #008000 } /* Name.Builtin.Pseudo */ +.vc { color: #19177C } /* Name.Variable.Class */ +.vg { color: #19177C } /* Name.Variable.Global */ +.vi { color: #19177C } /* Name.Variable.Instance */ +.il { color: #666666 } /* Literal.Number.Integer.Long */ diff --git a/generator/_assets/css/responsive.css b/generator/_assets/css/responsive.css new file mode 100644 index 000000000..0a964a538 --- /dev/null +++ b/generator/_assets/css/responsive.css @@ -0,0 +1,22 @@ +/* + ========================================================================== + Media Queries + ========================================================================== +*/ +@media only screen and (min-width: 320px) { + + #top_version_wrapper, + .top_search_form { + display: inline-block; + margin-left:20px; + } + + #top_version_wrapper select { + width: 100% + } + +} + +#mobile_naw_wrapper { + display: none +} \ No newline at end of file diff --git a/generator/_assets/css/sidr_mobile_menu.css b/generator/_assets/css/sidr_mobile_menu.css new file mode 100644 index 000000000..f87219ac7 --- /dev/null +++ b/generator/_assets/css/sidr_mobile_menu.css @@ -0,0 +1,20 @@ +/*styles for left and right meny placeholder used in mobile*/ +.sidr { + -moz-box-shadow: inset 0 0 5px 5px #ccc; + -webkit-box-shadow: inset 0 0 5px 5px #ccc; + background: #fff; + box-shadow: inset 0 0 5px 5px #222; + color: #fff; + display: none; + font-family: "lucida grande",tahoma,verdana,arial,sans-serif; + font-size: 15px; + height: 100%; + overflow-x: none; + overflow-y: auto; + padding: 2em 0 0 2em; + position: absolute; + position: fixed; + top: 0; + width: 260px; + z-index: 999999; +} \ No newline at end of file diff --git a/generator/_assets/css/style.css b/generator/_assets/css/style.css new file mode 100644 index 000000000..8e9b63154 --- /dev/null +++ b/generator/_assets/css/style.css @@ -0,0 +1,561 @@ +html{color:#555;background:#FFF; font-size: 62.5%; // 1 rem = 10 px}body,div,dl,dt,dd,h1,h2,h3,h4,h5,h6,pre,code,form,fieldset,legend,input,button,textarea,select,p,blockquote,th,td{margin:0;padding:0}table{border-collapse:collapse;border-spacing:0}fieldset,img{border:0}address,button,caption,cite,code,dfn,em,input,optgroup,option,select,strong,textarea,th,var{font:inherit}del,ins{text-decoration:none}caption,th{text-align:left}h1,h2,h3,h4,h5,h6{font-size:100%;font-weight:normal}q:before,q:after{content:''}abbr,acronym{border:0;font-variant:normal}sup{vertical-align:baseline}sub{vertical-align:baseline} +table {border-collapse: collapse; border-spacing: 0} +.clear {clear:both; height:0; font-size:0; overflow:hidden} +.invisible {display:none} + +@font-face { + font-family: 'DINRoundOT-Bold'; + src: url('../media/fonts/DINRoundOT-Bold.otf') format('opentype'); +} + +html,body { + width:100%; + height:100%; +} + +body { + background: #fff; + font-family: museo-sans, "Helvetica Neue", Helvetica, Arial, Tahoma, sans-serif; + font-size: 1em; + line-height: 1.6; + font-weight: 500; + height:100%; +} + +#full_container { + min-height: 100%; + min-width: 100%; + position: absolute; +} + +a:link { + color: #008cbc; +} + +a:link {color: #008cbc;text-decoration:none;} +a:visited {color: #008cbc;text-decoration:none;} +a:hover {color: #008cbc;text-decoration:underline;} +a:active {color: #008cbc;text-decoration:none;} + +.float-right { + float:right; +} + +.uppercase { + text-transform: uppercase; +} + +.wrapper { + margin:0 auto; + width: 960px; + padding:0 20px; +} + +header { + clear: both; + padding: 20px 0 12px; + border-bottom: 7px solid #A4BFC8; + width:100%; + min-width: 960px; +} + +.group:after { + content: ""; + display: table; + clear: both; +} + +footer { + background: #7E98A3; + border-top: 20px solid #E8F0F1; + padding: 60px 0 40px; + font-weight:300; + font-size: 13px; + color: #FFFFFF; + width:100%; + min-width: 960px; + position: absolute; + bottom: 0; +} + + footer a, + footer a:link, + footer a:visited { + color: #fff; + text-decoration: underline; + } + + footer .license_info { + float: left; + width: 640px; + } + + footer .contact_info { + margin-left: 797px; + color: #222; + } + + footer .contact_info h5 { + color: #FFF; + padding-bottom: 8px; + font-weight: 300; + } + + footer .contact_info a { + color: #222; + } + + footer .download_info { + margin-bottom: 10px; + } + + footer .download_info i { + font-family: FontAwesome; + font-size: 20px; + line-height: 20px; + margin-right: 8px; + vertical-align:top; + } + + footer a:hover { + color: #F5821F; + } + + +ul li { + list-style: disc; + list-style-position: outside; +} + +ol li { + list-style: decimal; + list-style-position: outside; +} + +strong, +b { + font-weight: 500 +} + +em, +i { + font-style: italic +} + +aside { + display: block; + float: right; + width: 390px; +} + +blockquote { + border-left: .5em solid #eee; + margin-left: 0; + max-width: 476px; + padding: 0 2em; +} + + blockquote cite:before { + content: '\2014 \00A0' + } + + blockquote cite { + font-size: 14px; + line-height: 20px; + } + + blockquote p { + max-width: 460px + } + +dd { + margin-bottom: 1em +} + + dd pre { + margin: 1em 0px + } + + dd p { + margin-top: 0px + } + +dt { + font-style: italic +} + +sub { + font-size: xx-small; + vertical-align: sub; +} + +sup { + font-size: xx-small; + vertical-align: super; +} + +code { + background: #EEEFF0; + border-radius: 4px; + padding: 0 5px; + line-height: 24px; + font-size: 15px; +} + +div.highlight { + background: #F7F7F7; + padding-right: 40px; + position: relative; + border: 1px solid #D1D2D3; + margin: 1.2rem 0 4rem 0; +} + +div.highlight .copy-to-clipboard { + position: absolute; + right: 20px; + top: 20px; + cursor: pointer; + background: #F7F7F7; +} + +div.highlight .copy-to-clipboard:hover { + color: #1F83E7; +} + +div.highlight .copy-to-clipboard:hover:after { + color: #010101; + font-style: normal; + content: "Copy to clipboard"; + position: absolute; + left: 20px; + top: 3px; + width: max-content; + background: #efefef; + border: 1px solid #d1d1d1; + border-radius: 0.4rem; + padding: 0 0.8rem; + font-size: 11px; + line-height: 16px; +} + +div.highlight .copy-to-clipboard:hover.bi-check2:after { + content: "Copied!"; +} + +pre { + background: #F7F7F7; + overflow: visible; + overflow-y: hidden; + padding: 2rem; + font-family: Menlo, monospace, sans-serif; + font-style: normal; + font-weight: 400; + font-size: 13px; + line-height: 150%; +} + +pre code { + background: none; + padding: 0; + margin: 0; + font-size: inherit; +} + +.tableOverflow { + overflow: auto; + padding-bottom: 10px; + width: 100%; +} + +table { + border-collapse: collapse; + border-spacing: 0; + table-layout: fixed; +} + +table, +td { + border: 1px solid #ccc +} + +td { + font-weight: normal; + text-align: left; +} + +th { + background-color: #E5E5E5 +} + +td, +th { + margin: 0px; + padding: 0.1em 1em; +} + +#main_content, #googleSearchPage { + padding:26px 0 150px; + margin-bottom: 150px; +} + +#left_col { + float: left; + width:304px; +} + +#content_col { + margin-left: 324px; +} + +#googleSearchPage #content_col { + margin: 0 auto !important; + padding: 0 2%; + width: 96%; +} + + .googleSearch #cse-search-box { + margin: 30px auto; + width: 98%; + } + +#top_form { + display:block; + width:100%; + clear:both; + font-size:12px; + line-height: 14px; +} + +#top_form select { + background: -moz-linear-gradient(center top, #E9E9E9, #FFFFFF) repeat scroll 0 0 transparent; + border: 1px solid #CCCCCC; + border-radius: 5px; + line-height: 2em; + padding: 5px 10px; + text-align: left; + vertical-align: middle; + min-width:110px; +} + +#top_search { + border-radius: 5px; + background: #F6F6F6; + border: 1px solid #87ABAE; + font-weight: 300; + font-style: italic; + color: #87ABAE; + line-height: 12px; + padding:5px 10px; + width:180px; +} + +#top_search_searchpage { + border: 2px solid #aac4d0; + border-radius: 5px; + color: #777777; + height: 34px; + margin-right: 1em; + padding-left: 13px; + text-align: left; + width: 55%; + +} + +#logo { + float:left; +} + +#logo a img { + vertical-align: baseline; +} + + +article { + margin-top: 20px; + font-weight: 400; + font-size: 16px; + line-height: 24px; +} + .article_title { + width: 477px; + display:inline-block; + } + + article i.fa-link { + font-size: 18px; + color: #4386A5; + line-height: 18px; + visibility: hidden; + display: inline-block; + position:relative; + top:-3px; + margin-right: 8px; + } + + .article_title i.fa-link { + margin-right: 0; + } + + article h1, + article h2, + article h3, + article h4, + article h5, + article h6 { + font-weight: 700; + margin-bottom: 1.2rem; + } + + article h1:hover i, + article h2:hover i, + article h3:hover i, + article h4:hover i, + article h5:hover i { + visibility: visible; + } + + article h1 { + color: #0b132a; + font-size: 36px; + line-height: 44px; + } + + article h2 { + font-size: 24px; + line-height: 32px; + margin-top: 4rem; + } + + article h3 { + font-size: 19px; + line-height: 24px; + margin-top: 2.4rem; + } + + article h4 { + font-size: 18px; + line-height: 27px; + padding-bottom: 8px; + } + + article h5 { + font-size: 16px; + font-style: italic; + line-height: 24px; + padding-bottom: 6px; + } + + article h6 { + font-size: 1.0em + } + + article p { + font-weight: 400; + font-size: 16px; + line-height: 24px; + margin-bottom: 16px; + } + + .article a { + color: #1D71D3 !important; + text-decoration: none; + font-weight: 400; + font-size: 16px; + line-height: 24px; + } + +.article a:hover { + font-weight: 400; + font-size: 16px; + color: #4386A5 !important; + line-height: 24px; +} + + .article a:hover { + border-bottom: .1rem solid #4386A5 !important; + } + + .article a.anchor { + margin-left: -26px; + } + + .article ul, .article ol { + margin-left: 3.2rem; + margin-bottom: 2.4rem; + padding-left: 0; + } + +.article img, .article iframe { + margin: 32px 16px 40px; + max-width: 100%; +} + +.article img { + box-shadow: 0px 2px 4px rgba(0, 0, 0, 0.1); +} + +#nav_show_category { + border-top: 1px solid #e5e5e5; + margin-top: 10px; + max-width: 300px; + padding-top: 10px; + width: 100%; +} + +#pageControls_block { + float: right; + width: 70px; +} + +.gsc-above-wrapper-area table, +.gsc-above-wrapper-area td, +.gsc-above-wrapper-area-container, +.gsc-above-wrapper-area-container td, +.gsc-resultsHeader, +.gsc-resultsHeader td, +.gsc-twiddleRegionCell, +.gsc-configLabelCell, +.gsc-above-wrapper-area, +.gsc-table-result, +.gsc-table-result th, +.gsc-table-result td { + border: none !important +} + +#pageControls_block .print { + background-position: -54px 0; + width: 30px; +} + +#pageControls_block .pdf, +#pageControls_block .print { + background-image: url("../media/images/btn_pdf_print.png"); + background-repeat: no-repeat; + display: inline-block; + font-size: 0; + height: 27px; + margin-top: 7px; + text-decoration: none; + vertical-align: top; +} + +#pageControls_block .print:hover { + background-position: -85px 0 +} + +#pageControls_block .pdf:hover { + background-position: -27px 0 +} + +#pageControls_block .pdf { + background-position: 0 0; + width: 26px; +} + +#print_TOC_wrapper { + display: none +} + +#edit_link { + display:none; +} \ No newline at end of file diff --git a/generator/_assets/js/custom.js b/generator/_assets/js/custom.js new file mode 100644 index 000000000..2ec8651ad --- /dev/null +++ b/generator/_assets/js/custom.js @@ -0,0 +1,143 @@ +'use strict'; +var is_mobile = true; +$(document).ready(function() { + if ($(window).width() > 800) + { + is_mobile = false; + } + var scrollPos = 380; + /*show/hide moveTop link */ + var showMoveTop = false; + + if (is_mobile === false) + { + $("#moveTop a").click(function(e) { + $(window).scrollTop($('#nav_wrapper').offset().top); + showMoveTop = false; + $("#moveTop").hide(); + e.preventDefault(); + }); + + $(window).scroll(function() { + if ($(window).scrollTop() > scrollPos && showMoveTop===false) { + $("#moveTop").show(); + showMoveTop = true; + } else if($(window).scrollTop() < scrollPos && showMoveTop === true ) { + $("#moveTop").hide(); + showMoveTop = false; + } + }); + } + + if ($(window).scrollTop() > scrollPos && is_mobile === false) + { + $("#moveTop").show(); + showMoveTop = true; + } + + + var ToC_start = + "
Table of Contents
" + + ""; + + var newLine, el, title, link, elClass, url, ToC=''; + $("article h1, article h2, article h3, article h4").each(function() { + el = $(this); + title = el.text(); + link = "#" + el.attr("id"); + elClass = "link_" + el.prop("tagName").toLowerCase() + url = window.location.pathname; + + // if header has data-behavior= exclude-from-toc - do not include it to the TOC list + if (el.children('center').attr('data-behavior') !=="exclude-from-toc") + { + newLine = + "
  • " + + "" + + title + + "" + + "
    • "; + ToC += newLine; + } + }); + + if (ToC.length) + { + if (is_mobile === false) + { + $("#TOCbox_wrapper").show(); + } + var result = ToC_start + ToC + ToC_End; + + $("#TOCbox_wrapper #TOCbox_list").html(''); + $("#print_TOC").html(''); + + $("#TOCbox_wrapper #TOCbox_list").append(result); + $("#print_TOC").append(result); + } + + // detect print event, display TOC + var beforePrint = function() { + $("#print_TOC_wrapper").show(); + }; + var afterPrint = function() { + $("#print_TOC_wrapper").hide(); + }; + + if (window.matchMedia) { + var mediaQueryList = window.matchMedia('print'); + mediaQueryList.addListener(function(mql) { + if (mql.matches) { + beforePrint(); + } else { + afterPrint(); + } + }); + } + + window.onbeforeprint = beforePrint; + window.onafterprint = afterPrint; + + +}); + +$(document).ready(function() { + $("#TOCbox_wrapper").click(function() { + $("#TOCbox_list").toggle(); + }); + + $(document).click(function(event) { + if(!$(event.target).closest('#TOCbox_wrapper').length) { + if($('#TOCbox_list').is(":visible")) { + $('#TOCbox_list').hide(); + } + } + }); + + $(".article :header").each(function(){ + var url = window.location.href; + url = url.replace(window.location.hash,''); + $(this).prepend(''); + }); + +}); + +document.querySelectorAll('pre').forEach(function (pre) { + pre.closest('div.highlight').innerHTML += ''; +}); + +document.querySelectorAll(".copy-to-clipboard").forEach(function (el) { + el.addEventListener("click", function (event) { + event.preventDefault(); + var target = event.target; + var copyText = target.closest(event.target.dataset.closest).querySelector(event.target.dataset.copyfrom).innerText; + navigator.clipboard.writeText(copyText); + target.classList.remove('bi-clipboard'); + target.className += ' bi-check2 '; + setTimeout(function () { target.className = 'bi bi-clipboard copy-to-clipboard' }, 2000); + }) +}); diff --git a/generator/_assets/js/google_analytics_search.js b/generator/_assets/js/google_analytics_search.js new file mode 100644 index 000000000..9839f6778 --- /dev/null +++ b/generator/_assets/js/google_analytics_search.js @@ -0,0 +1,28 @@ +//google search +(function() { + + var cx = 'c5f68dd2aab4228c4'; + var gcse = document.createElement('script'); + gcse.type = 'text/javascript'; + gcse.async = true; + gcse.src = (document.location.protocol == 'https:' ? 'https:' : 'http:') + + '//www.google.com/cse/cse.js?cx=' + cx; + var s = document.getElementsByTagName('script')[0]; + s.parentNode.insertBefore(gcse, s); + +})(); + +//google analytics +var _gaq = _gaq || []; +_gaq.push(['_setAccount', 'UA-2576171-2']); +_gaq.push(['_setDomainName', 'cfengine.com']); +_gaq.push(['_trackPageview']); + +(function() { + var ga = document.createElement('script'); + ga.type = 'text/javascript'; + ga.async = true; + ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; + var s = document.getElementsByTagName('script')[0]; + s.parentNode.insertBefore(ga, s); +})(); diff --git a/generator/_assets/js/jquery-1.9.1.min.js b/generator/_assets/js/jquery-1.9.1.min.js new file mode 100644 index 000000000..006e95310 --- /dev/null +++ b/generator/_assets/js/jquery-1.9.1.min.js @@ -0,0 +1,5 @@ +/*! jQuery v1.9.1 | (c) 2005, 2012 jQuery Foundation, Inc. | jquery.org/license +//@ sourceMappingURL=jquery.min.map +*/(function(e,t){var n,r,i=typeof t,o=e.document,a=e.location,s=e.jQuery,u=e.$,l={},c=[],p="1.9.1",f=c.concat,d=c.push,h=c.slice,g=c.indexOf,m=l.toString,y=l.hasOwnProperty,v=p.trim,b=function(e,t){return new b.fn.init(e,t,r)},x=/[+-]?(?:\d*\.|)\d+(?:[eE][+-]?\d+|)/.source,w=/\S+/g,T=/^[\s\uFEFF\xA0]+|[\s\uFEFF\xA0]+$/g,N=/^(?:(<[\w\W]+>)[^>]*|#([\w-]*))$/,C=/^<(\w+)\s*\/?>(?:<\/\1>|)$/,k=/^[\],:{}\s]*$/,E=/(?:^|:|,)(?:\s*\[)+/g,S=/\\(?:["\\\/bfnrt]|u[\da-fA-F]{4})/g,A=/"[^"\\\r\n]*"|true|false|null|-?(?:\d+\.|)\d+(?:[eE][+-]?\d+|)/g,j=/^-ms-/,D=/-([\da-z])/gi,L=function(e,t){return t.toUpperCase()},H=function(e){(o.addEventListener||"load"===e.type||"complete"===o.readyState)&&(q(),b.ready())},q=function(){o.addEventListener?(o.removeEventListener("DOMContentLoaded",H,!1),e.removeEventListener("load",H,!1)):(o.detachEvent("onreadystatechange",H),e.detachEvent("onload",H))};b.fn=b.prototype={jquery:p,constructor:b,init:function(e,n,r){var i,a;if(!e)return this;if("string"==typeof e){if(i="<"===e.charAt(0)&&">"===e.charAt(e.length-1)&&e.length>=3?[null,e,null]:N.exec(e),!i||!i[1]&&n)return!n||n.jquery?(n||r).find(e):this.constructor(n).find(e);if(i[1]){if(n=n instanceof b?n[0]:n,b.merge(this,b.parseHTML(i[1],n&&n.nodeType?n.ownerDocument||n:o,!0)),C.test(i[1])&&b.isPlainObject(n))for(i in n)b.isFunction(this[i])?this[i](n[i]):this.attr(i,n[i]);return this}if(a=o.getElementById(i[2]),a&&a.parentNode){if(a.id!==i[2])return r.find(e);this.length=1,this[0]=a}return this.context=o,this.selector=e,this}return e.nodeType?(this.context=this[0]=e,this.length=1,this):b.isFunction(e)?r.ready(e):(e.selector!==t&&(this.selector=e.selector,this.context=e.context),b.makeArray(e,this))},selector:"",length:0,size:function(){return this.length},toArray:function(){return h.call(this)},get:function(e){return null==e?this.toArray():0>e?this[this.length+e]:this[e]},pushStack:function(e){var t=b.merge(this.constructor(),e);return t.prevObject=this,t.context=this.context,t},each:function(e,t){return b.each(this,e,t)},ready:function(e){return b.ready.promise().done(e),this},slice:function(){return this.pushStack(h.apply(this,arguments))},first:function(){return this.eq(0)},last:function(){return this.eq(-1)},eq:function(e){var t=this.length,n=+e+(0>e?t:0);return this.pushStack(n>=0&&t>n?[this[n]]:[])},map:function(e){return this.pushStack(b.map(this,function(t,n){return e.call(t,n,t)}))},end:function(){return this.prevObject||this.constructor(null)},push:d,sort:[].sort,splice:[].splice},b.fn.init.prototype=b.fn,b.extend=b.fn.extend=function(){var e,n,r,i,o,a,s=arguments[0]||{},u=1,l=arguments.length,c=!1;for("boolean"==typeof s&&(c=s,s=arguments[1]||{},u=2),"object"==typeof s||b.isFunction(s)||(s={}),l===u&&(s=this,--u);l>u;u++)if(null!=(o=arguments[u]))for(i in o)e=s[i],r=o[i],s!==r&&(c&&r&&(b.isPlainObject(r)||(n=b.isArray(r)))?(n?(n=!1,a=e&&b.isArray(e)?e:[]):a=e&&b.isPlainObject(e)?e:{},s[i]=b.extend(c,a,r)):r!==t&&(s[i]=r));return s},b.extend({noConflict:function(t){return e.$===b&&(e.$=u),t&&e.jQuery===b&&(e.jQuery=s),b},isReady:!1,readyWait:1,holdReady:function(e){e?b.readyWait++:b.ready(!0)},ready:function(e){if(e===!0?!--b.readyWait:!b.isReady){if(!o.body)return setTimeout(b.ready);b.isReady=!0,e!==!0&&--b.readyWait>0||(n.resolveWith(o,[b]),b.fn.trigger&&b(o).trigger("ready").off("ready"))}},isFunction:function(e){return"function"===b.type(e)},isArray:Array.isArray||function(e){return"array"===b.type(e)},isWindow:function(e){return null!=e&&e==e.window},isNumeric:function(e){return!isNaN(parseFloat(e))&&isFinite(e)},type:function(e){return null==e?e+"":"object"==typeof e||"function"==typeof e?l[m.call(e)]||"object":typeof e},isPlainObject:function(e){if(!e||"object"!==b.type(e)||e.nodeType||b.isWindow(e))return!1;try{if(e.constructor&&!y.call(e,"constructor")&&!y.call(e.constructor.prototype,"isPrototypeOf"))return!1}catch(n){return!1}var r;for(r in e);return r===t||y.call(e,r)},isEmptyObject:function(e){var t;for(t in e)return!1;return!0},error:function(e){throw Error(e)},parseHTML:function(e,t,n){if(!e||"string"!=typeof e)return null;"boolean"==typeof t&&(n=t,t=!1),t=t||o;var r=C.exec(e),i=!n&&[];return r?[t.createElement(r[1])]:(r=b.buildFragment([e],t,i),i&&b(i).remove(),b.merge([],r.childNodes))},parseJSON:function(n){return e.JSON&&e.JSON.parse?e.JSON.parse(n):null===n?n:"string"==typeof n&&(n=b.trim(n),n&&k.test(n.replace(S,"@").replace(A,"]").replace(E,"")))?Function("return "+n)():(b.error("Invalid JSON: "+n),t)},parseXML:function(n){var r,i;if(!n||"string"!=typeof n)return null;try{e.DOMParser?(i=new DOMParser,r=i.parseFromString(n,"text/xml")):(r=new ActiveXObject("Microsoft.XMLDOM"),r.async="false",r.loadXML(n))}catch(o){r=t}return r&&r.documentElement&&!r.getElementsByTagName("parsererror").length||b.error("Invalid XML: "+n),r},noop:function(){},globalEval:function(t){t&&b.trim(t)&&(e.execScript||function(t){e.eval.call(e,t)})(t)},camelCase:function(e){return e.replace(j,"ms-").replace(D,L)},nodeName:function(e,t){return e.nodeName&&e.nodeName.toLowerCase()===t.toLowerCase()},each:function(e,t,n){var r,i=0,o=e.length,a=M(e);if(n){if(a){for(;o>i;i++)if(r=t.apply(e[i],n),r===!1)break}else for(i in e)if(r=t.apply(e[i],n),r===!1)break}else if(a){for(;o>i;i++)if(r=t.call(e[i],i,e[i]),r===!1)break}else for(i in e)if(r=t.call(e[i],i,e[i]),r===!1)break;return e},trim:v&&!v.call("\ufeff\u00a0")?function(e){return null==e?"":v.call(e)}:function(e){return null==e?"":(e+"").replace(T,"")},makeArray:function(e,t){var n=t||[];return null!=e&&(M(Object(e))?b.merge(n,"string"==typeof e?[e]:e):d.call(n,e)),n},inArray:function(e,t,n){var r;if(t){if(g)return g.call(t,e,n);for(r=t.length,n=n?0>n?Math.max(0,r+n):n:0;r>n;n++)if(n in t&&t[n]===e)return n}return-1},merge:function(e,n){var r=n.length,i=e.length,o=0;if("number"==typeof r)for(;r>o;o++)e[i++]=n[o];else while(n[o]!==t)e[i++]=n[o++];return e.length=i,e},grep:function(e,t,n){var r,i=[],o=0,a=e.length;for(n=!!n;a>o;o++)r=!!t(e[o],o),n!==r&&i.push(e[o]);return i},map:function(e,t,n){var r,i=0,o=e.length,a=M(e),s=[];if(a)for(;o>i;i++)r=t(e[i],i,n),null!=r&&(s[s.length]=r);else for(i in e)r=t(e[i],i,n),null!=r&&(s[s.length]=r);return f.apply([],s)},guid:1,proxy:function(e,n){var r,i,o;return"string"==typeof n&&(o=e[n],n=e,e=o),b.isFunction(e)?(r=h.call(arguments,2),i=function(){return e.apply(n||this,r.concat(h.call(arguments)))},i.guid=e.guid=e.guid||b.guid++,i):t},access:function(e,n,r,i,o,a,s){var u=0,l=e.length,c=null==r;if("object"===b.type(r)){o=!0;for(u in r)b.access(e,n,u,r[u],!0,a,s)}else if(i!==t&&(o=!0,b.isFunction(i)||(s=!0),c&&(s?(n.call(e,i),n=null):(c=n,n=function(e,t,n){return c.call(b(e),n)})),n))for(;l>u;u++)n(e[u],r,s?i:i.call(e[u],u,n(e[u],r)));return o?e:c?n.call(e):l?n(e[0],r):a},now:function(){return(new Date).getTime()}}),b.ready.promise=function(t){if(!n)if(n=b.Deferred(),"complete"===o.readyState)setTimeout(b.ready);else if(o.addEventListener)o.addEventListener("DOMContentLoaded",H,!1),e.addEventListener("load",H,!1);else{o.attachEvent("onreadystatechange",H),e.attachEvent("onload",H);var r=!1;try{r=null==e.frameElement&&o.documentElement}catch(i){}r&&r.doScroll&&function a(){if(!b.isReady){try{r.doScroll("left")}catch(e){return setTimeout(a,50)}q(),b.ready()}}()}return n.promise(t)},b.each("Boolean Number String Function Array Date RegExp Object Error".split(" "),function(e,t){l["[object "+t+"]"]=t.toLowerCase()});function M(e){var t=e.length,n=b.type(e);return b.isWindow(e)?!1:1===e.nodeType&&t?!0:"array"===n||"function"!==n&&(0===t||"number"==typeof t&&t>0&&t-1 in e)}r=b(o);var _={};function F(e){var t=_[e]={};return b.each(e.match(w)||[],function(e,n){t[n]=!0}),t}b.Callbacks=function(e){e="string"==typeof e?_[e]||F(e):b.extend({},e);var n,r,i,o,a,s,u=[],l=!e.once&&[],c=function(t){for(r=e.memory&&t,i=!0,a=s||0,s=0,o=u.length,n=!0;u&&o>a;a++)if(u[a].apply(t[0],t[1])===!1&&e.stopOnFalse){r=!1;break}n=!1,u&&(l?l.length&&c(l.shift()):r?u=[]:p.disable())},p={add:function(){if(u){var t=u.length;(function i(t){b.each(t,function(t,n){var r=b.type(n);"function"===r?e.unique&&p.has(n)||u.push(n):n&&n.length&&"string"!==r&&i(n)})})(arguments),n?o=u.length:r&&(s=t,c(r))}return this},remove:function(){return u&&b.each(arguments,function(e,t){var r;while((r=b.inArray(t,u,r))>-1)u.splice(r,1),n&&(o>=r&&o--,a>=r&&a--)}),this},has:function(e){return e?b.inArray(e,u)>-1:!(!u||!u.length)},empty:function(){return u=[],this},disable:function(){return u=l=r=t,this},disabled:function(){return!u},lock:function(){return l=t,r||p.disable(),this},locked:function(){return!l},fireWith:function(e,t){return t=t||[],t=[e,t.slice?t.slice():t],!u||i&&!l||(n?l.push(t):c(t)),this},fire:function(){return p.fireWith(this,arguments),this},fired:function(){return!!i}};return p},b.extend({Deferred:function(e){var t=[["resolve","done",b.Callbacks("once memory"),"resolved"],["reject","fail",b.Callbacks("once memory"),"rejected"],["notify","progress",b.Callbacks("memory")]],n="pending",r={state:function(){return n},always:function(){return i.done(arguments).fail(arguments),this},then:function(){var e=arguments;return b.Deferred(function(n){b.each(t,function(t,o){var a=o[0],s=b.isFunction(e[t])&&e[t];i[o[1]](function(){var e=s&&s.apply(this,arguments);e&&b.isFunction(e.promise)?e.promise().done(n.resolve).fail(n.reject).progress(n.notify):n[a+"With"](this===r?n.promise():this,s?[e]:arguments)})}),e=null}).promise()},promise:function(e){return null!=e?b.extend(e,r):r}},i={};return r.pipe=r.then,b.each(t,function(e,o){var a=o[2],s=o[3];r[o[1]]=a.add,s&&a.add(function(){n=s},t[1^e][2].disable,t[2][2].lock),i[o[0]]=function(){return i[o[0]+"With"](this===i?r:this,arguments),this},i[o[0]+"With"]=a.fireWith}),r.promise(i),e&&e.call(i,i),i},when:function(e){var t=0,n=h.call(arguments),r=n.length,i=1!==r||e&&b.isFunction(e.promise)?r:0,o=1===i?e:b.Deferred(),a=function(e,t,n){return function(r){t[e]=this,n[e]=arguments.length>1?h.call(arguments):r,n===s?o.notifyWith(t,n):--i||o.resolveWith(t,n)}},s,u,l;if(r>1)for(s=Array(r),u=Array(r),l=Array(r);r>t;t++)n[t]&&b.isFunction(n[t].promise)?n[t].promise().done(a(t,l,n)).fail(o.reject).progress(a(t,u,s)):--i;return i||o.resolveWith(l,n),o.promise()}}),b.support=function(){var t,n,r,a,s,u,l,c,p,f,d=o.createElement("div");if(d.setAttribute("className","t"),d.innerHTML="
    a",n=d.getElementsByTagName("*"),r=d.getElementsByTagName("a")[0],!n||!r||!n.length)return{};s=o.createElement("select"),l=s.appendChild(o.createElement("option")),a=d.getElementsByTagName("input")[0],r.style.cssText="top:1px;float:left;opacity:.5",t={getSetAttribute:"t"!==d.className,leadingWhitespace:3===d.firstChild.nodeType,tbody:!d.getElementsByTagName("tbody").length,htmlSerialize:!!d.getElementsByTagName("link").length,style:/top/.test(r.getAttribute("style")),hrefNormalized:"/a"===r.getAttribute("href"),opacity:/^0.5/.test(r.style.opacity),cssFloat:!!r.style.cssFloat,checkOn:!!a.value,optSelected:l.selected,enctype:!!o.createElement("form").enctype,html5Clone:"<:nav>"!==o.createElement("nav").cloneNode(!0).outerHTML,boxModel:"CSS1Compat"===o.compatMode,deleteExpando:!0,noCloneEvent:!0,inlineBlockNeedsLayout:!1,shrinkWrapBlocks:!1,reliableMarginRight:!0,boxSizingReliable:!0,pixelPosition:!1},a.checked=!0,t.noCloneChecked=a.cloneNode(!0).checked,s.disabled=!0,t.optDisabled=!l.disabled;try{delete d.test}catch(h){t.deleteExpando=!1}a=o.createElement("input"),a.setAttribute("value",""),t.input=""===a.getAttribute("value"),a.value="t",a.setAttribute("type","radio"),t.radioValue="t"===a.value,a.setAttribute("checked","t"),a.setAttribute("name","t"),u=o.createDocumentFragment(),u.appendChild(a),t.appendChecked=a.checked,t.checkClone=u.cloneNode(!0).cloneNode(!0).lastChild.checked,d.attachEvent&&(d.attachEvent("onclick",function(){t.noCloneEvent=!1}),d.cloneNode(!0).click());for(f in{submit:!0,change:!0,focusin:!0})d.setAttribute(c="on"+f,"t"),t[f+"Bubbles"]=c in e||d.attributes[c].expando===!1;return d.style.backgroundClip="content-box",d.cloneNode(!0).style.backgroundClip="",t.clearCloneStyle="content-box"===d.style.backgroundClip,b(function(){var n,r,a,s="padding:0;margin:0;border:0;display:block;box-sizing:content-box;-moz-box-sizing:content-box;-webkit-box-sizing:content-box;",u=o.getElementsByTagName("body")[0];u&&(n=o.createElement("div"),n.style.cssText="border:0;width:0;height:0;position:absolute;top:0;left:-9999px;margin-top:1px",u.appendChild(n).appendChild(d),d.innerHTML="
    t
    ",a=d.getElementsByTagName("td"),a[0].style.cssText="padding:0;margin:0;border:0;display:none",p=0===a[0].offsetHeight,a[0].style.display="",a[1].style.display="none",t.reliableHiddenOffsets=p&&0===a[0].offsetHeight,d.innerHTML="",d.style.cssText="box-sizing:border-box;-moz-box-sizing:border-box;-webkit-box-sizing:border-box;padding:1px;border:1px;display:block;width:4px;margin-top:1%;position:absolute;top:1%;",t.boxSizing=4===d.offsetWidth,t.doesNotIncludeMarginInBodyOffset=1!==u.offsetTop,e.getComputedStyle&&(t.pixelPosition="1%"!==(e.getComputedStyle(d,null)||{}).top,t.boxSizingReliable="4px"===(e.getComputedStyle(d,null)||{width:"4px"}).width,r=d.appendChild(o.createElement("div")),r.style.cssText=d.style.cssText=s,r.style.marginRight=r.style.width="0",d.style.width="1px",t.reliableMarginRight=!parseFloat((e.getComputedStyle(r,null)||{}).marginRight)),typeof d.style.zoom!==i&&(d.innerHTML="",d.style.cssText=s+"width:1px;padding:1px;display:inline;zoom:1",t.inlineBlockNeedsLayout=3===d.offsetWidth,d.style.display="block",d.innerHTML="
    ",d.firstChild.style.width="5px",t.shrinkWrapBlocks=3!==d.offsetWidth,t.inlineBlockNeedsLayout&&(u.style.zoom=1)),u.removeChild(n),n=d=a=r=null)}),n=s=u=l=r=a=null,t}();var O=/(?:\{[\s\S]*\}|\[[\s\S]*\])$/,B=/([A-Z])/g;function P(e,n,r,i){if(b.acceptData(e)){var o,a,s=b.expando,u="string"==typeof n,l=e.nodeType,p=l?b.cache:e,f=l?e[s]:e[s]&&s;if(f&&p[f]&&(i||p[f].data)||!u||r!==t)return f||(l?e[s]=f=c.pop()||b.guid++:f=s),p[f]||(p[f]={},l||(p[f].toJSON=b.noop)),("object"==typeof n||"function"==typeof n)&&(i?p[f]=b.extend(p[f],n):p[f].data=b.extend(p[f].data,n)),o=p[f],i||(o.data||(o.data={}),o=o.data),r!==t&&(o[b.camelCase(n)]=r),u?(a=o[n],null==a&&(a=o[b.camelCase(n)])):a=o,a}}function R(e,t,n){if(b.acceptData(e)){var r,i,o,a=e.nodeType,s=a?b.cache:e,u=a?e[b.expando]:b.expando;if(s[u]){if(t&&(o=n?s[u]:s[u].data)){b.isArray(t)?t=t.concat(b.map(t,b.camelCase)):t in o?t=[t]:(t=b.camelCase(t),t=t in o?[t]:t.split(" "));for(r=0,i=t.length;i>r;r++)delete o[t[r]];if(!(n?$:b.isEmptyObject)(o))return}(n||(delete s[u].data,$(s[u])))&&(a?b.cleanData([e],!0):b.support.deleteExpando||s!=s.window?delete s[u]:s[u]=null)}}}b.extend({cache:{},expando:"jQuery"+(p+Math.random()).replace(/\D/g,""),noData:{embed:!0,object:"clsid:D27CDB6E-AE6D-11cf-96B8-444553540000",applet:!0},hasData:function(e){return e=e.nodeType?b.cache[e[b.expando]]:e[b.expando],!!e&&!$(e)},data:function(e,t,n){return P(e,t,n)},removeData:function(e,t){return R(e,t)},_data:function(e,t,n){return P(e,t,n,!0)},_removeData:function(e,t){return R(e,t,!0)},acceptData:function(e){if(e.nodeType&&1!==e.nodeType&&9!==e.nodeType)return!1;var t=e.nodeName&&b.noData[e.nodeName.toLowerCase()];return!t||t!==!0&&e.getAttribute("classid")===t}}),b.fn.extend({data:function(e,n){var r,i,o=this[0],a=0,s=null;if(e===t){if(this.length&&(s=b.data(o),1===o.nodeType&&!b._data(o,"parsedAttrs"))){for(r=o.attributes;r.length>a;a++)i=r[a].name,i.indexOf("data-")||(i=b.camelCase(i.slice(5)),W(o,i,s[i]));b._data(o,"parsedAttrs",!0)}return s}return"object"==typeof e?this.each(function(){b.data(this,e)}):b.access(this,function(n){return n===t?o?W(o,e,b.data(o,e)):null:(this.each(function(){b.data(this,e,n)}),t)},null,n,arguments.length>1,null,!0)},removeData:function(e){return this.each(function(){b.removeData(this,e)})}});function W(e,n,r){if(r===t&&1===e.nodeType){var i="data-"+n.replace(B,"-$1").toLowerCase();if(r=e.getAttribute(i),"string"==typeof r){try{r="true"===r?!0:"false"===r?!1:"null"===r?null:+r+""===r?+r:O.test(r)?b.parseJSON(r):r}catch(o){}b.data(e,n,r)}else r=t}return r}function $(e){var t;for(t in e)if(("data"!==t||!b.isEmptyObject(e[t]))&&"toJSON"!==t)return!1;return!0}b.extend({queue:function(e,n,r){var i;return e?(n=(n||"fx")+"queue",i=b._data(e,n),r&&(!i||b.isArray(r)?i=b._data(e,n,b.makeArray(r)):i.push(r)),i||[]):t},dequeue:function(e,t){t=t||"fx";var n=b.queue(e,t),r=n.length,i=n.shift(),o=b._queueHooks(e,t),a=function(){b.dequeue(e,t)};"inprogress"===i&&(i=n.shift(),r--),o.cur=i,i&&("fx"===t&&n.unshift("inprogress"),delete o.stop,i.call(e,a,o)),!r&&o&&o.empty.fire()},_queueHooks:function(e,t){var n=t+"queueHooks";return b._data(e,n)||b._data(e,n,{empty:b.Callbacks("once memory").add(function(){b._removeData(e,t+"queue"),b._removeData(e,n)})})}}),b.fn.extend({queue:function(e,n){var r=2;return"string"!=typeof e&&(n=e,e="fx",r--),r>arguments.length?b.queue(this[0],e):n===t?this:this.each(function(){var t=b.queue(this,e,n);b._queueHooks(this,e),"fx"===e&&"inprogress"!==t[0]&&b.dequeue(this,e)})},dequeue:function(e){return this.each(function(){b.dequeue(this,e)})},delay:function(e,t){return e=b.fx?b.fx.speeds[e]||e:e,t=t||"fx",this.queue(t,function(t,n){var r=setTimeout(t,e);n.stop=function(){clearTimeout(r)}})},clearQueue:function(e){return this.queue(e||"fx",[])},promise:function(e,n){var r,i=1,o=b.Deferred(),a=this,s=this.length,u=function(){--i||o.resolveWith(a,[a])};"string"!=typeof e&&(n=e,e=t),e=e||"fx";while(s--)r=b._data(a[s],e+"queueHooks"),r&&r.empty&&(i++,r.empty.add(u));return u(),o.promise(n)}});var I,z,X=/[\t\r\n]/g,U=/\r/g,V=/^(?:input|select|textarea|button|object)$/i,Y=/^(?:a|area)$/i,J=/^(?:checked|selected|autofocus|autoplay|async|controls|defer|disabled|hidden|loop|multiple|open|readonly|required|scoped)$/i,G=/^(?:checked|selected)$/i,Q=b.support.getSetAttribute,K=b.support.input;b.fn.extend({attr:function(e,t){return b.access(this,b.attr,e,t,arguments.length>1)},removeAttr:function(e){return this.each(function(){b.removeAttr(this,e)})},prop:function(e,t){return b.access(this,b.prop,e,t,arguments.length>1)},removeProp:function(e){return e=b.propFix[e]||e,this.each(function(){try{this[e]=t,delete this[e]}catch(n){}})},addClass:function(e){var t,n,r,i,o,a=0,s=this.length,u="string"==typeof e&&e;if(b.isFunction(e))return this.each(function(t){b(this).addClass(e.call(this,t,this.className))});if(u)for(t=(e||"").match(w)||[];s>a;a++)if(n=this[a],r=1===n.nodeType&&(n.className?(" "+n.className+" ").replace(X," "):" ")){o=0;while(i=t[o++])0>r.indexOf(" "+i+" ")&&(r+=i+" ");n.className=b.trim(r)}return this},removeClass:function(e){var t,n,r,i,o,a=0,s=this.length,u=0===arguments.length||"string"==typeof e&&e;if(b.isFunction(e))return this.each(function(t){b(this).removeClass(e.call(this,t,this.className))});if(u)for(t=(e||"").match(w)||[];s>a;a++)if(n=this[a],r=1===n.nodeType&&(n.className?(" "+n.className+" ").replace(X," "):"")){o=0;while(i=t[o++])while(r.indexOf(" "+i+" ")>=0)r=r.replace(" "+i+" "," ");n.className=e?b.trim(r):""}return this},toggleClass:function(e,t){var n=typeof e,r="boolean"==typeof t;return b.isFunction(e)?this.each(function(n){b(this).toggleClass(e.call(this,n,this.className,t),t)}):this.each(function(){if("string"===n){var o,a=0,s=b(this),u=t,l=e.match(w)||[];while(o=l[a++])u=r?u:!s.hasClass(o),s[u?"addClass":"removeClass"](o)}else(n===i||"boolean"===n)&&(this.className&&b._data(this,"__className__",this.className),this.className=this.className||e===!1?"":b._data(this,"__className__")||"")})},hasClass:function(e){var t=" "+e+" ",n=0,r=this.length;for(;r>n;n++)if(1===this[n].nodeType&&(" "+this[n].className+" ").replace(X," ").indexOf(t)>=0)return!0;return!1},val:function(e){var n,r,i,o=this[0];{if(arguments.length)return i=b.isFunction(e),this.each(function(n){var o,a=b(this);1===this.nodeType&&(o=i?e.call(this,n,a.val()):e,null==o?o="":"number"==typeof o?o+="":b.isArray(o)&&(o=b.map(o,function(e){return null==e?"":e+""})),r=b.valHooks[this.type]||b.valHooks[this.nodeName.toLowerCase()],r&&"set"in r&&r.set(this,o,"value")!==t||(this.value=o))});if(o)return r=b.valHooks[o.type]||b.valHooks[o.nodeName.toLowerCase()],r&&"get"in r&&(n=r.get(o,"value"))!==t?n:(n=o.value,"string"==typeof n?n.replace(U,""):null==n?"":n)}}}),b.extend({valHooks:{option:{get:function(e){var t=e.attributes.value;return!t||t.specified?e.value:e.text}},select:{get:function(e){var t,n,r=e.options,i=e.selectedIndex,o="select-one"===e.type||0>i,a=o?null:[],s=o?i+1:r.length,u=0>i?s:o?i:0;for(;s>u;u++)if(n=r[u],!(!n.selected&&u!==i||(b.support.optDisabled?n.disabled:null!==n.getAttribute("disabled"))||n.parentNode.disabled&&b.nodeName(n.parentNode,"optgroup"))){if(t=b(n).val(),o)return t;a.push(t)}return a},set:function(e,t){var n=b.makeArray(t);return b(e).find("option").each(function(){this.selected=b.inArray(b(this).val(),n)>=0}),n.length||(e.selectedIndex=-1),n}}},attr:function(e,n,r){var o,a,s,u=e.nodeType;if(e&&3!==u&&8!==u&&2!==u)return typeof e.getAttribute===i?b.prop(e,n,r):(a=1!==u||!b.isXMLDoc(e),a&&(n=n.toLowerCase(),o=b.attrHooks[n]||(J.test(n)?z:I)),r===t?o&&a&&"get"in o&&null!==(s=o.get(e,n))?s:(typeof e.getAttribute!==i&&(s=e.getAttribute(n)),null==s?t:s):null!==r?o&&a&&"set"in o&&(s=o.set(e,r,n))!==t?s:(e.setAttribute(n,r+""),r):(b.removeAttr(e,n),t))},removeAttr:function(e,t){var n,r,i=0,o=t&&t.match(w);if(o&&1===e.nodeType)while(n=o[i++])r=b.propFix[n]||n,J.test(n)?!Q&&G.test(n)?e[b.camelCase("default-"+n)]=e[r]=!1:e[r]=!1:b.attr(e,n,""),e.removeAttribute(Q?n:r)},attrHooks:{type:{set:function(e,t){if(!b.support.radioValue&&"radio"===t&&b.nodeName(e,"input")){var n=e.value;return e.setAttribute("type",t),n&&(e.value=n),t}}}},propFix:{tabindex:"tabIndex",readonly:"readOnly","for":"htmlFor","class":"className",maxlength:"maxLength",cellspacing:"cellSpacing",cellpadding:"cellPadding",rowspan:"rowSpan",colspan:"colSpan",usemap:"useMap",frameborder:"frameBorder",contenteditable:"contentEditable"},prop:function(e,n,r){var i,o,a,s=e.nodeType;if(e&&3!==s&&8!==s&&2!==s)return a=1!==s||!b.isXMLDoc(e),a&&(n=b.propFix[n]||n,o=b.propHooks[n]),r!==t?o&&"set"in o&&(i=o.set(e,r,n))!==t?i:e[n]=r:o&&"get"in o&&null!==(i=o.get(e,n))?i:e[n]},propHooks:{tabIndex:{get:function(e){var n=e.getAttributeNode("tabindex");return n&&n.specified?parseInt(n.value,10):V.test(e.nodeName)||Y.test(e.nodeName)&&e.href?0:t}}}}),z={get:function(e,n){var r=b.prop(e,n),i="boolean"==typeof r&&e.getAttribute(n),o="boolean"==typeof r?K&&Q?null!=i:G.test(n)?e[b.camelCase("default-"+n)]:!!i:e.getAttributeNode(n);return o&&o.value!==!1?n.toLowerCase():t},set:function(e,t,n){return t===!1?b.removeAttr(e,n):K&&Q||!G.test(n)?e.setAttribute(!Q&&b.propFix[n]||n,n):e[b.camelCase("default-"+n)]=e[n]=!0,n}},K&&Q||(b.attrHooks.value={get:function(e,n){var r=e.getAttributeNode(n);return b.nodeName(e,"input")?e.defaultValue:r&&r.specified?r.value:t},set:function(e,n,r){return b.nodeName(e,"input")?(e.defaultValue=n,t):I&&I.set(e,n,r)}}),Q||(I=b.valHooks.button={get:function(e,n){var r=e.getAttributeNode(n);return r&&("id"===n||"name"===n||"coords"===n?""!==r.value:r.specified)?r.value:t},set:function(e,n,r){var i=e.getAttributeNode(r);return i||e.setAttributeNode(i=e.ownerDocument.createAttribute(r)),i.value=n+="","value"===r||n===e.getAttribute(r)?n:t}},b.attrHooks.contenteditable={get:I.get,set:function(e,t,n){I.set(e,""===t?!1:t,n)}},b.each(["width","height"],function(e,n){b.attrHooks[n]=b.extend(b.attrHooks[n],{set:function(e,r){return""===r?(e.setAttribute(n,"auto"),r):t}})})),b.support.hrefNormalized||(b.each(["href","src","width","height"],function(e,n){b.attrHooks[n]=b.extend(b.attrHooks[n],{get:function(e){var r=e.getAttribute(n,2);return null==r?t:r}})}),b.each(["href","src"],function(e,t){b.propHooks[t]={get:function(e){return e.getAttribute(t,4)}}})),b.support.style||(b.attrHooks.style={get:function(e){return e.style.cssText||t},set:function(e,t){return e.style.cssText=t+""}}),b.support.optSelected||(b.propHooks.selected=b.extend(b.propHooks.selected,{get:function(e){var t=e.parentNode;return t&&(t.selectedIndex,t.parentNode&&t.parentNode.selectedIndex),null}})),b.support.enctype||(b.propFix.enctype="encoding"),b.support.checkOn||b.each(["radio","checkbox"],function(){b.valHooks[this]={get:function(e){return null===e.getAttribute("value")?"on":e.value}}}),b.each(["radio","checkbox"],function(){b.valHooks[this]=b.extend(b.valHooks[this],{set:function(e,n){return b.isArray(n)?e.checked=b.inArray(b(e).val(),n)>=0:t}})});var Z=/^(?:input|select|textarea)$/i,et=/^key/,tt=/^(?:mouse|contextmenu)|click/,nt=/^(?:focusinfocus|focusoutblur)$/,rt=/^([^.]*)(?:\.(.+)|)$/;function it(){return!0}function ot(){return!1}b.event={global:{},add:function(e,n,r,o,a){var s,u,l,c,p,f,d,h,g,m,y,v=b._data(e);if(v){r.handler&&(c=r,r=c.handler,a=c.selector),r.guid||(r.guid=b.guid++),(u=v.events)||(u=v.events={}),(f=v.handle)||(f=v.handle=function(e){return typeof b===i||e&&b.event.triggered===e.type?t:b.event.dispatch.apply(f.elem,arguments)},f.elem=e),n=(n||"").match(w)||[""],l=n.length;while(l--)s=rt.exec(n[l])||[],g=y=s[1],m=(s[2]||"").split(".").sort(),p=b.event.special[g]||{},g=(a?p.delegateType:p.bindType)||g,p=b.event.special[g]||{},d=b.extend({type:g,origType:y,data:o,handler:r,guid:r.guid,selector:a,needsContext:a&&b.expr.match.needsContext.test(a),namespace:m.join(".")},c),(h=u[g])||(h=u[g]=[],h.delegateCount=0,p.setup&&p.setup.call(e,o,m,f)!==!1||(e.addEventListener?e.addEventListener(g,f,!1):e.attachEvent&&e.attachEvent("on"+g,f))),p.add&&(p.add.call(e,d),d.handler.guid||(d.handler.guid=r.guid)),a?h.splice(h.delegateCount++,0,d):h.push(d),b.event.global[g]=!0;e=null}},remove:function(e,t,n,r,i){var o,a,s,u,l,c,p,f,d,h,g,m=b.hasData(e)&&b._data(e);if(m&&(c=m.events)){t=(t||"").match(w)||[""],l=t.length;while(l--)if(s=rt.exec(t[l])||[],d=g=s[1],h=(s[2]||"").split(".").sort(),d){p=b.event.special[d]||{},d=(r?p.delegateType:p.bindType)||d,f=c[d]||[],s=s[2]&&RegExp("(^|\\.)"+h.join("\\.(?:.*\\.|)")+"(\\.|$)"),u=o=f.length;while(o--)a=f[o],!i&&g!==a.origType||n&&n.guid!==a.guid||s&&!s.test(a.namespace)||r&&r!==a.selector&&("**"!==r||!a.selector)||(f.splice(o,1),a.selector&&f.delegateCount--,p.remove&&p.remove.call(e,a));u&&!f.length&&(p.teardown&&p.teardown.call(e,h,m.handle)!==!1||b.removeEvent(e,d,m.handle),delete c[d])}else for(d in c)b.event.remove(e,d+t[l],n,r,!0);b.isEmptyObject(c)&&(delete m.handle,b._removeData(e,"events"))}},trigger:function(n,r,i,a){var s,u,l,c,p,f,d,h=[i||o],g=y.call(n,"type")?n.type:n,m=y.call(n,"namespace")?n.namespace.split("."):[];if(l=f=i=i||o,3!==i.nodeType&&8!==i.nodeType&&!nt.test(g+b.event.triggered)&&(g.indexOf(".")>=0&&(m=g.split("."),g=m.shift(),m.sort()),u=0>g.indexOf(":")&&"on"+g,n=n[b.expando]?n:new b.Event(g,"object"==typeof n&&n),n.isTrigger=!0,n.namespace=m.join("."),n.namespace_re=n.namespace?RegExp("(^|\\.)"+m.join("\\.(?:.*\\.|)")+"(\\.|$)"):null,n.result=t,n.target||(n.target=i),r=null==r?[n]:b.makeArray(r,[n]),p=b.event.special[g]||{},a||!p.trigger||p.trigger.apply(i,r)!==!1)){if(!a&&!p.noBubble&&!b.isWindow(i)){for(c=p.delegateType||g,nt.test(c+g)||(l=l.parentNode);l;l=l.parentNode)h.push(l),f=l;f===(i.ownerDocument||o)&&h.push(f.defaultView||f.parentWindow||e)}d=0;while((l=h[d++])&&!n.isPropagationStopped())n.type=d>1?c:p.bindType||g,s=(b._data(l,"events")||{})[n.type]&&b._data(l,"handle"),s&&s.apply(l,r),s=u&&l[u],s&&b.acceptData(l)&&s.apply&&s.apply(l,r)===!1&&n.preventDefault();if(n.type=g,!(a||n.isDefaultPrevented()||p._default&&p._default.apply(i.ownerDocument,r)!==!1||"click"===g&&b.nodeName(i,"a")||!b.acceptData(i)||!u||!i[g]||b.isWindow(i))){f=i[u],f&&(i[u]=null),b.event.triggered=g;try{i[g]()}catch(v){}b.event.triggered=t,f&&(i[u]=f)}return n.result}},dispatch:function(e){e=b.event.fix(e);var n,r,i,o,a,s=[],u=h.call(arguments),l=(b._data(this,"events")||{})[e.type]||[],c=b.event.special[e.type]||{};if(u[0]=e,e.delegateTarget=this,!c.preDispatch||c.preDispatch.call(this,e)!==!1){s=b.event.handlers.call(this,e,l),n=0;while((o=s[n++])&&!e.isPropagationStopped()){e.currentTarget=o.elem,a=0;while((i=o.handlers[a++])&&!e.isImmediatePropagationStopped())(!e.namespace_re||e.namespace_re.test(i.namespace))&&(e.handleObj=i,e.data=i.data,r=((b.event.special[i.origType]||{}).handle||i.handler).apply(o.elem,u),r!==t&&(e.result=r)===!1&&(e.preventDefault(),e.stopPropagation()))}return c.postDispatch&&c.postDispatch.call(this,e),e.result}},handlers:function(e,n){var r,i,o,a,s=[],u=n.delegateCount,l=e.target;if(u&&l.nodeType&&(!e.button||"click"!==e.type))for(;l!=this;l=l.parentNode||this)if(1===l.nodeType&&(l.disabled!==!0||"click"!==e.type)){for(o=[],a=0;u>a;a++)i=n[a],r=i.selector+" ",o[r]===t&&(o[r]=i.needsContext?b(r,this).index(l)>=0:b.find(r,this,null,[l]).length),o[r]&&o.push(i);o.length&&s.push({elem:l,handlers:o})}return n.length>u&&s.push({elem:this,handlers:n.slice(u)}),s},fix:function(e){if(e[b.expando])return e;var t,n,r,i=e.type,a=e,s=this.fixHooks[i];s||(this.fixHooks[i]=s=tt.test(i)?this.mouseHooks:et.test(i)?this.keyHooks:{}),r=s.props?this.props.concat(s.props):this.props,e=new b.Event(a),t=r.length;while(t--)n=r[t],e[n]=a[n];return e.target||(e.target=a.srcElement||o),3===e.target.nodeType&&(e.target=e.target.parentNode),e.metaKey=!!e.metaKey,s.filter?s.filter(e,a):e},props:"altKey bubbles cancelable ctrlKey currentTarget eventPhase metaKey relatedTarget shiftKey target timeStamp view which".split(" "),fixHooks:{},keyHooks:{props:"char charCode key keyCode".split(" "),filter:function(e,t){return null==e.which&&(e.which=null!=t.charCode?t.charCode:t.keyCode),e}},mouseHooks:{props:"button buttons clientX clientY fromElement offsetX offsetY pageX pageY screenX screenY toElement".split(" "),filter:function(e,n){var r,i,a,s=n.button,u=n.fromElement;return null==e.pageX&&null!=n.clientX&&(i=e.target.ownerDocument||o,a=i.documentElement,r=i.body,e.pageX=n.clientX+(a&&a.scrollLeft||r&&r.scrollLeft||0)-(a&&a.clientLeft||r&&r.clientLeft||0),e.pageY=n.clientY+(a&&a.scrollTop||r&&r.scrollTop||0)-(a&&a.clientTop||r&&r.clientTop||0)),!e.relatedTarget&&u&&(e.relatedTarget=u===e.target?n.toElement:u),e.which||s===t||(e.which=1&s?1:2&s?3:4&s?2:0),e}},special:{load:{noBubble:!0},click:{trigger:function(){return b.nodeName(this,"input")&&"checkbox"===this.type&&this.click?(this.click(),!1):t}},focus:{trigger:function(){if(this!==o.activeElement&&this.focus)try{return this.focus(),!1}catch(e){}},delegateType:"focusin"},blur:{trigger:function(){return this===o.activeElement&&this.blur?(this.blur(),!1):t},delegateType:"focusout"},beforeunload:{postDispatch:function(e){e.result!==t&&(e.originalEvent.returnValue=e.result)}}},simulate:function(e,t,n,r){var i=b.extend(new b.Event,n,{type:e,isSimulated:!0,originalEvent:{}});r?b.event.trigger(i,null,t):b.event.dispatch.call(t,i),i.isDefaultPrevented()&&n.preventDefault()}},b.removeEvent=o.removeEventListener?function(e,t,n){e.removeEventListener&&e.removeEventListener(t,n,!1)}:function(e,t,n){var r="on"+t;e.detachEvent&&(typeof e[r]===i&&(e[r]=null),e.detachEvent(r,n))},b.Event=function(e,n){return this instanceof b.Event?(e&&e.type?(this.originalEvent=e,this.type=e.type,this.isDefaultPrevented=e.defaultPrevented||e.returnValue===!1||e.getPreventDefault&&e.getPreventDefault()?it:ot):this.type=e,n&&b.extend(this,n),this.timeStamp=e&&e.timeStamp||b.now(),this[b.expando]=!0,t):new b.Event(e,n)},b.Event.prototype={isDefaultPrevented:ot,isPropagationStopped:ot,isImmediatePropagationStopped:ot,preventDefault:function(){var e=this.originalEvent;this.isDefaultPrevented=it,e&&(e.preventDefault?e.preventDefault():e.returnValue=!1)},stopPropagation:function(){var e=this.originalEvent;this.isPropagationStopped=it,e&&(e.stopPropagation&&e.stopPropagation(),e.cancelBubble=!0)},stopImmediatePropagation:function(){this.isImmediatePropagationStopped=it,this.stopPropagation()}},b.each({mouseenter:"mouseover",mouseleave:"mouseout"},function(e,t){b.event.special[e]={delegateType:t,bindType:t,handle:function(e){var n,r=this,i=e.relatedTarget,o=e.handleObj; +return(!i||i!==r&&!b.contains(r,i))&&(e.type=o.origType,n=o.handler.apply(this,arguments),e.type=t),n}}}),b.support.submitBubbles||(b.event.special.submit={setup:function(){return b.nodeName(this,"form")?!1:(b.event.add(this,"click._submit keypress._submit",function(e){var n=e.target,r=b.nodeName(n,"input")||b.nodeName(n,"button")?n.form:t;r&&!b._data(r,"submitBubbles")&&(b.event.add(r,"submit._submit",function(e){e._submit_bubble=!0}),b._data(r,"submitBubbles",!0))}),t)},postDispatch:function(e){e._submit_bubble&&(delete e._submit_bubble,this.parentNode&&!e.isTrigger&&b.event.simulate("submit",this.parentNode,e,!0))},teardown:function(){return b.nodeName(this,"form")?!1:(b.event.remove(this,"._submit"),t)}}),b.support.changeBubbles||(b.event.special.change={setup:function(){return Z.test(this.nodeName)?(("checkbox"===this.type||"radio"===this.type)&&(b.event.add(this,"propertychange._change",function(e){"checked"===e.originalEvent.propertyName&&(this._just_changed=!0)}),b.event.add(this,"click._change",function(e){this._just_changed&&!e.isTrigger&&(this._just_changed=!1),b.event.simulate("change",this,e,!0)})),!1):(b.event.add(this,"beforeactivate._change",function(e){var t=e.target;Z.test(t.nodeName)&&!b._data(t,"changeBubbles")&&(b.event.add(t,"change._change",function(e){!this.parentNode||e.isSimulated||e.isTrigger||b.event.simulate("change",this.parentNode,e,!0)}),b._data(t,"changeBubbles",!0))}),t)},handle:function(e){var n=e.target;return this!==n||e.isSimulated||e.isTrigger||"radio"!==n.type&&"checkbox"!==n.type?e.handleObj.handler.apply(this,arguments):t},teardown:function(){return b.event.remove(this,"._change"),!Z.test(this.nodeName)}}),b.support.focusinBubbles||b.each({focus:"focusin",blur:"focusout"},function(e,t){var n=0,r=function(e){b.event.simulate(t,e.target,b.event.fix(e),!0)};b.event.special[t]={setup:function(){0===n++&&o.addEventListener(e,r,!0)},teardown:function(){0===--n&&o.removeEventListener(e,r,!0)}}}),b.fn.extend({on:function(e,n,r,i,o){var a,s;if("object"==typeof e){"string"!=typeof n&&(r=r||n,n=t);for(a in e)this.on(a,n,r,e[a],o);return this}if(null==r&&null==i?(i=n,r=n=t):null==i&&("string"==typeof n?(i=r,r=t):(i=r,r=n,n=t)),i===!1)i=ot;else if(!i)return this;return 1===o&&(s=i,i=function(e){return b().off(e),s.apply(this,arguments)},i.guid=s.guid||(s.guid=b.guid++)),this.each(function(){b.event.add(this,e,i,r,n)})},one:function(e,t,n,r){return this.on(e,t,n,r,1)},off:function(e,n,r){var i,o;if(e&&e.preventDefault&&e.handleObj)return i=e.handleObj,b(e.delegateTarget).off(i.namespace?i.origType+"."+i.namespace:i.origType,i.selector,i.handler),this;if("object"==typeof e){for(o in e)this.off(o,n,e[o]);return this}return(n===!1||"function"==typeof n)&&(r=n,n=t),r===!1&&(r=ot),this.each(function(){b.event.remove(this,e,r,n)})},bind:function(e,t,n){return this.on(e,null,t,n)},unbind:function(e,t){return this.off(e,null,t)},delegate:function(e,t,n,r){return this.on(t,e,n,r)},undelegate:function(e,t,n){return 1===arguments.length?this.off(e,"**"):this.off(t,e||"**",n)},trigger:function(e,t){return this.each(function(){b.event.trigger(e,t,this)})},triggerHandler:function(e,n){var r=this[0];return r?b.event.trigger(e,n,r,!0):t}}),function(e,t){var n,r,i,o,a,s,u,l,c,p,f,d,h,g,m,y,v,x="sizzle"+-new Date,w=e.document,T={},N=0,C=0,k=it(),E=it(),S=it(),A=typeof t,j=1<<31,D=[],L=D.pop,H=D.push,q=D.slice,M=D.indexOf||function(e){var t=0,n=this.length;for(;n>t;t++)if(this[t]===e)return t;return-1},_="[\\x20\\t\\r\\n\\f]",F="(?:\\\\.|[\\w-]|[^\\x00-\\xa0])+",O=F.replace("w","w#"),B="([*^$|!~]?=)",P="\\["+_+"*("+F+")"+_+"*(?:"+B+_+"*(?:(['\"])((?:\\\\.|[^\\\\])*?)\\3|("+O+")|)|)"+_+"*\\]",R=":("+F+")(?:\\(((['\"])((?:\\\\.|[^\\\\])*?)\\3|((?:\\\\.|[^\\\\()[\\]]|"+P.replace(3,8)+")*)|.*)\\)|)",W=RegExp("^"+_+"+|((?:^|[^\\\\])(?:\\\\.)*)"+_+"+$","g"),$=RegExp("^"+_+"*,"+_+"*"),I=RegExp("^"+_+"*([\\x20\\t\\r\\n\\f>+~])"+_+"*"),z=RegExp(R),X=RegExp("^"+O+"$"),U={ID:RegExp("^#("+F+")"),CLASS:RegExp("^\\.("+F+")"),NAME:RegExp("^\\[name=['\"]?("+F+")['\"]?\\]"),TAG:RegExp("^("+F.replace("w","w*")+")"),ATTR:RegExp("^"+P),PSEUDO:RegExp("^"+R),CHILD:RegExp("^:(only|first|last|nth|nth-last)-(child|of-type)(?:\\("+_+"*(even|odd|(([+-]|)(\\d*)n|)"+_+"*(?:([+-]|)"+_+"*(\\d+)|))"+_+"*\\)|)","i"),needsContext:RegExp("^"+_+"*[>+~]|:(even|odd|eq|gt|lt|nth|first|last)(?:\\("+_+"*((?:-\\d)?\\d*)"+_+"*\\)|)(?=[^-]|$)","i")},V=/[\x20\t\r\n\f]*[+~]/,Y=/^[^{]+\{\s*\[native code/,J=/^(?:#([\w-]+)|(\w+)|\.([\w-]+))$/,G=/^(?:input|select|textarea|button)$/i,Q=/^h\d$/i,K=/'|\\/g,Z=/\=[\x20\t\r\n\f]*([^'"\]]*)[\x20\t\r\n\f]*\]/g,et=/\\([\da-fA-F]{1,6}[\x20\t\r\n\f]?|.)/g,tt=function(e,t){var n="0x"+t-65536;return n!==n?t:0>n?String.fromCharCode(n+65536):String.fromCharCode(55296|n>>10,56320|1023&n)};try{q.call(w.documentElement.childNodes,0)[0].nodeType}catch(nt){q=function(e){var t,n=[];while(t=this[e++])n.push(t);return n}}function rt(e){return Y.test(e+"")}function it(){var e,t=[];return e=function(n,r){return t.push(n+=" ")>i.cacheLength&&delete e[t.shift()],e[n]=r}}function ot(e){return e[x]=!0,e}function at(e){var t=p.createElement("div");try{return e(t)}catch(n){return!1}finally{t=null}}function st(e,t,n,r){var i,o,a,s,u,l,f,g,m,v;if((t?t.ownerDocument||t:w)!==p&&c(t),t=t||p,n=n||[],!e||"string"!=typeof e)return n;if(1!==(s=t.nodeType)&&9!==s)return[];if(!d&&!r){if(i=J.exec(e))if(a=i[1]){if(9===s){if(o=t.getElementById(a),!o||!o.parentNode)return n;if(o.id===a)return n.push(o),n}else if(t.ownerDocument&&(o=t.ownerDocument.getElementById(a))&&y(t,o)&&o.id===a)return n.push(o),n}else{if(i[2])return H.apply(n,q.call(t.getElementsByTagName(e),0)),n;if((a=i[3])&&T.getByClassName&&t.getElementsByClassName)return H.apply(n,q.call(t.getElementsByClassName(a),0)),n}if(T.qsa&&!h.test(e)){if(f=!0,g=x,m=t,v=9===s&&e,1===s&&"object"!==t.nodeName.toLowerCase()){l=ft(e),(f=t.getAttribute("id"))?g=f.replace(K,"\\$&"):t.setAttribute("id",g),g="[id='"+g+"'] ",u=l.length;while(u--)l[u]=g+dt(l[u]);m=V.test(e)&&t.parentNode||t,v=l.join(",")}if(v)try{return H.apply(n,q.call(m.querySelectorAll(v),0)),n}catch(b){}finally{f||t.removeAttribute("id")}}}return wt(e.replace(W,"$1"),t,n,r)}a=st.isXML=function(e){var t=e&&(e.ownerDocument||e).documentElement;return t?"HTML"!==t.nodeName:!1},c=st.setDocument=function(e){var n=e?e.ownerDocument||e:w;return n!==p&&9===n.nodeType&&n.documentElement?(p=n,f=n.documentElement,d=a(n),T.tagNameNoComments=at(function(e){return e.appendChild(n.createComment("")),!e.getElementsByTagName("*").length}),T.attributes=at(function(e){e.innerHTML="";var t=typeof e.lastChild.getAttribute("multiple");return"boolean"!==t&&"string"!==t}),T.getByClassName=at(function(e){return e.innerHTML="",e.getElementsByClassName&&e.getElementsByClassName("e").length?(e.lastChild.className="e",2===e.getElementsByClassName("e").length):!1}),T.getByName=at(function(e){e.id=x+0,e.innerHTML="
    ",f.insertBefore(e,f.firstChild);var t=n.getElementsByName&&n.getElementsByName(x).length===2+n.getElementsByName(x+0).length;return T.getIdNotName=!n.getElementById(x),f.removeChild(e),t}),i.attrHandle=at(function(e){return e.innerHTML="",e.firstChild&&typeof e.firstChild.getAttribute!==A&&"#"===e.firstChild.getAttribute("href")})?{}:{href:function(e){return e.getAttribute("href",2)},type:function(e){return e.getAttribute("type")}},T.getIdNotName?(i.find.ID=function(e,t){if(typeof t.getElementById!==A&&!d){var n=t.getElementById(e);return n&&n.parentNode?[n]:[]}},i.filter.ID=function(e){var t=e.replace(et,tt);return function(e){return e.getAttribute("id")===t}}):(i.find.ID=function(e,n){if(typeof n.getElementById!==A&&!d){var r=n.getElementById(e);return r?r.id===e||typeof r.getAttributeNode!==A&&r.getAttributeNode("id").value===e?[r]:t:[]}},i.filter.ID=function(e){var t=e.replace(et,tt);return function(e){var n=typeof e.getAttributeNode!==A&&e.getAttributeNode("id");return n&&n.value===t}}),i.find.TAG=T.tagNameNoComments?function(e,n){return typeof n.getElementsByTagName!==A?n.getElementsByTagName(e):t}:function(e,t){var n,r=[],i=0,o=t.getElementsByTagName(e);if("*"===e){while(n=o[i++])1===n.nodeType&&r.push(n);return r}return o},i.find.NAME=T.getByName&&function(e,n){return typeof n.getElementsByName!==A?n.getElementsByName(name):t},i.find.CLASS=T.getByClassName&&function(e,n){return typeof n.getElementsByClassName===A||d?t:n.getElementsByClassName(e)},g=[],h=[":focus"],(T.qsa=rt(n.querySelectorAll))&&(at(function(e){e.innerHTML="",e.querySelectorAll("[selected]").length||h.push("\\["+_+"*(?:checked|disabled|ismap|multiple|readonly|selected|value)"),e.querySelectorAll(":checked").length||h.push(":checked")}),at(function(e){e.innerHTML="",e.querySelectorAll("[i^='']").length&&h.push("[*^$]="+_+"*(?:\"\"|'')"),e.querySelectorAll(":enabled").length||h.push(":enabled",":disabled"),e.querySelectorAll("*,:x"),h.push(",.*:")})),(T.matchesSelector=rt(m=f.matchesSelector||f.mozMatchesSelector||f.webkitMatchesSelector||f.oMatchesSelector||f.msMatchesSelector))&&at(function(e){T.disconnectedMatch=m.call(e,"div"),m.call(e,"[s!='']:x"),g.push("!=",R)}),h=RegExp(h.join("|")),g=RegExp(g.join("|")),y=rt(f.contains)||f.compareDocumentPosition?function(e,t){var n=9===e.nodeType?e.documentElement:e,r=t&&t.parentNode;return e===r||!(!r||1!==r.nodeType||!(n.contains?n.contains(r):e.compareDocumentPosition&&16&e.compareDocumentPosition(r)))}:function(e,t){if(t)while(t=t.parentNode)if(t===e)return!0;return!1},v=f.compareDocumentPosition?function(e,t){var r;return e===t?(u=!0,0):(r=t.compareDocumentPosition&&e.compareDocumentPosition&&e.compareDocumentPosition(t))?1&r||e.parentNode&&11===e.parentNode.nodeType?e===n||y(w,e)?-1:t===n||y(w,t)?1:0:4&r?-1:1:e.compareDocumentPosition?-1:1}:function(e,t){var r,i=0,o=e.parentNode,a=t.parentNode,s=[e],l=[t];if(e===t)return u=!0,0;if(!o||!a)return e===n?-1:t===n?1:o?-1:a?1:0;if(o===a)return ut(e,t);r=e;while(r=r.parentNode)s.unshift(r);r=t;while(r=r.parentNode)l.unshift(r);while(s[i]===l[i])i++;return i?ut(s[i],l[i]):s[i]===w?-1:l[i]===w?1:0},u=!1,[0,0].sort(v),T.detectDuplicates=u,p):p},st.matches=function(e,t){return st(e,null,null,t)},st.matchesSelector=function(e,t){if((e.ownerDocument||e)!==p&&c(e),t=t.replace(Z,"='$1']"),!(!T.matchesSelector||d||g&&g.test(t)||h.test(t)))try{var n=m.call(e,t);if(n||T.disconnectedMatch||e.document&&11!==e.document.nodeType)return n}catch(r){}return st(t,p,null,[e]).length>0},st.contains=function(e,t){return(e.ownerDocument||e)!==p&&c(e),y(e,t)},st.attr=function(e,t){var n;return(e.ownerDocument||e)!==p&&c(e),d||(t=t.toLowerCase()),(n=i.attrHandle[t])?n(e):d||T.attributes?e.getAttribute(t):((n=e.getAttributeNode(t))||e.getAttribute(t))&&e[t]===!0?t:n&&n.specified?n.value:null},st.error=function(e){throw Error("Syntax error, unrecognized expression: "+e)},st.uniqueSort=function(e){var t,n=[],r=1,i=0;if(u=!T.detectDuplicates,e.sort(v),u){for(;t=e[r];r++)t===e[r-1]&&(i=n.push(r));while(i--)e.splice(n[i],1)}return e};function ut(e,t){var n=t&&e,r=n&&(~t.sourceIndex||j)-(~e.sourceIndex||j);if(r)return r;if(n)while(n=n.nextSibling)if(n===t)return-1;return e?1:-1}function lt(e){return function(t){var n=t.nodeName.toLowerCase();return"input"===n&&t.type===e}}function ct(e){return function(t){var n=t.nodeName.toLowerCase();return("input"===n||"button"===n)&&t.type===e}}function pt(e){return ot(function(t){return t=+t,ot(function(n,r){var i,o=e([],n.length,t),a=o.length;while(a--)n[i=o[a]]&&(n[i]=!(r[i]=n[i]))})})}o=st.getText=function(e){var t,n="",r=0,i=e.nodeType;if(i){if(1===i||9===i||11===i){if("string"==typeof e.textContent)return e.textContent;for(e=e.firstChild;e;e=e.nextSibling)n+=o(e)}else if(3===i||4===i)return e.nodeValue}else for(;t=e[r];r++)n+=o(t);return n},i=st.selectors={cacheLength:50,createPseudo:ot,match:U,find:{},relative:{">":{dir:"parentNode",first:!0}," ":{dir:"parentNode"},"+":{dir:"previousSibling",first:!0},"~":{dir:"previousSibling"}},preFilter:{ATTR:function(e){return e[1]=e[1].replace(et,tt),e[3]=(e[4]||e[5]||"").replace(et,tt),"~="===e[2]&&(e[3]=" "+e[3]+" "),e.slice(0,4)},CHILD:function(e){return e[1]=e[1].toLowerCase(),"nth"===e[1].slice(0,3)?(e[3]||st.error(e[0]),e[4]=+(e[4]?e[5]+(e[6]||1):2*("even"===e[3]||"odd"===e[3])),e[5]=+(e[7]+e[8]||"odd"===e[3])):e[3]&&st.error(e[0]),e},PSEUDO:function(e){var t,n=!e[5]&&e[2];return U.CHILD.test(e[0])?null:(e[4]?e[2]=e[4]:n&&z.test(n)&&(t=ft(n,!0))&&(t=n.indexOf(")",n.length-t)-n.length)&&(e[0]=e[0].slice(0,t),e[2]=n.slice(0,t)),e.slice(0,3))}},filter:{TAG:function(e){return"*"===e?function(){return!0}:(e=e.replace(et,tt).toLowerCase(),function(t){return t.nodeName&&t.nodeName.toLowerCase()===e})},CLASS:function(e){var t=k[e+" "];return t||(t=RegExp("(^|"+_+")"+e+"("+_+"|$)"))&&k(e,function(e){return t.test(e.className||typeof e.getAttribute!==A&&e.getAttribute("class")||"")})},ATTR:function(e,t,n){return function(r){var i=st.attr(r,e);return null==i?"!="===t:t?(i+="","="===t?i===n:"!="===t?i!==n:"^="===t?n&&0===i.indexOf(n):"*="===t?n&&i.indexOf(n)>-1:"$="===t?n&&i.slice(-n.length)===n:"~="===t?(" "+i+" ").indexOf(n)>-1:"|="===t?i===n||i.slice(0,n.length+1)===n+"-":!1):!0}},CHILD:function(e,t,n,r,i){var o="nth"!==e.slice(0,3),a="last"!==e.slice(-4),s="of-type"===t;return 1===r&&0===i?function(e){return!!e.parentNode}:function(t,n,u){var l,c,p,f,d,h,g=o!==a?"nextSibling":"previousSibling",m=t.parentNode,y=s&&t.nodeName.toLowerCase(),v=!u&&!s;if(m){if(o){while(g){p=t;while(p=p[g])if(s?p.nodeName.toLowerCase()===y:1===p.nodeType)return!1;h=g="only"===e&&!h&&"nextSibling"}return!0}if(h=[a?m.firstChild:m.lastChild],a&&v){c=m[x]||(m[x]={}),l=c[e]||[],d=l[0]===N&&l[1],f=l[0]===N&&l[2],p=d&&m.childNodes[d];while(p=++d&&p&&p[g]||(f=d=0)||h.pop())if(1===p.nodeType&&++f&&p===t){c[e]=[N,d,f];break}}else if(v&&(l=(t[x]||(t[x]={}))[e])&&l[0]===N)f=l[1];else while(p=++d&&p&&p[g]||(f=d=0)||h.pop())if((s?p.nodeName.toLowerCase()===y:1===p.nodeType)&&++f&&(v&&((p[x]||(p[x]={}))[e]=[N,f]),p===t))break;return f-=i,f===r||0===f%r&&f/r>=0}}},PSEUDO:function(e,t){var n,r=i.pseudos[e]||i.setFilters[e.toLowerCase()]||st.error("unsupported pseudo: "+e);return r[x]?r(t):r.length>1?(n=[e,e,"",t],i.setFilters.hasOwnProperty(e.toLowerCase())?ot(function(e,n){var i,o=r(e,t),a=o.length;while(a--)i=M.call(e,o[a]),e[i]=!(n[i]=o[a])}):function(e){return r(e,0,n)}):r}},pseudos:{not:ot(function(e){var t=[],n=[],r=s(e.replace(W,"$1"));return r[x]?ot(function(e,t,n,i){var o,a=r(e,null,i,[]),s=e.length;while(s--)(o=a[s])&&(e[s]=!(t[s]=o))}):function(e,i,o){return t[0]=e,r(t,null,o,n),!n.pop()}}),has:ot(function(e){return function(t){return st(e,t).length>0}}),contains:ot(function(e){return function(t){return(t.textContent||t.innerText||o(t)).indexOf(e)>-1}}),lang:ot(function(e){return X.test(e||"")||st.error("unsupported lang: "+e),e=e.replace(et,tt).toLowerCase(),function(t){var n;do if(n=d?t.getAttribute("xml:lang")||t.getAttribute("lang"):t.lang)return n=n.toLowerCase(),n===e||0===n.indexOf(e+"-");while((t=t.parentNode)&&1===t.nodeType);return!1}}),target:function(t){var n=e.location&&e.location.hash;return n&&n.slice(1)===t.id},root:function(e){return e===f},focus:function(e){return e===p.activeElement&&(!p.hasFocus||p.hasFocus())&&!!(e.type||e.href||~e.tabIndex)},enabled:function(e){return e.disabled===!1},disabled:function(e){return e.disabled===!0},checked:function(e){var t=e.nodeName.toLowerCase();return"input"===t&&!!e.checked||"option"===t&&!!e.selected},selected:function(e){return e.parentNode&&e.parentNode.selectedIndex,e.selected===!0},empty:function(e){for(e=e.firstChild;e;e=e.nextSibling)if(e.nodeName>"@"||3===e.nodeType||4===e.nodeType)return!1;return!0},parent:function(e){return!i.pseudos.empty(e)},header:function(e){return Q.test(e.nodeName)},input:function(e){return G.test(e.nodeName)},button:function(e){var t=e.nodeName.toLowerCase();return"input"===t&&"button"===e.type||"button"===t},text:function(e){var t;return"input"===e.nodeName.toLowerCase()&&"text"===e.type&&(null==(t=e.getAttribute("type"))||t.toLowerCase()===e.type)},first:pt(function(){return[0]}),last:pt(function(e,t){return[t-1]}),eq:pt(function(e,t,n){return[0>n?n+t:n]}),even:pt(function(e,t){var n=0;for(;t>n;n+=2)e.push(n);return e}),odd:pt(function(e,t){var n=1;for(;t>n;n+=2)e.push(n);return e}),lt:pt(function(e,t,n){var r=0>n?n+t:n;for(;--r>=0;)e.push(r);return e}),gt:pt(function(e,t,n){var r=0>n?n+t:n;for(;t>++r;)e.push(r);return e})}};for(n in{radio:!0,checkbox:!0,file:!0,password:!0,image:!0})i.pseudos[n]=lt(n);for(n in{submit:!0,reset:!0})i.pseudos[n]=ct(n);function ft(e,t){var n,r,o,a,s,u,l,c=E[e+" "];if(c)return t?0:c.slice(0);s=e,u=[],l=i.preFilter;while(s){(!n||(r=$.exec(s)))&&(r&&(s=s.slice(r[0].length)||s),u.push(o=[])),n=!1,(r=I.exec(s))&&(n=r.shift(),o.push({value:n,type:r[0].replace(W," ")}),s=s.slice(n.length));for(a in i.filter)!(r=U[a].exec(s))||l[a]&&!(r=l[a](r))||(n=r.shift(),o.push({value:n,type:a,matches:r}),s=s.slice(n.length));if(!n)break}return t?s.length:s?st.error(e):E(e,u).slice(0)}function dt(e){var t=0,n=e.length,r="";for(;n>t;t++)r+=e[t].value;return r}function ht(e,t,n){var i=t.dir,o=n&&"parentNode"===i,a=C++;return t.first?function(t,n,r){while(t=t[i])if(1===t.nodeType||o)return e(t,n,r)}:function(t,n,s){var u,l,c,p=N+" "+a;if(s){while(t=t[i])if((1===t.nodeType||o)&&e(t,n,s))return!0}else while(t=t[i])if(1===t.nodeType||o)if(c=t[x]||(t[x]={}),(l=c[i])&&l[0]===p){if((u=l[1])===!0||u===r)return u===!0}else if(l=c[i]=[p],l[1]=e(t,n,s)||r,l[1]===!0)return!0}}function gt(e){return e.length>1?function(t,n,r){var i=e.length;while(i--)if(!e[i](t,n,r))return!1;return!0}:e[0]}function mt(e,t,n,r,i){var o,a=[],s=0,u=e.length,l=null!=t;for(;u>s;s++)(o=e[s])&&(!n||n(o,r,i))&&(a.push(o),l&&t.push(s));return a}function yt(e,t,n,r,i,o){return r&&!r[x]&&(r=yt(r)),i&&!i[x]&&(i=yt(i,o)),ot(function(o,a,s,u){var l,c,p,f=[],d=[],h=a.length,g=o||xt(t||"*",s.nodeType?[s]:s,[]),m=!e||!o&&t?g:mt(g,f,e,s,u),y=n?i||(o?e:h||r)?[]:a:m;if(n&&n(m,y,s,u),r){l=mt(y,d),r(l,[],s,u),c=l.length;while(c--)(p=l[c])&&(y[d[c]]=!(m[d[c]]=p))}if(o){if(i||e){if(i){l=[],c=y.length;while(c--)(p=y[c])&&l.push(m[c]=p);i(null,y=[],l,u)}c=y.length;while(c--)(p=y[c])&&(l=i?M.call(o,p):f[c])>-1&&(o[l]=!(a[l]=p))}}else y=mt(y===a?y.splice(h,y.length):y),i?i(null,a,y,u):H.apply(a,y)})}function vt(e){var t,n,r,o=e.length,a=i.relative[e[0].type],s=a||i.relative[" "],u=a?1:0,c=ht(function(e){return e===t},s,!0),p=ht(function(e){return M.call(t,e)>-1},s,!0),f=[function(e,n,r){return!a&&(r||n!==l)||((t=n).nodeType?c(e,n,r):p(e,n,r))}];for(;o>u;u++)if(n=i.relative[e[u].type])f=[ht(gt(f),n)];else{if(n=i.filter[e[u].type].apply(null,e[u].matches),n[x]){for(r=++u;o>r;r++)if(i.relative[e[r].type])break;return yt(u>1&>(f),u>1&&dt(e.slice(0,u-1)).replace(W,"$1"),n,r>u&&vt(e.slice(u,r)),o>r&&vt(e=e.slice(r)),o>r&&dt(e))}f.push(n)}return gt(f)}function bt(e,t){var n=0,o=t.length>0,a=e.length>0,s=function(s,u,c,f,d){var h,g,m,y=[],v=0,b="0",x=s&&[],w=null!=d,T=l,C=s||a&&i.find.TAG("*",d&&u.parentNode||u),k=N+=null==T?1:Math.random()||.1;for(w&&(l=u!==p&&u,r=n);null!=(h=C[b]);b++){if(a&&h){g=0;while(m=e[g++])if(m(h,u,c)){f.push(h);break}w&&(N=k,r=++n)}o&&((h=!m&&h)&&v--,s&&x.push(h))}if(v+=b,o&&b!==v){g=0;while(m=t[g++])m(x,y,u,c);if(s){if(v>0)while(b--)x[b]||y[b]||(y[b]=L.call(f));y=mt(y)}H.apply(f,y),w&&!s&&y.length>0&&v+t.length>1&&st.uniqueSort(f)}return w&&(N=k,l=T),x};return o?ot(s):s}s=st.compile=function(e,t){var n,r=[],i=[],o=S[e+" "];if(!o){t||(t=ft(e)),n=t.length;while(n--)o=vt(t[n]),o[x]?r.push(o):i.push(o);o=S(e,bt(i,r))}return o};function xt(e,t,n){var r=0,i=t.length;for(;i>r;r++)st(e,t[r],n);return n}function wt(e,t,n,r){var o,a,u,l,c,p=ft(e);if(!r&&1===p.length){if(a=p[0]=p[0].slice(0),a.length>2&&"ID"===(u=a[0]).type&&9===t.nodeType&&!d&&i.relative[a[1].type]){if(t=i.find.ID(u.matches[0].replace(et,tt),t)[0],!t)return n;e=e.slice(a.shift().value.length)}o=U.needsContext.test(e)?0:a.length;while(o--){if(u=a[o],i.relative[l=u.type])break;if((c=i.find[l])&&(r=c(u.matches[0].replace(et,tt),V.test(a[0].type)&&t.parentNode||t))){if(a.splice(o,1),e=r.length&&dt(a),!e)return H.apply(n,q.call(r,0)),n;break}}}return s(e,p)(r,t,d,n,V.test(e)),n}i.pseudos.nth=i.pseudos.eq;function Tt(){}i.filters=Tt.prototype=i.pseudos,i.setFilters=new Tt,c(),st.attr=b.attr,b.find=st,b.expr=st.selectors,b.expr[":"]=b.expr.pseudos,b.unique=st.uniqueSort,b.text=st.getText,b.isXMLDoc=st.isXML,b.contains=st.contains}(e);var at=/Until$/,st=/^(?:parents|prev(?:Until|All))/,ut=/^.[^:#\[\.,]*$/,lt=b.expr.match.needsContext,ct={children:!0,contents:!0,next:!0,prev:!0};b.fn.extend({find:function(e){var t,n,r,i=this.length;if("string"!=typeof e)return r=this,this.pushStack(b(e).filter(function(){for(t=0;i>t;t++)if(b.contains(r[t],this))return!0}));for(n=[],t=0;i>t;t++)b.find(e,this[t],n);return n=this.pushStack(i>1?b.unique(n):n),n.selector=(this.selector?this.selector+" ":"")+e,n},has:function(e){var t,n=b(e,this),r=n.length;return this.filter(function(){for(t=0;r>t;t++)if(b.contains(this,n[t]))return!0})},not:function(e){return this.pushStack(ft(this,e,!1))},filter:function(e){return this.pushStack(ft(this,e,!0))},is:function(e){return!!e&&("string"==typeof e?lt.test(e)?b(e,this.context).index(this[0])>=0:b.filter(e,this).length>0:this.filter(e).length>0)},closest:function(e,t){var n,r=0,i=this.length,o=[],a=lt.test(e)||"string"!=typeof e?b(e,t||this.context):0;for(;i>r;r++){n=this[r];while(n&&n.ownerDocument&&n!==t&&11!==n.nodeType){if(a?a.index(n)>-1:b.find.matchesSelector(n,e)){o.push(n);break}n=n.parentNode}}return this.pushStack(o.length>1?b.unique(o):o)},index:function(e){return e?"string"==typeof e?b.inArray(this[0],b(e)):b.inArray(e.jquery?e[0]:e,this):this[0]&&this[0].parentNode?this.first().prevAll().length:-1},add:function(e,t){var n="string"==typeof e?b(e,t):b.makeArray(e&&e.nodeType?[e]:e),r=b.merge(this.get(),n);return this.pushStack(b.unique(r))},addBack:function(e){return this.add(null==e?this.prevObject:this.prevObject.filter(e))}}),b.fn.andSelf=b.fn.addBack;function pt(e,t){do e=e[t];while(e&&1!==e.nodeType);return e}b.each({parent:function(e){var t=e.parentNode;return t&&11!==t.nodeType?t:null},parents:function(e){return b.dir(e,"parentNode")},parentsUntil:function(e,t,n){return b.dir(e,"parentNode",n)},next:function(e){return pt(e,"nextSibling")},prev:function(e){return pt(e,"previousSibling")},nextAll:function(e){return b.dir(e,"nextSibling")},prevAll:function(e){return b.dir(e,"previousSibling")},nextUntil:function(e,t,n){return b.dir(e,"nextSibling",n)},prevUntil:function(e,t,n){return b.dir(e,"previousSibling",n)},siblings:function(e){return b.sibling((e.parentNode||{}).firstChild,e)},children:function(e){return b.sibling(e.firstChild)},contents:function(e){return b.nodeName(e,"iframe")?e.contentDocument||e.contentWindow.document:b.merge([],e.childNodes)}},function(e,t){b.fn[e]=function(n,r){var i=b.map(this,t,n);return at.test(e)||(r=n),r&&"string"==typeof r&&(i=b.filter(r,i)),i=this.length>1&&!ct[e]?b.unique(i):i,this.length>1&&st.test(e)&&(i=i.reverse()),this.pushStack(i)}}),b.extend({filter:function(e,t,n){return n&&(e=":not("+e+")"),1===t.length?b.find.matchesSelector(t[0],e)?[t[0]]:[]:b.find.matches(e,t)},dir:function(e,n,r){var i=[],o=e[n];while(o&&9!==o.nodeType&&(r===t||1!==o.nodeType||!b(o).is(r)))1===o.nodeType&&i.push(o),o=o[n];return i},sibling:function(e,t){var n=[];for(;e;e=e.nextSibling)1===e.nodeType&&e!==t&&n.push(e);return n}});function ft(e,t,n){if(t=t||0,b.isFunction(t))return b.grep(e,function(e,r){var i=!!t.call(e,r,e);return i===n});if(t.nodeType)return b.grep(e,function(e){return e===t===n});if("string"==typeof t){var r=b.grep(e,function(e){return 1===e.nodeType});if(ut.test(t))return b.filter(t,r,!n);t=b.filter(t,r)}return b.grep(e,function(e){return b.inArray(e,t)>=0===n})}function dt(e){var t=ht.split("|"),n=e.createDocumentFragment();if(n.createElement)while(t.length)n.createElement(t.pop());return n}var ht="abbr|article|aside|audio|bdi|canvas|data|datalist|details|figcaption|figure|footer|header|hgroup|mark|meter|nav|output|progress|section|summary|time|video",gt=/ jQuery\d+="(?:null|\d+)"/g,mt=RegExp("<(?:"+ht+")[\\s/>]","i"),yt=/^\s+/,vt=/<(?!area|br|col|embed|hr|img|input|link|meta|param)(([\w:]+)[^>]*)\/>/gi,bt=/<([\w:]+)/,xt=/\s*$/g,At={option:[1,""],legend:[1,"
    ","
    "],area:[1,"",""],param:[1,"",""],thead:[1,"","
    "],tr:[2,"","
    "],col:[2,"","
    "],td:[3,"","
    "],_default:b.support.htmlSerialize?[0,"",""]:[1,"X
    ","
    "]},jt=dt(o),Dt=jt.appendChild(o.createElement("div"));At.optgroup=At.option,At.tbody=At.tfoot=At.colgroup=At.caption=At.thead,At.th=At.td,b.fn.extend({text:function(e){return b.access(this,function(e){return e===t?b.text(this):this.empty().append((this[0]&&this[0].ownerDocument||o).createTextNode(e))},null,e,arguments.length)},wrapAll:function(e){if(b.isFunction(e))return this.each(function(t){b(this).wrapAll(e.call(this,t))});if(this[0]){var t=b(e,this[0].ownerDocument).eq(0).clone(!0);this[0].parentNode&&t.insertBefore(this[0]),t.map(function(){var e=this;while(e.firstChild&&1===e.firstChild.nodeType)e=e.firstChild;return e}).append(this)}return this},wrapInner:function(e){return b.isFunction(e)?this.each(function(t){b(this).wrapInner(e.call(this,t))}):this.each(function(){var t=b(this),n=t.contents();n.length?n.wrapAll(e):t.append(e)})},wrap:function(e){var t=b.isFunction(e);return this.each(function(n){b(this).wrapAll(t?e.call(this,n):e)})},unwrap:function(){return this.parent().each(function(){b.nodeName(this,"body")||b(this).replaceWith(this.childNodes)}).end()},append:function(){return this.domManip(arguments,!0,function(e){(1===this.nodeType||11===this.nodeType||9===this.nodeType)&&this.appendChild(e)})},prepend:function(){return this.domManip(arguments,!0,function(e){(1===this.nodeType||11===this.nodeType||9===this.nodeType)&&this.insertBefore(e,this.firstChild)})},before:function(){return this.domManip(arguments,!1,function(e){this.parentNode&&this.parentNode.insertBefore(e,this)})},after:function(){return this.domManip(arguments,!1,function(e){this.parentNode&&this.parentNode.insertBefore(e,this.nextSibling)})},remove:function(e,t){var n,r=0;for(;null!=(n=this[r]);r++)(!e||b.filter(e,[n]).length>0)&&(t||1!==n.nodeType||b.cleanData(Ot(n)),n.parentNode&&(t&&b.contains(n.ownerDocument,n)&&Mt(Ot(n,"script")),n.parentNode.removeChild(n)));return this},empty:function(){var e,t=0;for(;null!=(e=this[t]);t++){1===e.nodeType&&b.cleanData(Ot(e,!1));while(e.firstChild)e.removeChild(e.firstChild);e.options&&b.nodeName(e,"select")&&(e.options.length=0)}return this},clone:function(e,t){return e=null==e?!1:e,t=null==t?e:t,this.map(function(){return b.clone(this,e,t)})},html:function(e){return b.access(this,function(e){var n=this[0]||{},r=0,i=this.length;if(e===t)return 1===n.nodeType?n.innerHTML.replace(gt,""):t;if(!("string"!=typeof e||Tt.test(e)||!b.support.htmlSerialize&&mt.test(e)||!b.support.leadingWhitespace&&yt.test(e)||At[(bt.exec(e)||["",""])[1].toLowerCase()])){e=e.replace(vt,"<$1>");try{for(;i>r;r++)n=this[r]||{},1===n.nodeType&&(b.cleanData(Ot(n,!1)),n.innerHTML=e);n=0}catch(o){}}n&&this.empty().append(e)},null,e,arguments.length)},replaceWith:function(e){var t=b.isFunction(e);return t||"string"==typeof e||(e=b(e).not(this).detach()),this.domManip([e],!0,function(e){var t=this.nextSibling,n=this.parentNode;n&&(b(this).remove(),n.insertBefore(e,t))})},detach:function(e){return this.remove(e,!0)},domManip:function(e,n,r){e=f.apply([],e);var i,o,a,s,u,l,c=0,p=this.length,d=this,h=p-1,g=e[0],m=b.isFunction(g);if(m||!(1>=p||"string"!=typeof g||b.support.checkClone)&&Ct.test(g))return this.each(function(i){var o=d.eq(i);m&&(e[0]=g.call(this,i,n?o.html():t)),o.domManip(e,n,r)});if(p&&(l=b.buildFragment(e,this[0].ownerDocument,!1,this),i=l.firstChild,1===l.childNodes.length&&(l=i),i)){for(n=n&&b.nodeName(i,"tr"),s=b.map(Ot(l,"script"),Ht),a=s.length;p>c;c++)o=l,c!==h&&(o=b.clone(o,!0,!0),a&&b.merge(s,Ot(o,"script"))),r.call(n&&b.nodeName(this[c],"table")?Lt(this[c],"tbody"):this[c],o,c);if(a)for(u=s[s.length-1].ownerDocument,b.map(s,qt),c=0;a>c;c++)o=s[c],kt.test(o.type||"")&&!b._data(o,"globalEval")&&b.contains(u,o)&&(o.src?b.ajax({url:o.src,type:"GET",dataType:"script",async:!1,global:!1,"throws":!0}):b.globalEval((o.text||o.textContent||o.innerHTML||"").replace(St,"")));l=i=null}return this}});function Lt(e,t){return e.getElementsByTagName(t)[0]||e.appendChild(e.ownerDocument.createElement(t))}function Ht(e){var t=e.getAttributeNode("type");return e.type=(t&&t.specified)+"/"+e.type,e}function qt(e){var t=Et.exec(e.type);return t?e.type=t[1]:e.removeAttribute("type"),e}function Mt(e,t){var n,r=0;for(;null!=(n=e[r]);r++)b._data(n,"globalEval",!t||b._data(t[r],"globalEval"))}function _t(e,t){if(1===t.nodeType&&b.hasData(e)){var n,r,i,o=b._data(e),a=b._data(t,o),s=o.events;if(s){delete a.handle,a.events={};for(n in s)for(r=0,i=s[n].length;i>r;r++)b.event.add(t,n,s[n][r])}a.data&&(a.data=b.extend({},a.data))}}function Ft(e,t){var n,r,i;if(1===t.nodeType){if(n=t.nodeName.toLowerCase(),!b.support.noCloneEvent&&t[b.expando]){i=b._data(t);for(r in i.events)b.removeEvent(t,r,i.handle);t.removeAttribute(b.expando)}"script"===n&&t.text!==e.text?(Ht(t).text=e.text,qt(t)):"object"===n?(t.parentNode&&(t.outerHTML=e.outerHTML),b.support.html5Clone&&e.innerHTML&&!b.trim(t.innerHTML)&&(t.innerHTML=e.innerHTML)):"input"===n&&Nt.test(e.type)?(t.defaultChecked=t.checked=e.checked,t.value!==e.value&&(t.value=e.value)):"option"===n?t.defaultSelected=t.selected=e.defaultSelected:("input"===n||"textarea"===n)&&(t.defaultValue=e.defaultValue)}}b.each({appendTo:"append",prependTo:"prepend",insertBefore:"before",insertAfter:"after",replaceAll:"replaceWith"},function(e,t){b.fn[e]=function(e){var n,r=0,i=[],o=b(e),a=o.length-1;for(;a>=r;r++)n=r===a?this:this.clone(!0),b(o[r])[t](n),d.apply(i,n.get());return this.pushStack(i)}});function Ot(e,n){var r,o,a=0,s=typeof e.getElementsByTagName!==i?e.getElementsByTagName(n||"*"):typeof e.querySelectorAll!==i?e.querySelectorAll(n||"*"):t;if(!s)for(s=[],r=e.childNodes||e;null!=(o=r[a]);a++)!n||b.nodeName(o,n)?s.push(o):b.merge(s,Ot(o,n));return n===t||n&&b.nodeName(e,n)?b.merge([e],s):s}function Bt(e){Nt.test(e.type)&&(e.defaultChecked=e.checked)}b.extend({clone:function(e,t,n){var r,i,o,a,s,u=b.contains(e.ownerDocument,e);if(b.support.html5Clone||b.isXMLDoc(e)||!mt.test("<"+e.nodeName+">")?o=e.cloneNode(!0):(Dt.innerHTML=e.outerHTML,Dt.removeChild(o=Dt.firstChild)),!(b.support.noCloneEvent&&b.support.noCloneChecked||1!==e.nodeType&&11!==e.nodeType||b.isXMLDoc(e)))for(r=Ot(o),s=Ot(e),a=0;null!=(i=s[a]);++a)r[a]&&Ft(i,r[a]);if(t)if(n)for(s=s||Ot(e),r=r||Ot(o),a=0;null!=(i=s[a]);a++)_t(i,r[a]);else _t(e,o);return r=Ot(o,"script"),r.length>0&&Mt(r,!u&&Ot(e,"script")),r=s=i=null,o},buildFragment:function(e,t,n,r){var i,o,a,s,u,l,c,p=e.length,f=dt(t),d=[],h=0;for(;p>h;h++)if(o=e[h],o||0===o)if("object"===b.type(o))b.merge(d,o.nodeType?[o]:o);else if(wt.test(o)){s=s||f.appendChild(t.createElement("div")),u=(bt.exec(o)||["",""])[1].toLowerCase(),c=At[u]||At._default,s.innerHTML=c[1]+o.replace(vt,"<$1>")+c[2],i=c[0];while(i--)s=s.lastChild;if(!b.support.leadingWhitespace&&yt.test(o)&&d.push(t.createTextNode(yt.exec(o)[0])),!b.support.tbody){o="table"!==u||xt.test(o)?""!==c[1]||xt.test(o)?0:s:s.firstChild,i=o&&o.childNodes.length;while(i--)b.nodeName(l=o.childNodes[i],"tbody")&&!l.childNodes.length&&o.removeChild(l) +}b.merge(d,s.childNodes),s.textContent="";while(s.firstChild)s.removeChild(s.firstChild);s=f.lastChild}else d.push(t.createTextNode(o));s&&f.removeChild(s),b.support.appendChecked||b.grep(Ot(d,"input"),Bt),h=0;while(o=d[h++])if((!r||-1===b.inArray(o,r))&&(a=b.contains(o.ownerDocument,o),s=Ot(f.appendChild(o),"script"),a&&Mt(s),n)){i=0;while(o=s[i++])kt.test(o.type||"")&&n.push(o)}return s=null,f},cleanData:function(e,t){var n,r,o,a,s=0,u=b.expando,l=b.cache,p=b.support.deleteExpando,f=b.event.special;for(;null!=(n=e[s]);s++)if((t||b.acceptData(n))&&(o=n[u],a=o&&l[o])){if(a.events)for(r in a.events)f[r]?b.event.remove(n,r):b.removeEvent(n,r,a.handle);l[o]&&(delete l[o],p?delete n[u]:typeof n.removeAttribute!==i?n.removeAttribute(u):n[u]=null,c.push(o))}}});var Pt,Rt,Wt,$t=/alpha\([^)]*\)/i,It=/opacity\s*=\s*([^)]*)/,zt=/^(top|right|bottom|left)$/,Xt=/^(none|table(?!-c[ea]).+)/,Ut=/^margin/,Vt=RegExp("^("+x+")(.*)$","i"),Yt=RegExp("^("+x+")(?!px)[a-z%]+$","i"),Jt=RegExp("^([+-])=("+x+")","i"),Gt={BODY:"block"},Qt={position:"absolute",visibility:"hidden",display:"block"},Kt={letterSpacing:0,fontWeight:400},Zt=["Top","Right","Bottom","Left"],en=["Webkit","O","Moz","ms"];function tn(e,t){if(t in e)return t;var n=t.charAt(0).toUpperCase()+t.slice(1),r=t,i=en.length;while(i--)if(t=en[i]+n,t in e)return t;return r}function nn(e,t){return e=t||e,"none"===b.css(e,"display")||!b.contains(e.ownerDocument,e)}function rn(e,t){var n,r,i,o=[],a=0,s=e.length;for(;s>a;a++)r=e[a],r.style&&(o[a]=b._data(r,"olddisplay"),n=r.style.display,t?(o[a]||"none"!==n||(r.style.display=""),""===r.style.display&&nn(r)&&(o[a]=b._data(r,"olddisplay",un(r.nodeName)))):o[a]||(i=nn(r),(n&&"none"!==n||!i)&&b._data(r,"olddisplay",i?n:b.css(r,"display"))));for(a=0;s>a;a++)r=e[a],r.style&&(t&&"none"!==r.style.display&&""!==r.style.display||(r.style.display=t?o[a]||"":"none"));return e}b.fn.extend({css:function(e,n){return b.access(this,function(e,n,r){var i,o,a={},s=0;if(b.isArray(n)){for(o=Rt(e),i=n.length;i>s;s++)a[n[s]]=b.css(e,n[s],!1,o);return a}return r!==t?b.style(e,n,r):b.css(e,n)},e,n,arguments.length>1)},show:function(){return rn(this,!0)},hide:function(){return rn(this)},toggle:function(e){var t="boolean"==typeof e;return this.each(function(){(t?e:nn(this))?b(this).show():b(this).hide()})}}),b.extend({cssHooks:{opacity:{get:function(e,t){if(t){var n=Wt(e,"opacity");return""===n?"1":n}}}},cssNumber:{columnCount:!0,fillOpacity:!0,fontWeight:!0,lineHeight:!0,opacity:!0,orphans:!0,widows:!0,zIndex:!0,zoom:!0},cssProps:{"float":b.support.cssFloat?"cssFloat":"styleFloat"},style:function(e,n,r,i){if(e&&3!==e.nodeType&&8!==e.nodeType&&e.style){var o,a,s,u=b.camelCase(n),l=e.style;if(n=b.cssProps[u]||(b.cssProps[u]=tn(l,u)),s=b.cssHooks[n]||b.cssHooks[u],r===t)return s&&"get"in s&&(o=s.get(e,!1,i))!==t?o:l[n];if(a=typeof r,"string"===a&&(o=Jt.exec(r))&&(r=(o[1]+1)*o[2]+parseFloat(b.css(e,n)),a="number"),!(null==r||"number"===a&&isNaN(r)||("number"!==a||b.cssNumber[u]||(r+="px"),b.support.clearCloneStyle||""!==r||0!==n.indexOf("background")||(l[n]="inherit"),s&&"set"in s&&(r=s.set(e,r,i))===t)))try{l[n]=r}catch(c){}}},css:function(e,n,r,i){var o,a,s,u=b.camelCase(n);return n=b.cssProps[u]||(b.cssProps[u]=tn(e.style,u)),s=b.cssHooks[n]||b.cssHooks[u],s&&"get"in s&&(a=s.get(e,!0,r)),a===t&&(a=Wt(e,n,i)),"normal"===a&&n in Kt&&(a=Kt[n]),""===r||r?(o=parseFloat(a),r===!0||b.isNumeric(o)?o||0:a):a},swap:function(e,t,n,r){var i,o,a={};for(o in t)a[o]=e.style[o],e.style[o]=t[o];i=n.apply(e,r||[]);for(o in t)e.style[o]=a[o];return i}}),e.getComputedStyle?(Rt=function(t){return e.getComputedStyle(t,null)},Wt=function(e,n,r){var i,o,a,s=r||Rt(e),u=s?s.getPropertyValue(n)||s[n]:t,l=e.style;return s&&(""!==u||b.contains(e.ownerDocument,e)||(u=b.style(e,n)),Yt.test(u)&&Ut.test(n)&&(i=l.width,o=l.minWidth,a=l.maxWidth,l.minWidth=l.maxWidth=l.width=u,u=s.width,l.width=i,l.minWidth=o,l.maxWidth=a)),u}):o.documentElement.currentStyle&&(Rt=function(e){return e.currentStyle},Wt=function(e,n,r){var i,o,a,s=r||Rt(e),u=s?s[n]:t,l=e.style;return null==u&&l&&l[n]&&(u=l[n]),Yt.test(u)&&!zt.test(n)&&(i=l.left,o=e.runtimeStyle,a=o&&o.left,a&&(o.left=e.currentStyle.left),l.left="fontSize"===n?"1em":u,u=l.pixelLeft+"px",l.left=i,a&&(o.left=a)),""===u?"auto":u});function on(e,t,n){var r=Vt.exec(t);return r?Math.max(0,r[1]-(n||0))+(r[2]||"px"):t}function an(e,t,n,r,i){var o=n===(r?"border":"content")?4:"width"===t?1:0,a=0;for(;4>o;o+=2)"margin"===n&&(a+=b.css(e,n+Zt[o],!0,i)),r?("content"===n&&(a-=b.css(e,"padding"+Zt[o],!0,i)),"margin"!==n&&(a-=b.css(e,"border"+Zt[o]+"Width",!0,i))):(a+=b.css(e,"padding"+Zt[o],!0,i),"padding"!==n&&(a+=b.css(e,"border"+Zt[o]+"Width",!0,i)));return a}function sn(e,t,n){var r=!0,i="width"===t?e.offsetWidth:e.offsetHeight,o=Rt(e),a=b.support.boxSizing&&"border-box"===b.css(e,"boxSizing",!1,o);if(0>=i||null==i){if(i=Wt(e,t,o),(0>i||null==i)&&(i=e.style[t]),Yt.test(i))return i;r=a&&(b.support.boxSizingReliable||i===e.style[t]),i=parseFloat(i)||0}return i+an(e,t,n||(a?"border":"content"),r,o)+"px"}function un(e){var t=o,n=Gt[e];return n||(n=ln(e,t),"none"!==n&&n||(Pt=(Pt||b(" + +## Architecture Principles CFEngine is agent based software. It resides on and runs processes on each individual computer under its management. That means you do not need to grant any security credentials for login to CFEngine. Instead, for normal operation, -CFEngine runs in privileged `root' or `Administrator' mode to get access to +CFEngine runs in privileged `root` or `Administrator` mode to get access to system resources and makes these available safely to authorized enquiries. A CFEngine installation is thus required on every machine you want to manage: @@ -45,7 +52,7 @@ communication (e.g. network connectivity) and that each host is responsible for maintaining its own state. This affects the security and scalability of the solution. -## Single point of coordination +### Single point of coordination The default CFEngine Nova architecture uses a single hub or policy server to publish changes of policy and to aggregate knowledge about the environment, but @@ -54,8 +61,8 @@ organization independently. The CFEngine technology is not centralized by nature. Most users choose to centralize updating of policy and report aggregation for convenience however. -Figure: A policy server or `hub' is implemented in CFEngine Nova -as a simple solution that will scale for most sites `out of the box'. +Figure: A policy server or `hub` is implemented in CFEngine Nova +as a simple solution that will scale for most sites out of the box. If you operate CFEngine Nova in its default mode, the hub acts as a server from @@ -69,7 +76,7 @@ outside of normal production environment. This can be done using the specialized editor embedded in CFEngine Nova, or it can be done using any text editor of your choice. -## Policy information flow +### Policy information flow Edits are made in conjunction with a version control repository1, which should be used to document the reasons for changes to policy2. When a change has been @@ -82,20 +89,13 @@ Figure: Policy coordinated from a central root location is implemented in a distributed manner at every leaf node. -## Robustness to failure +### Robustness to failure If an agent receives a policy proposal that is badly formed or in some way non-executable, it switches to a failover strategy to recover. It will continue in this mode until a new policy proposal is available that can be executed. -The CFEngine agent clones itself to avoid limitations of operating systems like -Windows, where programs and disk files cannot be altered while in use. When new -software updates are available, CFEngine can update itself from a suitable -source, and restart its own services. Should the new version be corrupt, the -twin will still be the old working version, hence the software will be able to -recover as soon as a new valid version is available. - -## Distributed execution and federation +### Distributed execution and federation Each agent runs independently of others, unless it promises to acquire services from other hosts. Thus all processing capacity and decision-making computation @@ -110,12 +110,12 @@ where attention to local requirements is paramount for whatever reason. Federation is typically a recommended strategy when the cost of avoiding local specialization outweighs the price of having local policy-makers. Universities and large companies (e.g. formed through acquisition) are typical candidates for -federated management. Federation is facilitated by an essentially `service -oriented architecture'3, i.e. a weak coupling. +federated management. Federation is facilitated by a service +oriented architecture, i.e. a weak coupling. -# Security Principles +## Security Principles -## What is security? +### What is security? The concept of security, while various in its interpretation and intented use, @@ -125,7 +125,7 @@ defined by a model, an attitude, and a policy. It involves a set of compromises called atrust modelthat determines where you draw the line in the sand between trusted and risky. -## The principles of CFEngine security +### The principles of CFEngine security CFEngine adheres to the following design principles: @@ -145,7 +145,7 @@ CFEngine adheres to the following design principles: * CFEngine shall always provide safe defaults, that grant no access to other hosts. -## Communications +### Communications CFEngine uses a simple, private protocol that is based on (but not identical to) that used by OpenSSH (the free version of the Secure Shel). It is based on @@ -155,15 +155,14 @@ Infrastructure. * Authentication by Public Key is mandatory. * Encryption of data transfer is optional. -# Communication Security - -## TCP wrappers +## Communication Security +### TCP wrappers The right to connect to the server is the first line of defence. CFEngine has -built into it the equivalent of the `TCP wrappers' software to deny +built into it the equivalent of the `TCP wrappers` software to deny non-authorized hosts the ability to connect to the server at all. -## The connection sequence +### The connection sequence A client attempts to connect to port 5308 Server examines IP address of connection and applies rules from @@ -199,7 +198,7 @@ connection and applies rules from * All other deny rules are ignored * No deny rule means you're admitted -## Encryption algorithms +### Encryption algorithms CFEngine Community Edition uses RSA 2048 public key encryption for authentication. These are generated by the ‘cf-key’ command. It generates a 128 @@ -211,17 +210,17 @@ and then AES 256 with a 256 bit random key for data transmission. The latter is validated for FIPS 140-2 government use in the United States of America. Challenge response is verified by a SHA256 hash. -## Remote communication +### Remote communication The concept of voluntary cooperation used by CFEngine places restrictions on how -files can be copied between hosts. CFEngine allows only `pull' (download) but -not `push' (upload). Users cannot force an agent to perform an operation against +files can be copied between hosts. CFEngine allows only _pull_ (download) but +not _push_ (upload). Users cannot force an agent to perform an operation against local policy. To allow remote copying between two systems each of the system must explicitly grant access before the operation can take place. -## Authentication +### Authentication Authentication is about making sure users are who they say you are. Traditionally, there are two approaches: the trusted third party (arbiter of the @@ -247,7 +246,7 @@ instead of ‘ssh key-gen’ program. Key acceptance is accomplished in CFEngine using trust-key method. Once the keys have been exchange the trust settings are irrelevant. -## Security FAQ +### Security FAQ Doesn't opening a port on a machine on the inside of the firewall make it vulnerable to both Denial of Service and buffer overflow attacks? @@ -291,14 +290,14 @@ firewall security model of trusted/untrusted regions. The firewall does not mitigate the responsibility of security every host in a network regardless of which side of the firewall it is connected. -# CFEngine and Firewalls +### CFEngine and Firewalls Some users want to use CFEngine's remote copying mechanism through a firewall, in particular to update the CFEngine policy on hosts inside a DMZ (so-called de-militarized zone). In making a risk assessment, it is important to see the firewall security model together with the CFEngine security model. CFEngine's security record is better than most firewalls, but Firewalls are nearly always -trusted because they are `security products'. +trusted because they are "_security products_". Any piece of software that traverses a firewall can, in principle, weaken the security of the barrier. On the other hand, a strong piece or software might @@ -328,7 +327,7 @@ inside of the firewall, information has to traverse the firewall. * Policy Mirror in the DMZ * Pulling through a wormhole -### CFEngine trust model +#### CFEngine trust model CFEngine's trust model is fundamentally at odds with the external firewall concept. CFEngine says: “I am my own boss. I don't trust anyone to push me @@ -362,7 +361,7 @@ latter to offer at this point: ludicrous to suggest that an arbitrary employee's machine is more secure than an inaccessible host in the DMZ. -### Policy Mirror in the DMZ +#### Policy Mirror in the DMZ By creating a policy mirror in the DMZ, these issues can be worked around. This is the recommended way to copy files, so that normal CFEngine pull methods can @@ -379,7 +378,7 @@ on the mirror, except that now you have to trust the security of another piece of software too. Since this is not a CFEngine port, no guarantees can be made about what access attackers will get to the mirror host. -### Pulling through a wormhole +#### Pulling through a wormhole Suppose you are allowed to open a hole in your firewall to a single policy host on the inside. To distribute files to hosts that are outside the firewall it is @@ -413,7 +412,7 @@ Once a new copy of the policy is downloaded by CFEngine to the policy mirror, other clients in the DMZ can download that copy from the mirror. The security of other hosts in the DMZ is dependent on the security of the policy mirror. -## Tamperproof data and distributed monitoring +### Tamperproof data and distributed monitoring Message digests are supposed to be unbreakable, tamperproof technologies, but of course everything can be broken by a sufficiently determined attacker. Suppose @@ -484,7 +483,7 @@ cannot see the contents of your files from this. A possibly greater problem is that this configuration will unleash an avalanche of messages if a change is detected. This makes messages visible at least. -# Footnotes +## Footnotes [1] CFEngine supports integration with Subversion through its Mission Portal, but any versioning system can of course be used. diff --git a/guide/writing-and-serving-policy/authoring-policy-tools-and-workflow.markdown b/guide/writing-and-serving-policy/authoring-policy-tools-and-workflow.markdown index 506923a9b..b55b53322 100644 --- a/guide/writing-and-serving-policy/authoring-policy-tools-and-workflow.markdown +++ b/guide/writing-and-serving-policy/authoring-policy-tools-and-workflow.markdown @@ -50,10 +50,14 @@ Method Two: Create Masterfiles Repository Using the GitHub Application #### Initialize Git Repository in Masterfiles on the Hub #### 1. `> cd /var/cfengine/masterfiles` -2. `> git init` -3. `> git commit -m "First commit"` -4. `> git remote add origin https://github.com/GitUserName/cfengine-masterfiles.git` -5. `> git push -u origin master` +2. `> echo cf_promises_validated >> .gitignore` +3. `> echo cf_promises_release_id >> .gitignore` +4. `> git init` +5. `> git commit -m "First commit"` +6. `> git remote add origin https://github.com/GitUserName/cfengine-masterfiles.git` +7. `> git push -u origin master` + +**Note:** `cf_promises_validated` and `cf_promises_release_id` should be explicitly excluded from VCS as shown above. They are generated files and involved in controlling policy updates. If these files are checked into the repository it can create issues with policy distribution. Using the above steps on a private repository will fail with a 403 error. There are different approaches to deal with this: @@ -85,7 +89,7 @@ B) Or, change the remote url to `https://GitUserName@password:github.com/GitUser 2. Create the remote using the following pattern: * `> git remote add upstream ssh://git@github.com/GitUserName/cfengine-masterfiles.git`. 3. Verify the remote was registered properly by typing `git remote -v` and pressing enter. - * You will see the remote definition in a list alongside any other previously defined remote enteries. + * You will see the remote definition in a list alongside any other previously defined remote entries. #### Add a Promise that Pulls Changes to Masterfiles on the Hub from Masterfiles on GitHub #### diff --git a/guide/writing-and-serving-policy/controlling-frequency.markdown b/guide/writing-and-serving-policy/controlling-frequency.markdown index fa687a2b2..cf99fae17 100644 --- a/guide/writing-and-serving-policy/controlling-frequency.markdown +++ b/guide/writing-and-serving-policy/controlling-frequency.markdown @@ -6,9 +6,55 @@ sorting: 90 tags: [manuals, systems, configuration management, automation, control, frequency, performance] --- -When checking a series of expensive functions and verifying complex promises, -you may want to make sure that CFEngine is not checking too frequently. One -way of doing this is classes and class expression, another is using locks. +By default CFEngine runs relatively frequently (every 5 minutes) but you may not +want every promise to be evaluated each agent execution. Classes and promise +locks are the two primary ways in which a promises frequency can be controlled. +Classes are the canonical way of controlling if a promise is in context and +should be evaluated. Promise locks control frequency based on the number of +minutes since the last promise actuation. + +## Controlling frequency using classes + +Classes are the canonical way of controlling promise executions in CFEngine. + +Use time based classes to restrict promises to run during a specific period of time. For example, here `sshd` promises to be the latest version available, but only on Tuesdays during the first 15 minutes of the 5:00 hour. + +```cf3 +bundle agent __main__ +{ + packages: + Tuesday.Hr05_Q1:: + "sshd" + version => "latest", + comment => "Make sure sshd is at the latest version, but only Tuesday between 5:00 and 5:15am"; +} +``` + +Persistent classes can exist for a period of time, across multiple executions of +`cf-agent`. Persistent classes can be used to avoid re-execution of a promise. +For example, here `/tmp/heartbeat.dat` promises to update it's timestamp when +`heartbeat_repaired` is not defined. When the file is repaired the class +`heartbeat_repaired` is defined for 10 minutes causing the promise to be out of +context during subsequent executions for the next 10 minutes. + +```cf3 +bundle agent __main__ +{ + files: + !heartbeat_repaired:: + "/tmp/heartbeat.dat" + create => "true", + touch => "true", + classes => persistent_results( "heartbeat", 10 ); +} +body classes persistent_results( prefix, time ) +{ + inherit_from => results( "namespace", "$(prefix)" ); + persist_time => "$(time)"; +} +``` + +## Controlling frequency using promise locks CFEngine incorporates a series of locks which prevent it from checking promises too often, and which prevent it from spending too long trying to @@ -17,19 +63,17 @@ a way that you can start several CFEngine components simultaneously without them interfering with each other. You can control two things about each kind of action in CFEngine: - ifelapsed - -The minimum time (in minutes) which should have passed since the last time -that promise was verified. It will not be executed again until this amount of -time has elapsed. Default time is 1 minute. - - expireafter +* `ifelapsed` - The minimum time (in minutes) which should have passed since the + last time that promise was verified. It will not be executed again until this + amount of time has elapsed. If the value is `0` the promise has no lock and + will always be executed when in context. Additionally, a value of `0` disables + function caching. Default time is `1` minute. -The maximum amount (in minutes) of time `cf-agent` should wait for an old -instantiation to finish before killing it and starting again. You can think -about [`expireafter`][cf-agent#expireafter] as a timeout to use when a promise verification may -involve an operation that could wait indefinitely. Default time is 120 -minutes. +* `expireafter` - The maximum amount (in minutes) of time `cf-agent` should wait + for an old instantiation to finish before killing it and starting again. You + can think about [`expireafter`][cf-agent#expireafter] as a timeout to use when + a promise verification may involve an operation that could wait indefinitely. + Default time is `120` minutes. You can set these values either globally (for all actions) or for each action separately. If you set global and local values, the local values override the @@ -46,7 +90,7 @@ setting is defined in `body agent control`. This setting tells CFEngine not to verify promises until 60 minutes have elapsed, ie ensures that the global frequency for all promise verification is one hour. This global setting of one hour could be changed for a specific -promise body by setting [`ifelapsed`][Promise Types and Attributes#ifelapsed] in the promise body. +promise body by setting [`ifelapsed`][Promise Types#ifelapsed] in the promise body. ```cf3 body action example @@ -63,3 +107,29 @@ atomic promise checks on the same objects (packages, users, files, etc.). Several different `cf-agent` instances can run concurrently. The locks ensure that promises will not be verified by two cf-agents at the same time or too soon after a verification. + +For example, here the `sshd` package promises to be at the latest version. It +has the `if_elapsed_day` action body attached which sets `ifelapsed` to `1440` +causing the promise lock to persist for a day effectively restricting the +promise to run just once a day. + +```cf3 +bundle agent __main__ +{ + packages: + "sshd" + version => "latest", + action => if_elapsed_day, + comment => "Make sure sshd is at the latest version, but just once a day."; +} +``` + +**Note:** + +* Promise locks are ignored when CFEngine is run with the `--no-lock` or `-K` + option, e.g. a common **manual** execution of the agent, `cf-agent -KI` would + not respect promises that are locked from a recent execution. +* Locks are purged based on database utilization and age in order to maintain + the integrity and health of the underlying lock database. + +**See also:** [cf_lock.lmdb][CFEngine directory structure#state/cf_lock.lmdb] diff --git a/guide/writing-and-serving-policy/editors.markdown b/guide/writing-and-serving-policy/editors.markdown new file mode 100644 index 000000000..b47f15c80 --- /dev/null +++ b/guide/writing-and-serving-policy/editors.markdown @@ -0,0 +1,63 @@ +--- +layout: default +title: Editors +published: true +sorting: 10 +tags: [tools, editor, vim, emacs, vscode, kate, sublime text, atom, eclipse ] +--- + +Using an editor that provides syntax highlighting and other features can significantly enhance prodcutivity and quality of life. + +## Emacs + +For Emacs users, editing CFEngine policies is easy with the built-in CFEngine 3 mode in the [cfengine.el library](https://github.com/cfengine/core/blob/master/contrib/cfengine.el). For an overview of the capabilities, see the [webinar by Ted Zlatanov](https://www.youtube.com/watch?v=-PPVhwSKNdE) and Appendix A of Diego Zamboni’s [Learning CFEngine](https://leanpub.com/learning-cfengine/) book. + +![Emacs](guide-writing-and-serving-policy-editors-emacs.png) + +## Spacemacs + +Spacemacs has a [CFEngine layer](https://github.com/syl20bnr/spacemacs/blob/develop/layers/%2Btools/cfengine/README.org) that configures many of the features shown in the Emacs webinar above out of the box as well as integration for executing `cfengine3` `src` blocks in `org-mode`. It’s a great way for `vi/vim` lovers to leverage the power of Emacs. + +![Spacemacs](guide-writing-and-serving-policy-editors-spacemacs.png) + +## Vi/Vim + +Vi/Vim users can edit CFEngine policies with Neil Watson’s CFEngine 3 scripts, available as GPL-software [on GitHub](https://github.com/neilhwatson/vim_cf3). Neil’s vi mode is also described in Appendix B of Diego Zamboni’s “Learning CFEngine” book. + +![Vim](guide-writing-and-serving-policy-editors-vim.png) + +## Visual Studio Code + +Microsoft VS Code users have syntax highlighting thanks to AZaugg. Install the syntax highlighting and snippets directly from within Visual Studio Code by running ext install vscode-cfengine. + +![Visual Studio Code](guide-writing-and-serving-policy-editors-visual-studio-code.png) + +## Sublime Text + +Sublime Text 2 and 3 users have syntax highlighting and snippets thanks to Valery Astraverkhau. Get the syntax highlighting and snippets from his github repository. Aki Vanhatalo has contributed a beautifier to automatically re-indent policy in Sublime Text. + Sublime Screenshot + +![Sublime Text](guide-writing-and-serving-policy-editors-sublime-text.jpg) + +## Atom + +Using Githubs hackable editor? You can get syntax highlighting with the language-cfengine3 package. + +![Atom](guide-writing-and-serving-policy-editors-atom.png) + +## Eclipse + +Interested in syntax highlighting for your CFEngine policy in Eclipse? Try this contributed syntax definition. + +Want more out of your Eclipse & CFEngine experience? Itemis Xtext expert Boris Holzer developed a CFEngine workbench for Eclipse. They even published a brief screen-cast highlighting many of its features. For more information about their workbench please contact them using this form. + +![Eclipse](guide-writing-and-serving-policy-editors-eclipse-0.png) +![Eclipse](guide-writing-and-serving-policy-editors-eclipse-1.png) +![Eclipse](guide-writing-and-serving-policy-editors-eclipse-2.png) +![Eclipse](guide-writing-and-serving-policy-editors-eclipse-3.png) + +## Kate + +Users of the editor of the KDE desktop, Kate, have syntax highlighting available thanks to Jessica Greer, John Coleman, and Panos Christeas. The syntax highlighting definition can be found with the CFEngine source in contrib. + +![Kate](guide-writing-and-serving-policy-editors-kate.jpg) diff --git a/guide/writing-and-serving-policy/guide-writing-and-serving-policy-editors-atom.png b/guide/writing-and-serving-policy/guide-writing-and-serving-policy-editors-atom.png new file mode 100644 index 000000000..3f8e5316e Binary files /dev/null and b/guide/writing-and-serving-policy/guide-writing-and-serving-policy-editors-atom.png differ diff --git a/guide/writing-and-serving-policy/guide-writing-and-serving-policy-editors-eclipse-0.png b/guide/writing-and-serving-policy/guide-writing-and-serving-policy-editors-eclipse-0.png new file mode 100644 index 000000000..c79c484fa Binary files /dev/null and b/guide/writing-and-serving-policy/guide-writing-and-serving-policy-editors-eclipse-0.png differ diff --git a/guide/writing-and-serving-policy/guide-writing-and-serving-policy-editors-eclipse-1.png b/guide/writing-and-serving-policy/guide-writing-and-serving-policy-editors-eclipse-1.png new file mode 100644 index 000000000..96f854e55 Binary files /dev/null and b/guide/writing-and-serving-policy/guide-writing-and-serving-policy-editors-eclipse-1.png differ diff --git a/guide/writing-and-serving-policy/guide-writing-and-serving-policy-editors-eclipse-2.png b/guide/writing-and-serving-policy/guide-writing-and-serving-policy-editors-eclipse-2.png new file mode 100644 index 000000000..1e3413ffe Binary files /dev/null and b/guide/writing-and-serving-policy/guide-writing-and-serving-policy-editors-eclipse-2.png differ diff --git a/guide/writing-and-serving-policy/guide-writing-and-serving-policy-editors-eclipse-3.png b/guide/writing-and-serving-policy/guide-writing-and-serving-policy-editors-eclipse-3.png new file mode 100644 index 000000000..96f854e55 Binary files /dev/null and b/guide/writing-and-serving-policy/guide-writing-and-serving-policy-editors-eclipse-3.png differ diff --git a/guide/writing-and-serving-policy/guide-writing-and-serving-policy-editors-emacs.png b/guide/writing-and-serving-policy/guide-writing-and-serving-policy-editors-emacs.png new file mode 100644 index 000000000..81f97a212 Binary files /dev/null and b/guide/writing-and-serving-policy/guide-writing-and-serving-policy-editors-emacs.png differ diff --git a/guide/writing-and-serving-policy/guide-writing-and-serving-policy-editors-kate.jpg b/guide/writing-and-serving-policy/guide-writing-and-serving-policy-editors-kate.jpg new file mode 100644 index 000000000..1855aa4de Binary files /dev/null and b/guide/writing-and-serving-policy/guide-writing-and-serving-policy-editors-kate.jpg differ diff --git a/guide/writing-and-serving-policy/guide-writing-and-serving-policy-editors-spacemacs.png b/guide/writing-and-serving-policy/guide-writing-and-serving-policy-editors-spacemacs.png new file mode 100644 index 000000000..293840867 Binary files /dev/null and b/guide/writing-and-serving-policy/guide-writing-and-serving-policy-editors-spacemacs.png differ diff --git a/guide/writing-and-serving-policy/guide-writing-and-serving-policy-editors-sublime-text.jpg b/guide/writing-and-serving-policy/guide-writing-and-serving-policy-editors-sublime-text.jpg new file mode 100644 index 000000000..3d5667e32 Binary files /dev/null and b/guide/writing-and-serving-policy/guide-writing-and-serving-policy-editors-sublime-text.jpg differ diff --git a/guide/writing-and-serving-policy/guide-writing-and-serving-policy-editors-vim.png b/guide/writing-and-serving-policy/guide-writing-and-serving-policy-editors-vim.png new file mode 100644 index 000000000..a2f878016 Binary files /dev/null and b/guide/writing-and-serving-policy/guide-writing-and-serving-policy-editors-vim.png differ diff --git a/guide/writing-and-serving-policy/guide-writing-and-serving-policy-editors-visual-studio-code.png b/guide/writing-and-serving-policy/guide-writing-and-serving-policy-editors-visual-studio-code.png new file mode 100644 index 000000000..443aa61f0 Binary files /dev/null and b/guide/writing-and-serving-policy/guide-writing-and-serving-policy-editors-visual-studio-code.png differ diff --git a/guide/writing-and-serving-policy/policy-style.markdown b/guide/writing-and-serving-policy/policy-style.markdown index b1efaf0b5..baf38520c 100644 --- a/guide/writing-and-serving-policy/policy-style.markdown +++ b/guide/writing-and-serving-policy/policy-style.markdown @@ -408,12 +408,6 @@ bundle agent main } ``` -### Internal variables & classes - -Variables and classes that have no centralized reporting value are considered -"internal". By convention internal variables and classes should be prefixed with -an underscore "_". - ## Deprecating Bundles As your policy library changes over time you may want to deprecate various bundles in favor of newer implimentations. To indicate that a bundle is @@ -432,7 +426,7 @@ bundle agent old ``` ## Tooling -Currently, there is no canonical policy linting or reformatting tool. There are a few different tools that can be useful apart from an [editor with syntax support](https://cfengine.com/cfengine-code-editors/) for achieving regular formatting. +Currently, there is no canonical policy linting or reformatting tool. There are a few different tools that can be useful apart from an [editor with syntax support][Editors] for achieving regular formatting. ### cf-promises diff --git a/index.markdown b/index.markdown index 53a667f5c..85a568872 100644 --- a/index.markdown +++ b/index.markdown @@ -32,13 +32,13 @@ frequently asked questions (and answers :) *Shortcuts:* -## [Components][Components and Common Control] +## [Components][Components] Cf-agent, cf-serverd, cf-execd, cf-promises, etc. ## [Language concepts][Language Concepts] Bundles, Bodies, Promises, Classes and Decisions, Variables, etc. -## [Promise types (aka resources to be managed)][Promise Types and Attributes] +## [Promise types (aka resources to be managed)][Promise Types] processes, packages, users, files, storage, services, etc ## [Masterfiles framework][Masterfiles Policy Framework] diff --git a/legal.markdown b/legal.markdown index 622eefc8d..e5c341ca1 100644 --- a/legal.markdown +++ b/legal.markdown @@ -1,14 +1,15 @@ --- layout: default -title: Legal and Licenses +title: Legal and licenses published: false sorting: 999 alias: legal.html --- -## General Legal Disclaimer +## General legal disclaimer -Please note that CFEngine is offered on an "as is" basis without warranty of +Please note that unless otherwise noted (through a customer agreement or similar) +CFEngine is offered on an "as is" basis without warranty of any kind, and that our products are not error or bug free. To the maximum extent permitted by applicable law, CFEngine on behalf of itself and its suppliers, disclaims all warranties and conditions, either express or implied, @@ -16,53 +17,102 @@ including, but not limited to, implied warranties of merchantability, fitness for a particular purpose, title and non-infringement with regard to the Licensed Software. -## 3rd Party Licenses and Libraries +## CFEngine documentation license + +The documentation is licensed under a [Creative Commons Attribution-ShareAlike 3.0 Unported License](https://creativecommons.org/licenses/by-sa/3.0/deed.en_US). + +## 3rd party licenses and libraries CFEngine includes the following 3rd party libraries and components: -### Community +### Common dependencies + +These dependencies are used by both CFEngine Community (Open Source) as well as CFEngine Enterprise: + +* [libacl](https://savannah.nongnu.org/projects/acl) under the [LGPL](https://git.savannah.gnu.org/cgit/acl.git/tree/include/acl.h) license +* [libattr](https://savannah.nongnu.org/projects/attr) under the [LGPL](https://git.savannah.gnu.org/cgit/attr.git/tree/include/libattr.h) license +* [libcurl](https://curl.se) under the [MIT/X derivative license](https://curl.se/docs/copyright.html) +* [libiconv](http://ftp.gnu.org/gnu/libiconv/) under the [LGPL](https://git.savannah.gnu.org/gitweb/?p=libiconv.git;a=blob;f=include/iconv.h.in) license +* [libxml2](https://gitlab.gnome.org/GNOME/libxml2/-/wikis/FAQ) under the [MIT license](https://opensource.org/license/mit/) +* [libyaml](https://pyyaml.org/wiki/LibYAML) under the [MIT license](https://github.com/yaml/libyaml/blob/master/License) +* [diffutils](https://ftpmirror.gnu.org/diffutils/) under the [GPLv3](https://git.savannah.gnu.org/cgit/diffutils.git/tree/src/diff.c) +* [LMDB](https://www.symas.com/lmdb) under the [OpenLDAP Public License](https://www.openldap.org/software/release/license.html) +* [OpenSSL](https://www.openssl.org) under the [OpenSSL (OpenSSL 1) or Apache v2 (OpenSSL 3) license](https://www.openssl.org/source/license.html) +* [PCRE](https://www.pcre.org) under the [PCRE license](https://www.pcre.org/licence.txt) or + [PCRE2](https://pcre2project.github.io/pcre2/) under the [PCRE2 + license](https://github.com/PCRE2Project/pcre2/blob/master/LICENCE) +* [PEG](https://piumarta.com/software/peg/) under the [MIT license](https://opensource.org/license/mit/) +* [zlib](https://www.zlib.net) under the [zlib license](https://www.zlib.net/zlib_license.html) + +### Enterprise only dependencies + +In addition to the common dependencies listed above, these dependencies are specific to CFEngine Enterprise: -* [libacl](http://savannah.nongnu.org/projects/acl) under the [LGPL](http://git.savannah.gnu.org/cgit/acl.git/tree/include/acl.h) -* [libattr](http://savannah.nongnu.org/projects/attr) under the [LGPL](http://git.savannah.gnu.org/cgit/attr.git/tree/include/libattr.h) -* [libcrypto](http://www.openssl.org/docs/crypto/crypto.html) under the [LGPL](http://api.libssh.org/master/libcrypto_8h_source.html) -* [libexpat](http://sourceforge.net/projects/expat/) under the [MIT License](http://opensource.org/licenses/mit-license.html) -* [libpam](http://www.linux-pam.org) as per the [copyright notice](https://git.fedorahosted.org/cgit/linux-pam.git/tree/Copyright) -* [libvirt](http://libvirt.org/FAQ.html) under the [LGPL version 2.1](http://www.opensource.org/licenses/lgpl-license.html) -* [libxml2](http://xmlsoft.org/FAQ.html) under the [MIT license](http://opensource.org/licenses/mit-license.html) -* [LMDB](http://symas.com/mdb/) under the [OpenLDAP Public License](http://www.openldap.org/software/release/license.html) -* [OpenSSL](http://www.openssl.org) under the [OpenSSL license](http://www.openssl.org/source/license.html) -* [PCRE](http://www.pcre.org) under the [PCRE license](http://www.pcre.org/licence.txt) -* [PEG](http://piumarta.com/software/peg/) under the MIT license -* [QDBM](http://sourceforge.net/projects/qdbm/) under the [GNU Library or Lesser General Public License 2.0 (LGPLv2)](http://www.opensource.org/licenses/lgpl-license.html) -* [TokyoCabinet](http://fallabs.com/tokyocabinet/) under the [GNU Lesser General Public License](http://www.opensource.org/licenses/lgpl-license.html) -* [Zlib](http://www.zlib.net) under the [zlib license](http://www.zlib.net/zlib_license.html) +* [Angular.js](https://angularjs.org) under the [MIT license](https://github.com/angular/angular.js/blob/master/LICENSE) +* [Apache](https://httpd.apache.org) under the [Apache License, Version 2.0](https://www.apache.org/licenses/LICENSE-2.0) +* [APR and APR-util](https://apr.apache.org) under the [Apache License, Version 2.0](https://www.apache.org/licenses/LICENSE-2.0) +* [Chosen](https://harvesthq.github.io/chosen/) under the [MIT license](https://github.com/harvesthq/chosen/blob/master/LICENSE.md) +* [CodeIgniter](https://github.com/bcit-ci/CodeIgniter/) under the [MIT license](https://github.com/bcit-ci/CodeIgniter/blob/develop/license.txt) +* [Disphelper](https://disphelper.sourceforge.net) (only Windows) under the [BSD license](https://opensource.org/licenses/bsd-license.php) +* [Flot](https://www.flotcharts.org/) under the [MIT license](https://github.com/flot/flot/blob/master/LICENSE.txt) +* [Font Awesome](https://fontawesome.com/) by Dave Gandy - https://fontawesome.io/license/ +* [git](https://git-scm.com) under the [GNU General Public License, version 2 (GPLv2)](https://opensource.org/licenses/GPL-2.0) +* [Glyphicons](https://glyphicons.com/license/) under [Creative Commons Attribution 3.0 Unported (CC BY 3.0)](https://creativecommons.org/licenses/by-sa/3.0/deed.en_US) +* [HighCharts](https://www.highcharts.com/) under the [OEM license by HighSoft](https://shop.highcharts.com/) +* [jQuery](https://jquery.com/) under the [MIT license](https://opensource.org/license/mit/) +* [libexpat](https://sourceforge.net/projects/expat/) under the [MIT license](https://opensource.org/license/mit/) +* [libgnurx](http://www.gnu.org/software/rx/rx.html) under the [LGPLv2.1](https://github.com/TimothyGu/libgnurx/blob/libgnurx-2.5.1/regex.h) license +* [mod_ssl](https://httpd.apache.org/docs/2.4/mod/mod_ssl.html) under a [BSD style license](http://www.modssl.org/docs/2.8/ssl_overview.html) +* [oauth2-server-php](https://github.com/bshaffer/oauth2-server-php) under the [MIT license](https://github.com/bshaffer/oauth2-server-php/blob/develop/LICENSE) +* [OpenLDAP and liblber](https://www.openldap.org) under the [OpenLDAP Public License](https://www.openldap.org/software/release/license.html) +* [PHP](https://php.net) under the [PHP license](https://www.php.net/license/3_01.txt) +* [PostgreSQL](https://www.postgresql.org) under the [PostgreSQL License](https://opensource.org/licenses/postgresql) +* [rsync](https://rsync.samba.org) under the [GPLv3](https://rsync.samba.org/GPL.html) +* [Twitter Bootstrap Framework](https://getbootstrap.com) under [Apache License, Version 2.0](https://www.apache.org/licenses/LICENSE-2.0) +* [Bootstrap Icons](https://icons.getbootstrap.com) under the [MIT license](https://github.com/twbs/icons/blob/main/LICENSE) +* [underscore.js](https://underscorejs.org) under the [MIT license](https://opensource.org/license/mit/) +* [ace-builds](https://github.com/ajaxorg/ace-builds) under the [BSD-3-Clause](https://github.com/ajaxorg/ace-builds/blob/master/LICENSE) license +* [angular](http://angularjs.org) under the [MIT](https://github.com/angular/angular.js/blob/master/LICENSE) license +* [angular-chosen-localytics](http://github.com/leocaseiro/angular-chosen) under the [MIT](https://github.com/leocaseiro/angular-chosen/blob/master/LICENSE) license +* [angular-daterangepicker](https://github.com/fragaria/angular-daterangepicker) under the [MIT](https://github.com/fragaria/angular-daterangepicker/blob/master/LICENSE.md) license +* [angular-ui](https://github.com/buildium/angular-ui) under the [MIT](https://github.com/buildium/angular-ui/blob/master/LICENSE) license +* [bootstrap-multiselect](http://davidstutz.github.io/bootstrap-multiselect/) under the [Apache License, Version 2.0](http://davidstutz.github.io/bootstrap-multiselect/#license) +* [bootstrap-tour](http://bootstraptour.com) under the [MIT](https://github.com/sorich87/bootstrap-tour/blob/master/LICENSE) license +* [chosen-js](https://harvesthq.github.io/chosen/) under the [MIT](https://github.com/harvesthq/chosen/blob/master/LICENSE.md) license +* [clipboard](https://clipboardjs.com) under the [MIT](https://github.com/zenorocha/clipboard.js/blob/master/LICENSE) license +* [datatables](http://datatables.net) under the [MIT](https://datatables.net/license/mit) license +* [daterangepicker](https://github.com/dangrossman/daterangepicker) under the [MIT](https://github.com/dangrossman/daterangepicker/blob/master/README.md#license) license +* [google-code-prettify](https://www.npmjs.com/package/google-code-prettify) under the [Apache License 2.0](https://github.com/googlearchive/code-prettify/blob/master/COPYING) +* [highcharts](http://www.highcharts.com) under the [SLA](https://shop.highcharts.com/license) license +* [html5shiv](https://github.com/aFarkas/html5shiv#readme) under the [MIT license and GPL2](https://github.com/aFarkas/html5shiv/blob/master/MIT%20and%20GPL2%20licenses.md) +* [ip-subnet-calculator](https://github.com/franksrevenge/IPSubnetCalculator) under the [MIT](https://github.com/salieri/IPSubnetCalculator/blob/master/LICENSE) license +* [jquery](https://jquery.com) under the [MIT](https://github.com/salieri/IPSubnetCalculator/blob/master/LICENSE) license +* [jquery-appear-original](https://github.com/morr/jquery.appear) under the [MIT](https://github.com/morr/jquery.appear/blob/master/LICENSE) license +* [jquery-form](https://github.com/jquery-form/form) under the [MIT](https://github.com/jquery-form/form/blob/master/LICENSE) license +* [jquery-multiselect](https://github.com/techhysahil/jquery-MultiSelect) under the [MIT](https://github.com/techhysahil/jquery-MultiSelect/blob/master/LICENSE) license +* [jquery-ui-timepicker-addon](http://trentrichardson.com/examples/timepicker) under the [MIT](https://github.com/trentrichardson/jQuery-Timepicker-Addon?tab=License-1-ov-file) license +* [jquery-validation](https://jqueryvalidation.org/) under the [MIT](https://github.com/jquery-validation/jquery-validation/blob/master/LICENSE.md) license +* [jquery-wheelcolorpicker](https://raffer.one/projects/jquery-wheelcolorpicker) under the [MIT](https://github.com/fujaru/jquery-wheelcolorpicker/blob/master/LICENSE) license +* [jquery.cookie](https://github.com/carhartl/jquery-cookie) under the [MIT](https://github.com/carhartl/jquery-cookie/blob/master/MIT-LICENSE.txt) license +* [jquery.flot](https://www.npmjs.com/package/jquery.flot) under the [MIT](https://github.com/flot/flot/blob/master/LICENSE.txt) license +* [json2](http://github.com/SamuraiJack/JSON2/tree) under the [GNU Lesser General Public License](https://github.com/canonic-epicure/JSON2/blob/master/README.md#copyright-and-license) +* [jstimezonedetect](https://github.com/pellepim/jstimezonedetect) under the [MIT](https://github.com/pellepim/jstimezonedetect/blob/master/LICENCE.txt) license +* [notifyjs](https://notifyjs.jpillora.com/) under the [MIT](https://github.com/jpillora/notifyjs/blob/master/LICENSE) license +* [pluralize](https://github.com/blakeembrey/pluralize) under the [MIT](https://github.com/plurals/pluralize/blob/master/LICENSE) license +* [zxcvbn](https://github.com/dropbox/zxcvbn) under the [MIT](https://github.com/dropbox/zxcvbn/blob/master/LICENSE.txt) license +* [FPDF](http://www.fpdf.org/) under the [permissive license](https://github.com/Setasign/FPDF/blob/master/license.txt) +* [guzzle](https://docs.guzzlephp.org/) under the [MIT](https://docs.guzzlephp.org/en/stable/overview.html#license) license +* [tcpdf](https://tcpdf.org/) under the [GNU LESSER GENERAL PUBLIC LICENSE](https://tcpdf.org/docs/license/) +* [Slim](https://www.slimframework.com/) under the [MIT](https://github.com/slimphp/Slim/blob/4.x/LICENSE.md) +* [monolog](https://seldaek.github.io/monolog/) under the [MIT](https://github.com/Seldaek/monolog/blob/master/LICENSE) +* [LdapRecord](https://ldaprecord.com/) under the [MIT](https://github.com/DirectoryTree/LdapRecord/blob/master/license.md) +* [phpseclib](https://phpseclib.com/) under the [MIT](https://github.com/phpseclib/phpseclib/blob/master/LICENSE) +* [tonic](http://peej.github.com/tonic/) under the [MIT](https://github.com/peej/tonic/blob/master/LICENSE) -### Enterprise +### Optional, non-default dependencies -* [Angular.js](https://angularjs.org) through [MIT Licence](https://github.com/angular/angular.js/blob/master/LICENSE) -* [Apache](http://httpd.apache.org) under the [Apache License, Version 2.0](http://www.apache.org/licenses/LICENSE-2.0) -* [APR and APR-util](https://apr.apache.org) under the [Apache License, Version 2.0](http://www.apache.org/licenses/LICENSE-2.0) -* [Chosen](http://harvesthq.github.io/chosen/) under the [MIT License](https://github.com/harvesthq/chosen/blob/master/LICENSE.md) -* [CodeIgniter](http://codeigniter.com/) under the [CodeIgniter License Agreement](http://ellislab.com/codeigniter/user-guide/license.html) -* [Disphelper](http://disphelper.sourceforge.net) (only Windows) under the [BSD](http://opensource.org/licenses/bsd-license.php) -* [Flot](http://www.flotcharts.org/) under a [permissive license](https://github.com/flot/flot/blob/master/LICENSE.txt) -* [Font Awesome](http://fontawesome.io) by Dave Gandy - http://fontawesome.io/license/ -* [git](http://git-scm.com) under the [GNU General Public License, version 2 (GPLv2)](http://opensource.org/licenses/GPL-2.0) -* [Glyphicons](http://glyphicons.com/license/) under [Creative Commons Attribution 3.0 Unported (CC BY 3.0)](http://creativecommons.org/licenses/by-sa/3.0/deed.en_US) -* [HighCharts](http://www.highcharts.com/) under the OEM license by HighSoft -* [jQuery](https://jquery.org) under the [MIT license](http://en.wikipedia.org/wiki/MIT_License) -* [libcurl](http://curl.haxx.se) under the [MIT/X derivative license](http://curl.haxx.se/docs/copyright.html) -* [libmcrypt](http://mcrypt.sourceforge.net) under the [LGPLv2](http://www.opensource.org/licenses/lgpl-license.html) -* [mod_ssl](http://www.modssl.org) under a BSD style license -* [oauth2-server-php](https://github.com/bshaffer/oauth2-server-php) under the [MIT License](https://github.com/bshaffer/oauth2-server-php/blob/develop/LICENSE) -* [OpenLDAP and liblber](http://www.openldap.org) under the [OpenLDAP Public License](http://www.openldap.org/software/release/license.html) -* [PHP](http://php.net) under the [PHP license version 3.01](http://www.php.net/license/3_01.txt) -* [php-apc](http://pecl.php.net/package/APC) under the [PHP License](http://www.php.net/license/3_01.txt) -* [PostgreSQL, libecpg and libecpg_compat](http://www.postgresql.org) under the [PostgreSQL License](http://opensource.org/licenses/postgresql) -* [redis](http://redis.io) under the [three clause BSD license](http://redis.io/topics/license) -* [rsync](http://rsync.samba.org) under the [GPLv3](http://rsync.samba.org/GPL.html) -* [Twitter Bootstrap Framework](http://getbootstrap.com) under [Apache 2.0](http://www.apache.org/licenses/LICENSE-2.0) -* [Bootstrap Icons](https://icons.getbootstrap.com) under the [MIT license](https://github.com/twbs/icons/blob/main/LICENSE.md) -* [underscore.js](http://underscorejs.org) under the MIT license +These dependencies are not a part of the packages we build and distribute, but specific users or customers may build CFEngine with support for custom functionality and with custom software dependencies: - +* [libvirt](https://libvirt.org/) under the [LGPL version 2.1](https://www.opensource.org/licenses/lgpl-license.html) +* [QDBM](https://sourceforge.net/projects/qdbm/) under the [GNU Library or Lesser General Public License 2.0 (LGPLv2)](https://opensource.org/license/lgpl-2-1/) +* [TokyoCabinet](https://github.com/hthetiot/Tokyo-Cabinet) under the [GNU Lesser General Public License](https://www.opensource.org/licenses/lgpl-license.html) diff --git a/reference.markdown b/reference.markdown index a8a4c022f..bed1d89c4 100644 --- a/reference.markdown +++ b/reference.markdown @@ -10,8 +10,8 @@ components, bodies, functions, variables, classes and attributes in detail. Language elements that belong together are typically documented on the same page. -* [Components and Common Control][Components and Common Control] -* [Promise Types and Attributes][Promise Types and Attributes] +* [Components][Components] +* [Promise Types][Promise Types] * [Functions][Functions] * [Language Concepts][Language Concepts] * [Special Variables][Special Variables] diff --git a/reference/common-attributes-include.markdown b/reference/common-attributes-include.markdown index 2e64fbd36..968a73d8b 100644 --- a/reference/common-attributes-include.markdown +++ b/reference/common-attributes-include.markdown @@ -1,23 +1,23 @@ ### Common Attributes Common attributes are available to all promise types. Full details for common -attributes can be found in the [Common Promise Attributes section][Promise Types and Attributes#Common Promise Attributes] of -the [Promise Types and Attributes] page. The common attributes are as follows: +attributes can be found in the [Common Promise Attributes section][Promise Types#Common Promise Attributes] of +the [Promise Types] page. The common attributes are as follows: -#### [action][Promise Types and Attributes#action] +#### [action][Promise Types#action] -#### [classes][Promise Types and Attributes#classes] +#### [classes][Promise Types#classes] -#### [comment][Promise Types and Attributes#comment] +#### [comment][Promise Types#comment] -#### [depends_on][Promise Types and Attributes#depends_on] +#### [depends_on][Promise Types#depends_on] -#### [handle][Promise Types and Attributes#handle] +#### [handle][Promise Types#handle] -#### [if][Promise Types and Attributes#if] +#### [if][Promise Types#if] -#### [meta][Promise Types and Attributes#meta] +#### [meta][Promise Types#meta] -#### [with][Promise Types and Attributes#with] +#### [with][Promise Types#with]
    diff --git a/reference/common-body-attributes-include.markdown b/reference/common-body-attributes-include.markdown index 6c3e10e8e..7dabe368e 100644 --- a/reference/common-body-attributes-include.markdown +++ b/reference/common-body-attributes-include.markdown @@ -2,12 +2,12 @@ Common body attributes are available to all body types. Full details for common body attributes can be found in the -[Common Body Attributes section][Promise Types and Attributes#Common Body Attributes] -of the [Promise Types and Attributes] page. The common attributes are as +[Common Body Attributes section][Promise Types#Common Body Attributes] +of the [Promise Types] page. The common attributes are as follows: -##### [inherit_from][Promise Types and Attributes#inherit_from] +##### [inherit_from][Promise Types#inherit_from] -##### [meta][Promise Types and Attributes#meta] +##### [meta][Promise Types#meta]
    diff --git a/reference/components.markdown b/reference/components.markdown index 53450dabe..a6e67526c 100644 --- a/reference/components.markdown +++ b/reference/components.markdown @@ -1,6 +1,6 @@ --- layout: default -title: Components and Common Control +title: Components published: true sorting: 10 tags: [Reference, Components] @@ -149,7 +149,7 @@ facilities. **Note:** Bandwidth limiting is currently not supported on Windows. -**Type:** [`float`][float] +**Type:** `float` **Default value:** none (no limit) @@ -193,7 +193,11 @@ runs of e.g. `cf-agent` and `cf-promises`. cache_system_functions => "true"; ``` -**History:** Was introduced in version 3.6.0. +**See also:** [`ifelapsed` in action bodies][Promise Types#ifelapsed] + +**History:** +- Introduced in version 3.6.0. + ### domain @@ -426,22 +430,18 @@ body common control **Description:** Defines the protocol to use for all outgoing connections. -[%CFEngine_promise_attribute(2)%] - -* 0 - -* undefined - -* 1 - Classic protocol -* classic - Alias to protocol 1 -* 2 - TLS -* latest - Alias to protocol 2 +[%CFEngine_promise_attribute(undefined)%] **Note:** `protocol_version` can be specified at the individual promise level using the [`body copy_from protocol_version`][files#protocol_version] -attribute. +attribute. When undefined (the default) peers automatically negotiate the latest protocol version. + +**See also:** [`body copy_from protocol_version`][files#protocol_version], `allowlegacyconnects`, [`allowtlsversion`][cf-serverd#allowtlsversion], [`allowciphers`][cf-serverd#allowciphers], [`tls_min_version`][Components#tls_min_version], [`tls_ciphers`][Components#tls_ciphers], [`encrypt`][files#encrypt], [`logencryptedtransfers`][cf-serverd#logencryptedtransfers], [`ifencrypted`][access#ifencrypted] -**See also:** [`body copy_from protocol_version`][files#protocol_version], `allowlegacyconnects`, [`allowtlsversion`][cf-serverd#allowtlsversion], [`allowciphers`][cf-serverd#allowciphers], [`tls_min_version`][Components and Common Control#tls_min_version], [`tls_ciphers`][Components and Common Control#tls_ciphers], [`encrypt`][files#encrypt], [`logencryptedtransfers`][cf-serverd#logencryptedtransfers], [`ifencrypted`][access#ifencrypted] +**History:** -**History:** Introduced in CFEngine 3.6.0 +* Introduced in CFEngine 3.6.0 with `protocol_version` `1` (`classic`) and `protocol_version` `2` (`tls`) +* Added `protocol_version` `3` (`cookie`) in CFEngine 3.15.0 ### require_comments @@ -550,9 +550,38 @@ components may promise to send data. } ``` +### system_log_level + +**Description:** The minimum log level required for log messages to go to the system log (e.g. syslog, Windows Event Log). + +**Type:** `string` + +**Allowed Input range:** `(critical|error|warning|notice|info)` + +**Default value:** ` ` (unspecified) + +**Example:** + +Prevent messages lower than _critical_ on Windows. + +```cf3 +body common control +{ +@if minimum_version(3.18.1) + windows:: + system_log_level => "critical"; + cfengine:: +@endif +} +``` + +**History:** + +* Introduced in 3.19.0, 3.18.1 + ### tls_ciphers -**Description:** List of ciphers allowed when making **outgoing** connections. +**Description:** List of ciphers allowed when making **outgoing** connections from components other than `cf-serverd`. For a list of possible ciphers, see man page for "openssl ciphers". @@ -568,13 +597,13 @@ body common control } ``` -**See also:** [`protocol_version`][Components and Common Control#protocol_version], [`allowciphers`][cf-serverd#allowciphers], [`tls_min_version`][Components and Common Control#tls_min_version], [`allowtlsversion`][cf-serverd#allowtlsversion], [`encrypt`][files#encrypt], [`logencryptedtransfers`][cf-serverd#logencryptedtransfers], [`ifencrypted`][access#ifencrypted] +**See also:** [`protocol_version`][Components#protocol_version], [`allowciphers`][cf-serverd#allowciphers], [`tls_min_version`][Components#tls_min_version], [`allowtlsversion`][cf-serverd#allowtlsversion], [`encrypt`][files#encrypt], [`logencryptedtransfers`][cf-serverd#logencryptedtransfers], [`ifencrypted`][access#ifencrypted] **History:** Introduced in CFEngine 3.7.0 ### tls_min_version -**Description:** Minimum tls version to allow for **outgoing** connections. +**Description:** Minimum tls version to allow for **outgoing** connections from components other than `cf-serverd`. [%CFEngine_promise_attribute(1.0)%] @@ -586,7 +615,7 @@ body common control } ``` -**See also:** [`protocol_version`][Components and Common Control#protocol_version], [`allowciphers`][cf-serverd#allowciphers], [`tls_ciphers`][Components and Common Control#tls_ciphers], [`allowtlsversion`][cf-serverd#allowtlsversion], [`encrypt`][files#encrypt], [`ifencrypted`][access#ifencrypted], [`logencryptedtransfers`][cf-serverd#logencryptedtransfers] +**See also:** [`protocol_version`][Components#protocol_version], [`allowciphers`][cf-serverd#allowciphers], [`tls_ciphers`][Components#tls_ciphers], [`allowtlsversion`][cf-serverd#allowtlsversion], [`encrypt`][files#encrypt], [`ifencrypted`][access#ifencrypted], [`logencryptedtransfers`][cf-serverd#logencryptedtransfers] **History:** Introduced in CFEngine 3.7.0 diff --git a/reference/components/cf-agent.markdown b/reference/components/cf-agent.markdown index a44b924aa..0345ef5f6 100644 --- a/reference/components/cf-agent.markdown +++ b/reference/components/cf-agent.markdown @@ -9,7 +9,7 @@ keywords: [agent] `cf-agent` evaluates policy code and makes changes to the system. Policy bundles are evaluated in the order of the provided `bundlesequence` (this is normally specified in the -[`common control body`][Components and Common Control#Common Control]). For +[`common control body`][Components#Common Control]). For each bundle, `cf-agent` groups promise statements according to their type. Promise types are then evaluated in a preset order to ensure fast system convergence to policy. @@ -17,10 +17,61 @@ convergence to policy. `cf-agent` keeps the promises made in `common` and `agent` bundles, and is affected by `common` and `agent` control bodies. +**Notes:** + +* `cf-agent` always considers the class ```agent``` to be defined. + ## Command reference ## [%CFEngine_include_snippet(cf-agent.help, [\s]*--[a-z], ^$)%] +### --simulate + +Like the `--dry-run` option, the `--simulate` option tries to identify changes +to your system without making changes to the system, however it goes further +than `--dry-run` by making changes in a `chroot` and making a distinction +between *safe* and *unsafe* functions, e.g. `execresult()`. + +The agent will execute promises with unsafe functions when the `--simulate` +options is given only if the promise using the function is tagged `simulate_safe`. + +For example: + +```cf3 +bundle agent __main__ +{ + vars: + "msg" + string => execresult( "/bin/echo Hello world!", "useshell" ), + meta => { "simulate_safe" }; + +} +``` + +The simulate option takes a parameter, `diff`, `manifest`, or `manifest-full` +which is used to determine the summary output shown at the end of the run. + +* `diff` - Show only things that changed during the simulated run. +* `manifest` - Show files and packages changed by the simulated run. +* `manifest-full` - Show all files evaluated by the simulated run (including unchanged ones) + + - cf-agent can now simulate the changes done to files in a chroot, printing + diff or manifest information about what it would do in a normal evaluation. + Use the new command line option: `--simulate=diff` or `--simulate=manifest`. + Please note that only files and packages promises are simulated currently. + + - Added a new --simulate=manifest-full mode + New simulation mode that manifests all changed files as well as + all other files evaluated by the agent run which were not skipped + (by file selection rules) (CFE-3506) + +#### Notes +* Supported on Linux for `files` and `packages` type promises + +#### History +* Introduced in version 3.17.0 +* `--simulate=manifest-full` introduced in version 3.18.0 + ## Automatic Bootstrapping Automatic bootstrapping allows the user to connect a CFEngine Host to a Policy @@ -715,7 +766,7 @@ kill and restart its attempt to keep a promise. } ``` -**See also:** [`body action expireafter`][Promise Types and Attributes#expireafter], [`body contain exec_timeout`][commands#exec_timeout], [`body executor control agent_expireafter`][cf-execd#agent_expireafter] +**See also:** [`body action expireafter`][Promise Types#expireafter], [`body contain exec_timeout`][commands#exec_timeout], [`body executor control agent_expireafter`][cf-execd#agent_expireafter] ### files_single_copy @@ -831,7 +882,14 @@ another which is not tied to a specific time. } ``` -**See also:** [Promise locking][Promises#Promise Locking], [ifelapsed action body attribute][Promise Types and Attributes#ifelapsed] +**Notes:** + +* A value of `0` means no locking, all promises will be executed each execution if in context. This also disables function caching. +* This is not a reliable way to control frequency over a long period of time. +* Locks provide simple but weak frequency control. +* Locks older than 4 weeks are automatically purged. + +**See also:** [Promise locking][Promises#Promise locking], [ifelapsed action body attribute][Promise types#ifelapsed] ### inform @@ -891,7 +949,7 @@ diminishing returns. } ``` -**See also:** [`background` in action bodies][Promise Types and Attributes#background] +**See also:** [`background` in action bodies][Promise Types#background] ### maxconnections diff --git a/reference/components/cf-execd.markdown b/reference/components/cf-execd.markdown index 50fe8a2e9..a9148e8d6 100644 --- a/reference/components/cf-execd.markdown +++ b/reference/components/cf-execd.markdown @@ -18,7 +18,11 @@ network. `cf-execd` keeps the promises made in `common` bundles, and is affected by `common` and `executor` control bodies. -**Note:** This daemon reloads it's config when the SIGHUP signal is received. +**Notes:** + +* This daemon reloads it's config when the SIGHUP signal is received. +* `cf-execd` always considers the class ```executor``` to be defined. + **History:** @@ -34,14 +38,14 @@ These body settings determine the behavior of `cf-execd`,including scheduling times and output capture to `WORKDIR/outputs` and relay via email. ```cf3 - body executor control - { - splaytime => "5"; - mailto => "cfengine@example.org"; - mailfrom => "cfengine@$(host).example.org"; - smtpserver => "localhost"; - schedule => { "Min00_05", "Min30_35" } - } +body executor control +{ + splaytime => "5"; + mailto => "cfengine@example.org"; + mailfrom => "cfengine@$(host).example.org"; + smtpserver => "localhost"; + schedule => { "Min00", "Min30" } +} ``` @@ -82,7 +86,7 @@ set it to `120` and you are using a 5-minute agent schedule, a maximum of 120 / 5 = 24 agents should be enforced. -**See also:** [`body action expireafter`][Promise Types and Attributes#expireafter], [`body contain exec_timeout`][commands#exec_timeout], [`body agent control expireafter`][cf-agent#expireafter] +**See also:** [`body action expireafter`][Promise Types#expireafter], [`body contain exec_timeout`][commands#exec_timeout], [`body agent control expireafter`][cf-agent#expireafter] ### executorfacility @@ -303,10 +307,10 @@ function may be affected by changing the `schedule`. **Example:** ```cf3 - body executor control - { - schedule => { "Min00", "(Evening|Night).Min15_20", "Min30", "(Evening|Night).Min45_50" }; - } +body executor control +{ +schedule => { "Min00", "(Evening|Night).Min15", "Min30", "(Evening|Night).Min45" }; +} ``` ### smtpserver @@ -371,3 +375,57 @@ The CFEngine default policy sets `splaytime` to 1. **See also:** The [`splayclass()`][splayclass] function for a task-specific means for setting splay times. + +### runagent_socket_allow_users + +**Description:** Users who are allowed access the socket (`STATEDIR/cf-execd.sockets/runagent.socket`). + +**Type:** `slist` + +**Allowed input range:** `.*` + +**Default value:** none + +**Notes:** + +* By default, in the Masterfiles Policy Framework, `cfapache` is allowed to access the socket on Enterprise Hubs. + +**Example:** + +```cf3 +body executor control +{ + runagent_socket_allow_users => { "yoda", "obi-wan" }; +} +``` + +**See also:** [`cf-runagent`][cf-runagent] + +**History:** + +* 3.18.0 Added `runagent_socket_allow_users` attribute + +## Sockets + +`cf-execd` creates `STATEDIR/cf-execd.sockets/runagent.socket` (`/var/cfengine/state/cf-execd.sockets/runagent.socket`). + +The `body executor control` attribute `runagent_socket_allow_users` controls the list of users that should be allowed to access (**RW**) the socket via ACLs. + +**Notes:** + +* Unlike execution triggered with the `cf-runagent` binary, there is currently no capability to define additional options like defining additional classes, or the remote bundlesequence. + +**Example:** + +Write the name or IP into the socket to request unscheduled execution on that host: + +```console +echo 'host001' > /var/cfengine/state/cf-execd.sockets/cf-runagent.socket +``` + +**See also:** [`cf-runagent`][cf-runagent], [`runagent_socket_allow_users`][cf-execd#runagent_socket_allow_users] + +**History:** + +* 3.18.0 Added socket for triggering `cf-runagent` by hostname or IP. + diff --git a/reference/components/cf-hub.markdown b/reference/components/cf-hub.markdown index a528f629f..6e87413f9 100644 --- a/reference/components/cf-hub.markdown +++ b/reference/components/cf-hub.markdown @@ -21,6 +21,10 @@ that have registered a connection with a collocated `cf-serverd` get if you ran `cf-agent` without specifying a file name. This is to avoid reporting on data generated by test or extraordinary executions. +**Notes:** + +* `cf-hub` always considers the class ```hub``` to be defined. + ## Command reference ## [%CFEngine_include_snippet(cf-hub.help, [\s]*--[a-z], ^$)%] @@ -78,9 +82,9 @@ body hub control { # Collect reports every at the top and half of the hour. Additionally collect - # reports during the evening or night between Minute 45 and 50. + # reports during the evening or night at Minute 45. - hub_schedule => { "Min00", "Min30", "(Evening|Night).Min45_50" }; + hub_schedule => { "Min00", "Min30", "(Evening|Night).Min45" }; } ``` diff --git a/reference/components/cf-key.markdown b/reference/components/cf-key.markdown index f3eaba9a8..52c79d157 100644 --- a/reference/components/cf-key.markdown +++ b/reference/components/cf-key.markdown @@ -6,8 +6,11 @@ sorting: 60 tags: [Components, cf-key] --- -The CFEngine key generator makes key pairs for [remote -authentication][Networking]. +The CFEngine key generator makes key pairs for [remote authentication][Client server communication]. + +**Notes:** + +* `cf-key` always considers the class ```keygenerator``` to be defined. ## Command reference ## diff --git a/reference/components/cf-monitord.markdown b/reference/components/cf-monitord.markdown index c8723b6c1..df07dfc7c 100644 --- a/reference/components/cf-monitord.markdown +++ b/reference/components/cf-monitord.markdown @@ -17,6 +17,10 @@ policy decisions. `cf-monitord` keeps the promises made in `common`and `monitor` bundles, and is affected by `common` and `monitor` control bodies. +**Notes:** + +* `cf-monitord` always considers the class ```monitor``` to be defined. + ## Command reference ## [%CFEngine_include_snippet(cf-monitord.help, [\s]*--[a-z], ^$)%] @@ -81,6 +85,32 @@ currently has less support for out-of-the-box probes. 52. cpu1: %CPU utilization core 1 53. cpu2: %CPU utilization core 2 54. cpu3: %CPU utilization core 3 +55. microsoft\_ds\_out: Samba/MS\_ds name sessions (out) +56. www\_alt\_in: Alternative web service connections (in) +57. www\_alt\_out: Alternative web client connections (out) +58. imaps\_in: encrypted imap mail service sessions (in) +59. imaps\_out: encrypted imap mail client sessions (out) +60. ldap\_in: LDAP directory service service sessions (in) +61. ldap\_out: LDAP directory service client sessions (out) +62. ldaps\_in: LDAP directory service service sessions (in) +63. ldaps\_out: LDAP directory service client sessions (out) +64. mongo\_in: Mongo database service sessions (in) +65. mongo\_out: Mongo database client sessions (out) +66. mysql\_in: MySQL database service sessions (in) +67. mysql\_out: MySQL database client sessions (out) +68. postgres\_in: PostgreSQL database service sessions (in) +69. postgres\_out: PostgreSQL database client sessions (out) +70. ipp\_in: Internet Printer Protocol (in) +71. ipp\_out: Internet Printer Protocol (out) +72. io\_reads: Number of I/O reads +73. io\_writes: Number of I/O writes +74. io\_readdata: Aggregate mount of data read across all devices +75. io\_writtendata: Aggregate amount of data written across all devices +76. mem\_total: Total system memory +77. mem\_free: Free system memory +78. mem\_cached: Size of disk cache +79. mem\_swap: Total swap size +80. mem\_freeswap: Free swap size Slots with a higher number are used for custom measurement promises in CFEngine Enterprise. @@ -90,6 +120,43 @@ into agent variables in the `$(mon.`name`)` context. Note: There is no way for force a refresh of the monitored data. +## Data storage ## + +`cf-monitord` records data in `$(sys.statedir)` (typically `/var/cfengine/state`). + +* `cf_observations.lmdb` +* `nova_measures.lmdb` +* `ts_key` +* `env_data` +* `cf_incoming.` +* `cf_outgoing.` +* `cf_state.lmdb` +* `history.lmdb` + +## Statistical Classes + +`cf-monitord` automatically defines classes based on the observation of the data +is has collected. Classes defined are named for the measurement id (the promise +handle in the case of custom measurement promises) with prefixes and or suffixes +depending on the measurement. + +The following suffixes may be used when defining classes: + +* `_high` :: The last measurement seemed high. It was greater than the average of all time and also greater than the recent average. This could indicate that the measured value is experiencing a "spike" or trending in a positive direction. +* `_low` :: The last measurement was low. It was lower than the average of all time and also lower than the recent average. This could indicate that the measured value is experiencing a "dip" or trending in a negative direction. +* `_normal` :: The value was neither high nor low, (as per how those are described above). +* `_ldt` :: A leap (step) detected, meaning a distinct (significant) change in the average. +* `_dev1` :: The last measurement was at least 1 standard deviation higher/lower than the average. +* `_dev2` :: The last measurement was at least 2 standard deviations higher/lower than the average. These classes are persistently defined for a number of minutes. +* `_anomaly` :: The last measurement was at least 3 standard deviations than the average. These classes are persistently defined for a number of minutes. +* `_microanomaly` :: The last measurement was at least 2 standard deviations higher than the average. + +The following prefixes may be used when defining classes: + +* `entropy_` :: + +Note: These suffixes and prefixes may be combined, resulting in a class like `rootprocs_high`, `loadavg_high_ldt`, `cpu1_high_dev3`, and `entropy_postgresql_out_low`. + ## Control Promises Settings describing the details of the fixed behavioral promises diff --git a/reference/components/cf-reactor.markdown b/reference/components/cf-reactor.markdown new file mode 100644 index 000000000..fd90ef6a1 --- /dev/null +++ b/reference/components/cf-reactor.markdown @@ -0,0 +1,30 @@ +--- +layout: default +title: cf-reactor +published: true +tags: [Components, cf-reactor, Enterprise] +keywords: [reactor] +--- + +`cf-reactor` is the CFEngine event reaction daemon, it lists to `NOTIFY` events +on the `cmdb_refresh`a PostgreSQL channel and upon a message received, it +refreshes the CMDB data file (`host_specific.json`) for the particular host. + +**Notes:** + +* `cf-reactor` is a CFEngine Enterprise hub specific component. + +* Unlike other components there is no control body for `cf-reactor`, all + promises are hard coded within the component. + +* In the future, the daemon should also take care of inventory refresh for hosts + (now part of `cf-hub`) and many DB maintenance tasks that are now promises in + the Masterfiles Policy Framework policy under `/cfe_internal/enterprise`. + +**History:** + +* 3.18.2, 3.20.0 Introduced new component (`cf-reactor`). + +## Command reference ## + +[%CFEngine_include_snippet(cf-reactor.help, [\s]*--[a-z], ^$)%] diff --git a/reference/components/cf-runagent.markdown b/reference/components/cf-runagent.markdown index 16384bdba..d33802c1d 100644 --- a/reference/components/cf-runagent.markdown +++ b/reference/components/cf-runagent.markdown @@ -15,6 +15,10 @@ which hosts `cf-agent` will be started, and classes that the user requests `cf-agent` should define on execution. The latter type is regulated by `cf-serverd`'s [role based access control][roles]. +**Notes:** + +* `cf-runagent` always considers the class ```runagent``` to be defined. + ## Command reference ## [%CFEngine_include_snippet(cf-runagent.help, [\s]*--[a-z], ^$)%] @@ -290,3 +294,23 @@ body runagent control ``` **See also:** [body `copy_from` timeout][files#timeout], [agent `default_timeout`][cf-agent#default_timeout] + +## Sockets + +`cf-runagent` can be triggered by writing a hostname or IP into a socket provided by `cf-execd`. + +**Notes:** + +* Unlike execution triggered with the `cf-runagent` binary, there is currently no capability to define additional options like defining additional classes, or the remote bundlesequence. + +**Example:** + +```console +echo 'host001' > /var/cfengine/state/cf-execd.sockets/cf-runagent.socket +``` + +**See also:** [`cf-execd`][cf-execd], [`runagent_socket_allow_users`][cf-execd#runagent_socket_allow_users] + +**History:** + +* 3.18.0 Added socket for triggering `cf-runagent` by hostname or IP. diff --git a/reference/components/cf-serverd.markdown b/reference/components/cf-serverd.markdown index c787334d5..eff628c07 100644 --- a/reference/components/cf-serverd.markdown +++ b/reference/components/cf-serverd.markdown @@ -18,11 +18,18 @@ requests. `cf-serverd` keeps the promises made in `common` and `server` bundles, and is affected by `common` and `server` control bodies. -**Note:** This daemon reloads it's config when the SIGHUP signal is received. +**Notes:** + +* This daemon reloads it's config when the SIGHUP signal is received. +* If `enable_report_dumps` exists in `WORKDIR` (`/var/cfengine/enable_report_dumps`) `cf-serverd` will log reports provided to `cf-hub` to `WORKDIR/diagnostics/report_dump` (`/var/cfengine/diagnostics/report_dumps`). This data is useful when troubleshooting reporting issues with CFEngine Enterprise. +* `cf-serverd` always considers the class ```server``` to be defined. +* `SIGUSR1` sets the log level to debug. +* `SIGUSR2` sets the log level to notice. **History:** - SIGHUP behavior added in 3.7.0 +- `enable_report_dumps` added in 3.16.0 ## Command reference ## @@ -134,12 +141,12 @@ specify a list of hosts allowed to use the legacy protocol. [%CFEngine_promise_attribute()%] -**See also:** [`protocol_version`][Components and Common Control#protocol_version] +**See also:** [`protocol_version`][Components#protocol_version] ### allowciphers -**Description:** List of TLS ciphers the server accepts for **incoming** connections. +**Description:** List of TLS ciphers the server accepts both **incoming** and **outgoing** (in the case of client initiated reporting with CFEngine Enterprise) connections using `cf-serverd`. For a list of possible ciphers, see man page for "openssl ciphers". [%CFEngine_promise_attribute(AES256-GCM-SHA384:AES256-SHA)%] @@ -155,13 +162,13 @@ body server control ``` **Note:** When used with -[`protocol_version`][Components and Common Control#protocol_version] 1 (classic protocol), +[`protocol_version`][Components#protocol_version] 1 (classic protocol), this does not do anything as the classic protocol does not support TLS ciphers. **See also:** -[`protocol_version`][Components and Common Control#protocol_version], -[`tls_ciphers`][Components and Common Control#tls_ciphers], -[`tls_min_version`][Components and Common Control#tls_min_version], +[`protocol_version`][Components#protocol_version], +[`tls_ciphers`][Components#tls_ciphers], +[`tls_min_version`][Components#tls_min_version], [`allowtlsversion`][cf-serverd#allowtlsversion], [`encrypt`][files#encrypt], [`logencryptedtransfers`][cf-serverd#logencryptedtransfers], @@ -172,7 +179,7 @@ this does not do anything as the classic protocol does not support TLS ciphers. ### allowtlsversion -**Description:** Minimum TLS version allowed for **incoming** connections. +**Description:** Minimum TLS version allowed for both **incoming** and **outgoing** (in the case of client initiated reporting with CFEngine Enterprise) connections using `cf-serverd`. [%CFEngine_promise_attribute(1.0)%] @@ -187,13 +194,13 @@ body server control ``` **Note:** When used with -[`protocol_version`][Components and Common Control#protocol_version] 1 (classic protocol), +[`protocol_version`][Components#protocol_version] 1 (classic protocol), this attribute does not do anything. **See also:** -[`protocol_version`][Components and Common Control#protocol_version], -[`tls_ciphers`][Components and Common Control#tls_ciphers], -[`tls_min_version`][Components and Common Control#tls_min_version], +[`protocol_version`][Components#protocol_version], +[`tls_ciphers`][Components#tls_ciphers], +[`tls_min_version`][Components#tls_min_version], [`allowciphers`][cf-serverd#allowciphers], [`encrypt`][files#encrypt], [`logencryptedtransfers`][cf-serverd#logencryptedtransfers], @@ -491,7 +498,7 @@ to syslog. These files are deemed to be particularly sensitive. } ``` -**See also:** [`ifencrypted`][access#ifencrypted], [`encrypt`][files#encrypt], [`tls_ciphers`][Components and Common Control#tls_ciphers], [`tls_min_version`][Components and Common Control#tls_min_version], [`allowciphers`][cf-serverd#allowciphers], [`allowtlsversion`][cf-serverd#allowtlsversion], [`protocol_version`][Components and Common Control#protocol_version] +**See also:** [`ifencrypted`][access#ifencrypted], [`encrypt`][files#encrypt], [`tls_ciphers`][Components#tls_ciphers], [`tls_min_version`][Components#tls_min_version], [`allowciphers`][cf-serverd#allowciphers], [`allowtlsversion`][cf-serverd#allowtlsversion], [`protocol_version`][Components#protocol_version] ### maxconnections diff --git a/reference/components/file_control_promises.markdown b/reference/components/file_control_promises.markdown index 5eeb01d9b..5ed6d9c8b 100644 --- a/reference/components/file_control_promises.markdown +++ b/reference/components/file_control_promises.markdown @@ -75,8 +75,8 @@ R: hello, from /home/agent/./two.cf ``` -**See also:** [`inputs`][Components and Common Control#inputs] in -[`body common control`][Components and Common Control] +**See also:** [`inputs`][Components#inputs] in +[`body common control`][Components] **History:** Was introduced in CFEngine 3.6.0 diff --git a/reference/enterprise-api-ref/changes.markdown b/reference/enterprise-api-ref/changes.markdown index 134f337e8..083ebcf12 100644 --- a/reference/enterprise-api-ref/changes.markdown +++ b/reference/enterprise-api-ref/changes.markdown @@ -174,7 +174,7 @@ List changes performed by CFEngine to the infrastructure. List can be narrowed d * **data.promiser** Object affected by a promise. * **data.promisetype** - [Type][Promise Types and Attributes] of the promise. + [Type][Promise Types] of the promise. * **data.stackpath** Call stack of the promise. diff --git a/reference/enterprise-api-ref/cmdb-api.markdown b/reference/enterprise-api-ref/cmdb-api.markdown index d0ea834e7..e153692a5 100644 --- a/reference/enterprise-api-ref/cmdb-api.markdown +++ b/reference/enterprise-api-ref/cmdb-api.markdown @@ -16,6 +16,7 @@ You can see a list of stored host-specific configurations **Method:** GET **Parameters:** + * **fromEpoch** *(integer)* Returns configurations with epoch value greater than set in the filter. Epoch is the sequence number of the latest CMDB change. In every API list request, @@ -85,6 +86,7 @@ HTTP 200 Ok **Method:** GET **Parameters:** + * **hostkey** *(string)* Unique host identifier. @@ -127,6 +129,7 @@ HTTP 200 Ok **Method:** GET **Parameters:** + * **hostkey** *(string)* Unique host identifier. @@ -171,6 +174,7 @@ HTTP 200 Ok **Method:** POST **Parameters:** + * **hostkey** *(string)* Unique host identifier. @@ -219,6 +223,7 @@ HTTP 200 Ok **Method:** POST **Parameters:** + * **hostkey** *(string)* Unique host identifier. * **classes** *(JSON object)* @@ -299,16 +304,7 @@ HTTP 201 Created **Method:** PATCH **Parameters:** -* **hostkey** *(string)* - Unique host identifier. -* **type** *(string)* - Configuration type. Allowed value: `variables`, `classes` - -* **name** *(string)* - Configuration name. Classes or variables name. - -**Parameters:** * **hostkey** *(string)* Unique host identifier. @@ -360,6 +356,7 @@ HTTP 200 Ok **Method:** PATCH **Parameters:** + * **hostkey** *(string)* Unique host identifier. * **classes** *(JSON object)* @@ -457,6 +454,7 @@ HTTP 200 Ok **Method:** DELETE **Parameters:** + * **hostkey** *(string)* Unique host identifier. @@ -481,6 +479,7 @@ HTTP 204 No Content **Method:** DELETE **Parameters:** + * **hostkey** *(string)* Unique host identifier. diff --git a/reference/enterprise-api-ref/export-import-api.markdown b/reference/enterprise-api-ref/export-import-api.markdown index ba4cd7146..7d1e5c1dc 100644 --- a/reference/enterprise-api-ref/export-import-api.markdown +++ b/reference/enterprise-api-ref/export-import-api.markdown @@ -7,6 +7,8 @@ tags: [reference, enterprise, API, import, export] Import & Export API provides users the ability to transfer Mission Portal data between hubs. +**See also:** [Export/Import Settings UI][Settings#Export/Import] + ## Get available items to export This API call provides a list of items available for export. Please note that the role of the user that authenticates to this API will affect what items are available. For example: the API user must have admin role in order to export settings. @@ -221,7 +223,7 @@ curl -k --user : \ -F file=@/path/to/file.phar \ -F encryptionKey=key \ -F skipDuplicates=1 \ -'https://hub.example/index.php/data_transfer/api/analyzeImportFile' +'https://hub.example/index.php/data_transfer/api/import' ``` **Example response:** diff --git a/reference/enterprise-api-ref/export-import-compliance-report-api.markdown b/reference/enterprise-api-ref/export-import-compliance-report-api.markdown new file mode 100644 index 000000000..aa9defcdc --- /dev/null +++ b/reference/enterprise-api-ref/export-import-compliance-report-api.markdown @@ -0,0 +1,237 @@ +--- +layout: default +title: Import & Export Compliance Report API +published: true +tags: [reference, enterprise, API, import, export, compliance report] +--- + +This provides users the ability to transfer compliance reports between hubs or create reports from a JSON definition file. + +## Export + +**URI:** https://hub.example/advancedreports/complianceReport/export?id=:ID + +**Method:** GET + +**Example request (curl):** + +``` +curl -k --user : \ + -X GET \ + https://hub.cfengine.com/advancedreports/complianceReport/export?id=20 +``` + +**Example response:** + +``` +HTTP 200 Ok +{ + "reports": { + "example-compliance-report": { + "id": "example-compliance-report", + "type": "compliance", + "title": "Example Compliance Report", + "conditions": [ + "software-updates-available", + "entire-cfengine-policy" + ] + } + }, + "conditions": { + "software-updates-available": { + "id": "software-updates-available", + "name": "Software updates available", + "description": "Available software updates to any package.", + "type": "softwareupdate", + "condition_for": "failing", + "rules": { + "package-name": null, + "condition": null, + "architecture": null + }, + "category": "uncategorized", + "severity": null, + "host_filter": null + }, + "entire-cfengine-policy": { + "id": "entire-cfengine-policy", + "name": "Entire CFEngine policy", + "description": "Promises not kept in the entire policy.", + "type": "policy", + "condition_for": "failing", + "rules": { + "filter-by": null, + "value": null, + "promise-handle": null, + "promise-status": "NOTKEPT" + }, + "category": "uncategorized", + "severity": null, + "host_filter": null + } + } +} +``` + + +## Import + +**URI:** https://hub.example/advancedreports/complianceReport/import + +**Method:** POST + +**Parameters:** + +* **data** *(json)* + Reports and conditions data to be imported. Data json object should have two nested objects: `reports` and `condtions`: + * **reports** + JSON object where the key is report ID, which will be used to identify if report already exists in the system. + * **id** *(text)* + Report ID + * **type** *(text)* + Report's type. Should be set to `complince` + * **title** *(text)* + Report's title. + * **conditions** *(array)* + Conditions list + * **conditions** + JSON object where the key is condition ID, which will be used to identify if condition already exists in the system. + * **id** *(text)* + Condition ID + * **name** *(text)* + Condition name + * **description** *(text)* + Condition description + * **condition_for** *(text)* + Condition for `passing` or `failing`. + * **type** *(text)* + Condition type. Possible values: `inventory`, `custom`, `fileChanged`, `policy`, `software` + * **rules** *(json object)* + JSON object that define rules. Each type has own set of fields: + * **inventory** + * **attribute** *(text)* + Inventory attribute + * **operator** *(text)* + Operator. Possible values: `matches`, `not_match`, `contains`, `not_contain`, `regex_matches`, `regex_not_match`, `is_not_reported`, `is_reported`, `<`, `>`, `<=`, `>=`, `=`, `!=` + * **value** *(text)* + Value. This field might be skipped in case of `is_reported` or `is_not_reported` operators + * **custom** + * **sql** *(text)* + Custom SQL + * **fileChanged** + * **file-name** *(text)* + File name + * **condition** *(text)* + Condition. Possible values: `matches`, `is` + * **time-period** *(int)* + Changed within the time period (hours). + * **policy** + * **filter-by** *(text)* + Filter by: `Bundlename`, `Promisees`, `Promiser` + * **value** *(text)* + Filter value + * **promise-handle** *(text)* + Promise handle + * **promise-status** *(text)* + Promise status: `KEPT`, `NOTKEPT`, `REPAIRED` + * **software** + * **package-name** *(text)* + Package name + * **condition** *(text)* + Condition: `matches`, `is` + * **architecture** *(text)* + Architecture + * **category** *(text)* + Conditions category + * **severity** *(text)* + Condition severity. Possible values: `low`, `medium`, `high` + * **host_filter** *(text)* + Host filter, should be valid class expression. + + +* **overwrite** *(booleans)* + Set true to overwrite existing reports or conditions that belong to you. Default: false + +* **public** *(booleans)* + Set true to make report publicly accessible. Default: false + +**Example request (curl):** + +``` +curl -k --user : \ +-X POST https://hub.cfengine.com/advancedreports/complianceReport/import \ +--form 'data={ + "reports": { + "example-report-1": { + "id": "example-report-1", + "type": "compliance", + "title": "Example report #1", + "conditions": ["os-is-reported", "supported-ubuntu"] + } + }, + "conditions": { + "os-is-reported": { + "id": "os-is-reported", + "name": "Operating system is reported", + "description": "", + "condition_for": "passing", + "type": "inventory", + "rules": [ + { + "attribute": "OS", + "operator": "is_reported" + } + ], + "category": "Operating System", + "severity": "high", + "host_filter": "linux" + }, + "supported-ubuntu": { + "id": "supported-ubuntu", + "name": "Ubuntu version is supported", + "description": "Only Ubuntu 18+ are supported", + "condition_for": "passing", + "type": "inventory", + "rules": [ + { + "attribute": "OS", + "operator": "matches", + "value": "Ubuntu" + } + ], + "category": "Operating System", + "severity": "high", + "host_filter": "linux" + } + } +}' \ +--form 'public=true' \ +--form 'overwrite=true' +``` + +**Example response:** + +``` +HTTP 200 OK +{ + "processed-conditions": { + "os-is-reported": 13, + "supported-ubuntu": 14 + }, + "processed-reports": { + "example-report-1": 22 + } +} +``` + +**Output:** + +* **processed-conditions** + List of processed conditions where the key is condition ID from the data JSON and the value is internal + ID from the database. +* **processed-reports** + List of processed reports where the key is condition ID from the data JSON and the value is internal + ID from the database. + +## History +* Introduced in CFEngine 3.19.0, 3.18.1 diff --git a/reference/enterprise-api-ref/federated-reporting-api.markdown b/reference/enterprise-api-ref/federated-reporting-api.markdown index a58951aae..02b2d1f00 100644 --- a/reference/enterprise-api-ref/federated-reporting-api.markdown +++ b/reference/enterprise-api-ref/federated-reporting-api.markdown @@ -198,7 +198,8 @@ HTTP 202 ACCEPTED "role": "feeder", "enable_request_sent": true, "transport_ssh_public_key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIFpTZhfubmkXONNReTAXA9v2eYo3xQ4GKcsB3J7i5arw root@ubuntu-xenial", - "ssh_fingerprint": false + "ssh_fingerprint": false, + "target_state": "on" } ``` diff --git a/reference/enterprise-api-ref/host.markdown b/reference/enterprise-api-ref/host.markdown index e2319daa9..c5cfda1a8 100644 --- a/reference/enterprise-api-ref/host.markdown +++ b/reference/enterprise-api-ref/host.markdown @@ -118,7 +118,7 @@ Host API allows to access host specific information. **Method:** DELETE Remove data about the host from reporting database and stop collecting reports from the host. -This should be done when the host is no longer active, activities such as a new bootstrap can cause the host to reappear. +This should be done when the host is no longer active. If host is found and scheduled for deletion, status code `202 ACCEPTED` is returned. If host is not found, status code `404 NOT FOUND` is returned. @@ -137,9 +137,9 @@ The hostkey is then removed from: * Public key directory, containing cryptographic keys exchaned during bootstrap (`/var/cfengine/ppkeys`). * The previously mentioned `KeysPendingForDeletion` table. -[Depending on the configuration][Masterfiles Policy Framework#trustkeysfrom] -of [`trustkeysfrom`][cf-serverd#trustkeysfrom] for the hub hosts may re-appear -and resume being collected from after being deleted. +Note: There is a record of the host retained that includes the time when the host was deleted and this record also prevents further collection from this host identity. + +**See also:** [Example removing host data][Browsing Host Information#example: removing host data] ## Hosts list grouped by hard classes @@ -154,6 +154,8 @@ and resume being collected from after being deleted. Output format. Default value is `json`. Allowed values: `json`, `yaml`. * **withInventory** *(boolean)* Include inventory data to the API response. Default value is `false`. Allowed values: `true`, `false` +* **inventoryFile** *(boolean)* + Make hosts' children values objects which aligns with Ansible inventory that is sourced from a file (so this format is appropriate for caching responses), by default when `inventoryFile` is `false`, the output format aligns with Ansible inventory sourced from a script. Default value is `false`. Allowed values: `true`, `false` **CURL unfiltered request example** @@ -183,6 +185,34 @@ curl -k --user admin:admin -X GET https://hub.example.com/api/hosts/by-class } ``` +**inventoryFile=true** + +``` +curl -k --user admin:admin -X GET https://hub.example.com/api/hosts/by-class?inventoryFile=true +``` + +**Example response:** + +``` +{ + "10_0_2_15": { + "hosts": [ + "ubuntu-xenial": {} + ] + }, + "127_0_0_1": { + "hosts": [ + "ubuntu-xenial": {} + ] + }, + "ubuntu_16": { + "hosts": [ + "ubuntu-xenial": {} + ] + } +} +``` + **CURL request with inventory data example** ``` @@ -222,6 +252,51 @@ curl -k --user admin:admin -X GET https://hub.example.com/api/hosts/by-class?wit } ``` +## Get deleted hosts list + +**URI:** https://hub.cfengine.com/api/hosts/deleted + +**Method:** GET + +**Parameters:** + +* **skip** *(integer)* + Number of results to skip for the processed query. + Optional parameter. +* **limit** *(integer)* + Limit the number of results in the query. + No limit when parameter is not set. Optional parameter. + +**Example request (curl):** +``` +curl -k --user admin:admin -X GET https://hub.example.com/api/hosts/deleted +``` +**Example response:** + +``` +HTTP 200 Ok +{ + "data": [ + { + "hostkey": "SHA=2123f85b38189008ae12be159fb961584dda1249c94efed43fec2c70f233975d", + "iscallcollected": false, + "lastreporttimestamp": "2017-02-17 18:00:17+00", + "firstreporttimestamp": "2017-01-09 17:35:35.427063+00", + "hostkeycollisions": 0, + "deleted": "2021-08-19 09:20:02.752463+00", + "ipaddress": "10.0.2.15" + } + ], + "meta": { + "total": 1, + "page": 1, + "count": 1, + "timestamp": 1629365174 + } +} +``` + + ## List monitoring attributes for host **URI:** https://hub.cfengine.com/api/host/:host-id/vital @@ -395,3 +470,6 @@ HTTP 200 Ok * **count** The bootstrapped hosts to the hub count. +## History + +* `inventoryFile=true` parameter added in CFEngine 3.19.0, 3.18.1 diff --git a/reference/enterprise-api-ref/sql-schema.markdown b/reference/enterprise-api-ref/sql-schema.markdown index 1a2346850..9bdbeee63 100644 --- a/reference/enterprise-api-ref/sql-schema.markdown +++ b/reference/enterprise-api-ref/sql-schema.markdown @@ -753,6 +753,7 @@ Stores 1 record for each observable per host. * **lastupdatedsample** *(integer)* Value of most recently collected measurement. + ## Table: PromiseExecutions Promises executed on hosts during their last reported cf-agent run. @@ -778,7 +779,7 @@ Promises executed on hosts during their last reported cf-agent run. [Bundle][Bundles] name where the promise is executed. * **PromiseType** *(text)* - [Type][Promise Types and Attributes] of the promise. + [Type][Promise Types] of the promise. * **Promiser** *(text)* Object affected by a promise. @@ -913,7 +914,7 @@ Promise status / outcome changes over period of time. [Bundle][Bundles] name where the promise is executed. * **PromiseType** *(text)* - [Type][Promise Types and Attributes] of the promise. + [Type][Promise Types] of the promise. * **Promiser** *(text)* Object affected by a promise. @@ -1048,7 +1049,7 @@ History of promises executed on hosts. [Bundle][Bundles] name where the promise is executed. * **PromiseType** *(text)* - [Type][Promise Types and Attributes] of the promise. + [Type][Promise Types] of the promise. * **Promiser** *(text)* Object affected by a promise. diff --git a/reference/enterprise-api-ref/vcs-settings.markdown b/reference/enterprise-api-ref/vcs-settings.markdown index 928eeeb2e..a4b758172 100644 --- a/reference/enterprise-api-ref/vcs-settings.markdown +++ b/reference/enterprise-api-ref/vcs-settings.markdown @@ -53,6 +53,8 @@ curl -k --user : \ **Parameters:** +* **vscType** *(string)* + VCS type. Allowed values: `GIT`, `GIT_CFBS`. Default value: `GIT` * **gitServer** *(string)* Git repository URL `Emp: https://github.com/cfengine/masterfiles.git`. Required parameter. * **gitRefspec** *(string)* @@ -91,4 +93,8 @@ curl -k --user : \ "gitPassword": "passwordOrToken", "gitPrivateKey": "/opt/cfengine/userworkdir/admin/.ssh/id_rsa.pvt" } -``` \ No newline at end of file +``` + +## History + +* `vscType` parameter added in 3.19.0, 3.18.1 diff --git a/reference/functions.markdown b/reference/functions.markdown index e871222ad..6b8299dcf 100644 --- a/reference/functions.markdown +++ b/reference/functions.markdown @@ -125,8 +125,10 @@ they are in the same bundle or not) only the first executed function will be cached. That cached result will be re-used for other identical function occurrences. -Function caching can be disabled by setting `cache_system_functions` in body -common control to `false`. +Function caching can be globally disabled by setting `cache_system_functions` in +body common control to `false` or locally for a specific promise by using +`ifelapsed => "0"` in the [action body][Promise Types#ifelapsed] +of the promise. ## Function Skipping diff --git a/reference/functions/accumulated.markdown b/reference/functions/accumulated.markdown index 4e16b3e28..9ac3ce174 100644 --- a/reference/functions/accumulated.markdown +++ b/reference/functions/accumulated.markdown @@ -78,3 +78,10 @@ Seconds of runtime In the example we look for processes that have accumulated between 2 and 20 minutes of total run time. + +**See also:** + +* Related functions + * [`ago()`][ago] + * [`now()`][now] + * [`irange()`][irange] diff --git a/reference/functions/ago.markdown b/reference/functions/ago.markdown index a58c3ccf5..1a21f0c8c 100644 --- a/reference/functions/ago.markdown +++ b/reference/functions/ago.markdown @@ -51,3 +51,9 @@ Output: [%CFEngine_include_snippet(ago.cf, #\+begin_src\s+example_output\s*, .*end_src)%] +**See also:** + +* Related functions + * [`now()`][now] + * [`accumulated()`][accumulated] + * [`irange()`][irange] diff --git a/reference/functions/and.markdown b/reference/functions/and.markdown index c730b3b5b..0451caf07 100644 --- a/reference/functions/and.markdown +++ b/reference/functions/and.markdown @@ -25,6 +25,9 @@ classes. **Notes:** Introduced primarily for use with `if` and `unless` promise attributes. -**See also:** `and`, `or`, `not` +**See also:** `or()`, `not()` -**History:** Was introduced in 3.2.0, Nova 2.1.0 (2011) +**History:** + +* Introduced in 3.2.0, Nova 2.1.0 (2011) +* Return type changed from `string` to `boolean` in 3.17.0 (2020) (CFE-3470) diff --git a/reference/functions/bundlestate.markdown b/reference/functions/bundlestate.markdown index 492eaa2b3..cc1473be0 100644 --- a/reference/functions/bundlestate.markdown +++ b/reference/functions/bundlestate.markdown @@ -32,4 +32,8 @@ Output: [%CFEngine_include_snippet(bundlestate.cf, #\+begin_src\s+example_output\s*, .*end_src)%] -**See also:** `getindices()`, `classesmatching()`, `variablesmatching()`, `mergedata()`, `template_method`, `mustache`, `inline_mustache`, `datastate()` +**See also:** `getindices()`, `classesmatching()`, `variablesmatching()`, `mergedata()`, [template_method][files#template_method], `mustache`, `inline_mustache`, `datastate()` + +**History:** + +* Introduced in CFEngine 3.7.0 diff --git a/reference/functions/classesmatching.markdown b/reference/functions/classesmatching.markdown index f36112d7d..1d1d8eb19 100644 --- a/reference/functions/classesmatching.markdown +++ b/reference/functions/classesmatching.markdown @@ -17,7 +17,7 @@ classes. The search order is hard, soft, then local to the current bundle. When any tags are given, only the classes with those tags matching the given [anchored][anchored] regular expressions are returned. Class tags are set -using the [`meta`][Promise Types and Attributes#meta] attribute. +using the [`meta`][Promise Types#meta] attribute. [%CFEngine_function_attributes(name, tag1, tag2, ...)%] diff --git a/reference/functions/classfiltercsv.markdown b/reference/functions/classfiltercsv.markdown index 54fedc8e7..bae19c528 100644 --- a/reference/functions/classfiltercsv.markdown +++ b/reference/functions/classfiltercsv.markdown @@ -8,7 +8,7 @@ tags: [reference, csv, data functions, functions, classfiltercsv] [%CFEngine_function_prototype(filename, has_header, class_column, optional_sort_column)%] **Description:** -Parses CSV data from the file `filename`, and returns a data variable that is +Parses CSV data from an [RFC 4180](http://www.ietf.org/rfc/rfc4180.txt) compliant file `filename` (`CRLF` line endings required), and returns a data variable that is filtered by defined classes. If `has_header` is set to `true`, the columns in the first line of the CSV file are used as keys for the data. `class_column` specifies which column contains class names to filter by. @@ -22,18 +22,31 @@ minus `1`. **Example:** -Prepare: +**Prepare CSV:** [%CFEngine_include_snippet(classfiltercsv.cf, #\+begin_src prep, .*end_src)%] -Policy: +**Policy:** [%CFEngine_include_snippet(classfiltercsv.cf, #\+begin_src cfengine3, .*end_src)%] -Output: +**Output:** [%CFEngine_include_snippet(classfiltercsv.cf, #\+begin_src\s+example_output\s*, .*end_src)%] +**Notes:** + +* If the CSV file is stored in a `git` repository the `.gitattributes` file can be used to ensure proper line endings. + + For example: + + ``` + # .gitattribtues + *.csv text eol=crlf + RFC-4180-non-compliant-line-endings.csv eol=lf + *.mustache text + *.sh text eol=lf + ``` **See also:** [readcsv()][readcsv], [classmatch()][classmatch] diff --git a/reference/functions/data_readstringarray.markdown b/reference/functions/data_readstringarray.markdown index 89be64e9c..1180bc271 100644 --- a/reference/functions/data_readstringarray.markdown +++ b/reference/functions/data_readstringarray.markdown @@ -36,3 +36,7 @@ Output: [%CFEngine_include_snippet(data_readstringarray.cf, #\+begin_src\s+example_output\s*, .*end_src)%] **See also:** `data_readstringarrayidx()`, `data` + +**History:** + +* Added in CFEngine 3.6.0 diff --git a/reference/functions/data_readstringarrayidx.markdown b/reference/functions/data_readstringarrayidx.markdown index d8635d018..9996c9851 100644 --- a/reference/functions/data_readstringarrayidx.markdown +++ b/reference/functions/data_readstringarrayidx.markdown @@ -34,3 +34,7 @@ Output: [%CFEngine_include_snippet(data_readstringarray.cf, #\+begin_src\s+example_output\s*, .*end_src)%] **See also:** `data_readstringarray()`, `data` + +**History:** + +* Added in CFEngine 3.6.0 diff --git a/reference/functions/data_sysctlvalues.markdown b/reference/functions/data_sysctlvalues.markdown index 79238c44b..fe7c525f2 100644 --- a/reference/functions/data_sysctlvalues.markdown +++ b/reference/functions/data_sysctlvalues.markdown @@ -13,21 +13,13 @@ tags: [reference, system functions, functions, sysctl, data_sysctlvalues] **Example:** -```cf3 - vars: - "sysctl" data => data_sysctlvalues(); # get everything! -``` +Policy: + +[%CFEngine_include_example(data_sysctlvalues.cf, #\+begin_src\s+cfengine3, .*end_src)%] Output: -``` -{ - "sysctl": { - ... - "net.unix.max_dgram_qlen": "512", - ... - } -``` +[%CFEngine_include_snippet(data_sysctlvalues.cf, #\+begin_src\s+mock_example_output\s*, .*end_src)%] **Notes:** diff --git a/reference/functions/datastate.markdown b/reference/functions/datastate.markdown index 7840b23f3..16fbc8383 100644 --- a/reference/functions/datastate.markdown +++ b/reference/functions/datastate.markdown @@ -20,7 +20,7 @@ map with the variable name as the key. The value is converted to a data container (JSON format) if necessary. The example should make it clearer. -Mustache templates (see `template_method`), if not given a +Mustache templates (see [template_method][files#template_method]), if not given a `template_data`, will use the output of `datastate()` as their input. [%CFEngine_function_attributes()%] @@ -33,7 +33,7 @@ Output: [%CFEngine_include_snippet(datastate.cf, #\+begin_src\s+example_output\s*, .*end_src)%] -**See also:** `getindices()`, `classesmatching()`, `variablesmatching()`, `mergedata()`, `template_method`, `mustache`, `inline_mustache`, `bundlestate()` +**See also:** `getindices()`, `classesmatching()`, `variablesmatching()`, `mergedata()`, [template_method][files#template_method], `mustache`, `inline_mustache`, `bundlestate()` **Notes:** diff --git a/reference/functions/findfiles.markdown b/reference/functions/findfiles.markdown index b057ca8ba..f648d896c 100644 --- a/reference/functions/findfiles.markdown +++ b/reference/functions/findfiles.markdown @@ -23,6 +23,13 @@ deep. This function, used together with the `bundlesmatching` function, allows you to do dynamic inputs and a dynamic bundle call chain. +**WARNING:** +- The current implementation of glob patterns on Windows contains bugs. + Therefore, we strongly recommend using the `!windows::` class guard + expression to safeguard against any use of the function on Windows platforms. + Rest assured, we are actively working on resolving these issues and improving + its functionality. + **Notes:** - Brace expansion is not currently supported, `{x,y,anything}` will not match `x` or `y` or `anything`. diff --git a/reference/functions/findfiles_up.markdown b/reference/functions/findfiles_up.markdown index 5e9401f89..f47f27b4b 100644 --- a/reference/functions/findfiles_up.markdown +++ b/reference/functions/findfiles_up.markdown @@ -14,9 +14,9 @@ This function searches for files matching a given glob pattern `glob` in the local filesystem by searching up the directory tree from a given absolute path `path`. The function searches at moast `level` levels of directories or until the root directory is reached. Argument `level` defaults to `inf` if -not specified. The function returnes a list of files as a data array where +not specified. The function returns a list of files as a data array where the first element _(element 0)_ and the last element _(element N)_ is first -and last file or directory found respectivly. +and last file or directory found respectively. Note that glob patterns are not regular expressions. They match like Unix shells: @@ -25,6 +25,13 @@ shells: * `?` matches a single letter * `[a-z]` matches any letter from `a` to `z` +**WARNING:** +- The current implementation of glob patterns on Windows contains bugs. + Therefore, we strongly recommend using the `!windows::` class guard + expression to safeguard against any use of the function on Windows platforms. + Rest assured, we are actively working on resolving these issues and improving + its functionality. + **Notes:** - Brace expansion is not currently supported, `{x,y,anything}` will not match `x` or `y` or `anything`. diff --git a/reference/functions/format.markdown b/reference/functions/format.markdown index ab0b028cc..4d0ece96a 100644 --- a/reference/functions/format.markdown +++ b/reference/functions/format.markdown @@ -30,3 +30,7 @@ Output: [%CFEngine_include_snippet(format.cf, #\+begin_src\s+example_output\s*, .*end_src)%] **Note:** the underlying `sprintf` system call may behave differently on some platforms for some formats. Test carefully. For example, the format `%08s` will use spaces to fill the string up to 8 characters on libc platforms, but on Darwin (Mac OS X) it will use zeroes. According to [SUSv4](http://pubs.opengroup.org/onlinepubs/9699919799/functions/sprintf.html) the behavior is undefined for this specific case. + +**History:** + +* Added in CFEngine 3.6.0 diff --git a/reference/functions/getclassmetatags.markdown b/reference/functions/getclassmetatags.markdown index 7efb8b330..7f84ff862 100644 --- a/reference/functions/getclassmetatags.markdown +++ b/reference/functions/getclassmetatags.markdown @@ -7,7 +7,7 @@ tags: [reference, data functions, functions, getclassmetatags, meta, tags] [%CFEngine_function_prototype(classname, optional_tag)%] -**Description:** Returns the list of [`meta`][Promise Types and Attributes#meta] tags for class `classname`. +**Description:** Returns the list of [`meta`][Promise Types#meta] tags for class `classname`. [%CFEngine_function_attributes(classname, optional_tag)%] @@ -28,4 +28,7 @@ Output: **See also:** `getvariablemetatags()` -**History:** Function added in 3.6.0. `optional_tag` added in 3.10.0 +**History:** + +* Function added in 3.6.0. +* `optional_tag` added in 3.10.0 diff --git a/reference/functions/getusers.markdown b/reference/functions/getusers.markdown index 9b3ec767d..8525fb328 100644 --- a/reference/functions/getusers.markdown +++ b/reference/functions/getusers.markdown @@ -7,7 +7,7 @@ tags: [reference, system functions, functions, getusers] [%CFEngine_function_prototype(exclude_names, exclude_ids)%] -**Description:** Returns a list of all users defined, except those names in `exclude_names` and uids in `exclude_ids` +**Description:** Returns a list of all users defined, except those names in the comma separated string of `exclude_names` and the comma separated string of uids in `exclude_ids` [%CFEngine_function_attributes(exclude_names, exclude_ids)%] diff --git a/reference/functions/getvariablemetatags.markdown b/reference/functions/getvariablemetatags.markdown index 59bbeb84b..590fdbcbc 100644 --- a/reference/functions/getvariablemetatags.markdown +++ b/reference/functions/getvariablemetatags.markdown @@ -7,7 +7,7 @@ tags: [reference, data functions, functions, getvariablemetatags, meta, tags] [%CFEngine_function_prototype(varname, optional_tag)%] -**Description:** Returns the list of [`meta`][Promise Types and Attributes#meta] tags for variable `varname`. +**Description:** Returns the list of [`meta`][Promise Types#meta] tags for variable `varname`. Make sure you specify the correct scope when supplying the name of the variable. @@ -31,4 +31,7 @@ Output: **See also:** `getclassmetatags()` -**History:** Function in 3.6.0. `optional_tag` added in 3.10.0 +**History:** + +* Introduced in CFEngine 3.6.0 +* `optional_tag` added in 3.10.0 diff --git a/reference/functions/hostsseen.markdown b/reference/functions/hostsseen.markdown index 6dd382f0f..a643ca7e4 100644 --- a/reference/functions/hostsseen.markdown +++ b/reference/functions/hostsseen.markdown @@ -31,4 +31,4 @@ reports: } ``` -**See also:** [lastseenexpireafter in body common control][Components and Common Control#lastseenexpireafter] +**See also:** [lastseenexpireafter in body common control][Components#lastseenexpireafter] diff --git a/reference/functions/irange.markdown b/reference/functions/irange.markdown index 3b0a818f3..0d5717252 100644 --- a/reference/functions/irange.markdown +++ b/reference/functions/irange.markdown @@ -23,3 +23,26 @@ also use the functions `ago()`, `now()`, `accumulated()`, etc. irange(ago(0,0,0,1,30,0), "0"); ``` + +**See also:** + +* Functions commonly used with [`irange()`][irange] + * [`ago()`][ago] + * [`now()`][now] + * [`accumulated()`][accumulated] +* Attributes of type irange + * [`atime` in body `file_select`][files#atime] + * [`copy_size` in body `copy_from`][files#copy_size] + * [`ctime` in body `file_select`][files#ctime] + * [`mtime` in body `file_select`][files#mtime] + * [`match_range` in body `process_count`][processes#match_range] + * [`pgid` in body `process_count`][processes#pgid] + * [`pid` in body `process_count`][processes#pid] + * [`ppid` in body `process_count`][processes#ppid] + * [`priority` in body `process_count`][processes#priority] + * [`rsize` in body `process_count`][processes#rsize] + * [`search_size` in body `file_select`][files#search_size] + * [`stime_range` in body `process_count`][processes#stime_range] + * [`threads` in body `process_count`][processes#threads] + * [`ttime_range` in body `process_count`][processes#ttime_range] + * [`vsize` in body `process_count`][processes#vsize] diff --git a/reference/functions/mergedata.markdown b/reference/functions/mergedata.markdown index c904faf69..e1c72cc16 100644 --- a/reference/functions/mergedata.markdown +++ b/reference/functions/mergedata.markdown @@ -30,6 +30,7 @@ traditional list and array data types in CFEngine. - Bare values try to expand a named cfengine data container - It is only possible to wrap data containers in the current namespace. - true and false are reserved bare values +- In the event of key collision the *last* key merged wins [%CFEngine_function_attributes()%] @@ -37,10 +38,21 @@ traditional list and array data types in CFEngine. [%CFEngine_include_snippet(mergedata.cf, #\+begin_src cfengine3, .*end_src)%] -Output: +**Output:** [%CFEngine_include_snippet(mergedata.cf, #\+begin_src\s+example_output\s*, .*end_src)%] -**History:** Was introduced in CFEngine 3.6.0 (2014). The [collecting function][Functions#collecting functions] behavior was added in 3.9. +**Example:** + +[%CFEngine_include_snippet(mergedata-last-key-wins.cf, #\+begin_src cfengine3, .*end_src)%] + +**Output:** + +[%CFEngine_include_snippet(mergedata-last-key-wins.cf, #\+begin_src\s+example_output\s*, .*end_src)%] + +**History:** + +* Introduced in CFEngine 3.6.0 (2014). +* The [collecting function][Functions#collecting functions] behavior was added in 3.9. **See also:** `getindices()`, `getvalues()`, `readjson()`, `parsejson()`, `readyaml()`, `parseyaml()`, [about collecting functions][Functions#collecting functions], and `data` documentation. diff --git a/reference/functions/not.markdown b/reference/functions/not.markdown index 9d8fa8ad5..857af3ca7 100644 --- a/reference/functions/not.markdown +++ b/reference/functions/not.markdown @@ -26,6 +26,9 @@ commands: **Notes:** Introduced primarily for use with `if` and `unless` promise attributes. -**See also:** `and`, `or`, `not` +**See also:** `and()`, `or()` -**History:** Was introduced in 3.2.0, Nova 2.1.0 (2011) +**History:** + +* Introduced in 3.2.0, Nova 2.1.0 (2011) +* Return type changed from `string` to `boolean` in 3.17.0 (2020) (CFE-3470) diff --git a/reference/functions/now.markdown b/reference/functions/now.markdown index f35ded268..e59b811df 100644 --- a/reference/functions/now.markdown +++ b/reference/functions/now.markdown @@ -79,3 +79,9 @@ select processes based on relative execution time. [%CFEngine_include_example(processes_define_class_based_on_process_runtime.cf)%] +**See also:** + +* Related functions + * [`ago()`][ago] + * [`accumulated()`][accumulated] + * [`irange()`][irange] diff --git a/reference/functions/or.markdown b/reference/functions/or.markdown index 9ed6a0fa5..e0b5ab01e 100644 --- a/reference/functions/or.markdown +++ b/reference/functions/or.markdown @@ -25,6 +25,9 @@ classes. **Notes:** Introduced primarily for use with `if` and `unless` promise attributes. -**See also:** `and`, `or`, `not` +**See also:** `and()`, `not()` -**History:** Was introduced in 3.2.0, Nova 2.1.0 (2011) +**History:** + +* Introduced in 3.2.0, Nova 2.1.0 (2011) +* Return type changed from `string` to `boolean` in 3.17.0 (2020) (CFE-3470) diff --git a/reference/functions/packagesmatching.markdown b/reference/functions/packagesmatching.markdown index c7096b0f6..92903ac02 100644 --- a/reference/functions/packagesmatching.markdown +++ b/reference/functions/packagesmatching.markdown @@ -16,7 +16,7 @@ list of currently installed packages. The return is a data container with a list of package descriptions, looking like this: -``` +```json [ { "arch":"default", @@ -30,28 +30,25 @@ this: [%CFEngine_function_attributes(package_regex, version_regex, arch_regex, method_regex)%] -**Argument Descriptions:** +**IMPORTANT:** The data source used when querying depends on policy configuration. +When `package_inventory` in `body common control` is configured, CFEngine will record the packages installed and the package updates available for the configured package modules. +In the [Masterfiles Policy Framework][Masterfiles Policy Framework] `package_inventory` will be [configured](https://github.com/cfengine/masterfiles/blob/3dc1f629544b24261975ecf86e02554d4daf346e/promises.cf.in#L92) to the default for the hosts platform. +Since only one `body common control` can be present in a policy set any bundles which use these functions will typically need to execute in the context of a full policy run. +However, the `packagesmatching` and `packageupdatesmatching` policy functions will look for and use the existing software inventory databases (available in `$(sys.statedir)`), even if the default package inventory is not configured. +This enables the usage of these policy functions in standalone policy files. But please note that you still need the default package inventory attribute specified in the policy framework for the software inventory databases to exist in the first place and for them to be maintained/updated. +If there is no `package_inventory` attribute (such as on package module unsupported platforms) and there are no software inventory databases available in `$(sys.statedir)` then the legacy package methods data will be used instead. +At no time will both the standard and the legacy data be available to these functions simultaneously. -* `package_regex` - Regular expression matching packge name -* `version_regex` - Regular expression matching package version -* `arch_regex` - Regular expression matching package architecutre -* `method_regex` - Regular expression matching package method (apt-get, rpm, etc ...) +**Example:** The following code extracts just the package names, then looks for some desired packages, and finally reports if they are installed. -**IMPORTANT:** Please note that you need to provide `package_inventory` attribute in `body common control` in order to be able to use this function. Also depending on the value(s) of `package_inventory` only packages from selected package modules will be returned. For more information about `package_inventory` please read [`package_inventory`][Components and Common Control#package_inventory] section. [%CFEngine_include_example(packagesmatching.cf)%] -**Example:** - -```cf3 - "all_packages" data => packagesmatching(".*", ".*", ".*", ".*"); -``` - **Refresh rules:** -* inastalled packages cache used by packagesmatching() is refreshed at the end of each agent run in accordance with constraints defined in the relevant package module body. +* installed packages cache used by packagesmatching() is refreshed at the end of each agent run in accordance with constraints defined in the relevant package module body. * installed packages cache is refreshed after installing or removing a package. * installed packages cache is refreshed if no local cache exists. This means a reliable way to force a refresh of CFEngine's internal package cache is to simply delete the local cache: @@ -60,6 +57,20 @@ some desired packages, and finally reports if they are installed. $(sys.statedir)/packages_installed_.lmdb* ``` -**History:** Introduced in CFEngine 3.6 +Or in the case of legacy package methods: + +```cf3 +$(sys.statedir)/software_packages.csv +``` + + +**History:** + +* Introduced in CFEngine 3.6 + +* Function started using `package_module` based data sources by default, even if + there is no `package_inventory` attribute defined in `body common control` if + available in 3.23.0, 3.21.3, 3.18.6 + -**See also:** `packageupdatesmatching()`. +**See also:** `packageupdatesmatching()`, [Package information cache tunables in the MPF][Masterfiles Policy Framework#Configure periodic package inventory refresh interval] diff --git a/reference/functions/packageupdatesmatching.markdown b/reference/functions/packageupdatesmatching.markdown index a49589dc3..9c7395640 100644 --- a/reference/functions/packageupdatesmatching.markdown +++ b/reference/functions/packageupdatesmatching.markdown @@ -16,7 +16,7 @@ list of currently available packages. The return is a data container with a list of package descriptions, looking like this: -``` +```json [ { "arch":"default", @@ -29,20 +29,25 @@ this: [%CFEngine_function_attributes(package_regex, version_regex, arch_regex, method_regex)%] -**Argument Descriptions:** - -* `package_regex` - Regular expression matching packge name -* `version_regex` - Regular expression matching package version -* `arch_regex` - Regular expression matching package architecutre -* `method_regex` - Regular expression matching package method (apt-get, rpm, etc ...) - -**IMPORTANT:** Please note that you need to provide `package_inventory` attribute in `body common control` in order to be able to use this function. Also depending on the value(s) of `package_inventory` only packages from selected package modules will be returned. For more information about `package_inventory` please read [`package_inventory`][Components and Common Control#package_inventory] section. +**IMPORTANT:** The data source used when querying depends on policy configuration. +When `package_inventory` in `body common control` is configured, CFEngine will record the packages installed and the package updates available for the configured package modules. +In the [Masterfiles Policy Framework][Masterfiles Policy Framework] `package_inventory` will be [configured](https://github.com/cfengine/masterfiles/blob/3dc1f629544b24261975ecf86e02554d4daf346e/promises.cf.in#L92) to the default for the hosts platform. +Since only one `body common control` can be present in a policy set any bundles which use these functions will typically need to execute in the context of a full policy run. +However, the `packagesmatching` and `packageupdatesmatching` policy functions will look for and use the existing software inventory databases (available in `$(sys.statedir)`), even if the default package inventory is not configured. +This enables the usage of these policy functions in standalone policy files. But please note that you still need the default package inventory attribute specified in the policy framework for the software inventory databases to exist in the first place and for them to be maintained/updated. +If there is no `package_inventory` attribute (such as on package module unsupported platforms) and there are no software inventory databases available in `$(sys.statedir)` then the legacy package methods data will be used instead. +At no time will both the standard and the legacy data be available to these functions simultaneously. **Example:** ```cf3 - "all_package_updates" data => packageupdatesmatching(".*", ".*", ".*", ".*"); +vars: + "all_package_updates" + data => packageupdatesmatching(".*", # Package name regex + ".*", # Version regex + ".*", # Arch regex + ".*"); # Method regex ``` **Refresh rules:** @@ -55,7 +60,19 @@ this: $(sys.statedir)/packages_updates_.lmdb* ``` +Or in the case of legacy package methods: + +```cf3 +$(sys.statedir)/software_patches_avail.csv +``` + +**History:** + +* Introduced in CFEngine 3.6 + +* Function started using `package_module` based data sources by default, even if + there is no `package_inventory` attribute defined in `body common control` if + available in 3.23.0, 3.21.3, 3.18.6 -**History:** Introduced in CFEngine 3.6 -**See also:** `packagesmatching()`. +**See also:** `packagesmatching()`, [Package information cache tunables in the MPF][Masterfiles Policy Framework#Configure periodic package inventory refresh interval] diff --git a/reference/functions/parsejson.markdown b/reference/functions/parsejson.markdown index 9c62f9647..bfa02885c 100644 --- a/reference/functions/parsejson.markdown +++ b/reference/functions/parsejson.markdown @@ -33,6 +33,13 @@ types of quotes around a string). data => '{ "key": "value" }'; ``` -**History:** The [collecting function][Functions#collecting functions] behavior was added in 3.9. +**Notes:** + +* This functions does not parse _primitives_. + +**History:** + +* Introduced in CFEngine 3.6.0 +* The [collecting function][Functions#collecting functions] behavior was added in 3.9. **See also:** `readjson()`, `parseyaml()`, `readyaml()`, `mergedata()`, `Inline YAML and JSON data`, [about collecting functions][Functions#collecting functions], and `data` documentation. diff --git a/reference/functions/readfile.markdown b/reference/functions/readfile.markdown index 727e953e9..ca9297688 100644 --- a/reference/functions/readfile.markdown +++ b/reference/functions/readfile.markdown @@ -10,7 +10,7 @@ tags: [reference, io functions, functions, readfile] **Description:** Returns the first `maxbytes` bytes from file `filename`. `maxbytes` is optional, if specified, only the first `maxbytes` bytes are read from `filename`. -When `maxbytes` is `0` or not specified, the whole file will be read (but see **Notes** below). +When `maxbytes` is `0`, `inf` or not specified, the whole file will be read (but see **Notes** below). [%CFEngine_function_attributes(filename, optional_maxbytes)%] @@ -30,23 +30,14 @@ Output: **Notes:** -* To reliably read files located within /proc or /sys directories, -`maxsize` has to be set to `0`. - -* At the moment, only 4095 bytes can fit into a string variable. This -limitation may be removed in the future. If this should happen, a -warning will be printed. - -* If you request more bytes than CFEngine can read into a string -variable (e.g. `999999999`), a warning will also be printed. - -* If either because you specified a large value, or you specified `0`, -more bytes are read than will fit in a string, the string is -truncated to the maximum. - * On Windows, the file will be read in text mode, which means that CRLF line endings will be converted to LF line endings in the resulting variable. This can make the variable length shorter than the size of the file being read. -**History:** Warnings about the size limit and the special `0` value were introduced in 3.6.0 +**History:** + +* Warnings about the size limit and the special `0` value were introduced in 3.6.0 +* 4095 bytes limitation removed in 3.6.3 + + diff --git a/reference/functions/readintlist.markdown b/reference/functions/readintlist.markdown index f4e312d5a..a101576eb 100644 --- a/reference/functions/readintlist.markdown +++ b/reference/functions/readintlist.markdown @@ -5,9 +5,7 @@ published: true tags: [reference, io functions, functions, readintlist] --- -**Prototype:** `readintlist(filename, comment, split, maxentries, maxbytes)`
    - -**Return type:** `ilist` +[%CFEngine_function_prototype(filename, comment, split, maxentries, maxbytes)%] **Description:** Splits the file `filename` into separated values and returns the list. @@ -16,14 +14,7 @@ The `comment` field is a multiline regular expression and will strip out unwanted patterns from the file being read, leaving unstripped characters to be split into fields. Using the empty string (`""`) indicates no comments. -**Arguments**: - -* `filename` : File name to read, in the range `"?(/.*)` -* `comment` : [Unanchored][unanchored] regex matching comments, in the range `.*` -* `split` : [Unanchored][unanchored] regex to split data, in the range `.*` -* `maxentries` : Maximum number of entries to read, in the range -`0,99999999999` -* `maxbytes` : Maximum bytes to read, in the range `0,99999999999` +[%CFEngine_function_attributes(filename, comment, split, maxentries, maxbytes)%] **Example:** diff --git a/reference/functions/readjson.markdown b/reference/functions/readjson.markdown index 85c6c6646..478807869 100644 --- a/reference/functions/readjson.markdown +++ b/reference/functions/readjson.markdown @@ -24,3 +24,7 @@ first `maxbytes` bytes are read from `filename`. ``` **See also:** `readdata()`, `parsejson()`, `storejson()`, `parseyaml()`, `readyaml()`, `mergedata()`, `validjson()`, and `data` documentation. + +**History:** + +* Introduced in CFEngine 3.6.0 diff --git a/reference/functions/regex_replace.markdown b/reference/functions/regex_replace.markdown index 5c3652145..c59bdefa3 100644 --- a/reference/functions/regex_replace.markdown +++ b/reference/functions/regex_replace.markdown @@ -16,6 +16,7 @@ string in any order. Consult http://pcre.org/pcre.txt for the exact meaning of the uppercase options, and note that some can be turned on inside the regular expression, e.g. `(?s)`. +* `g`: global, replace all matches * `i`: case-insensitive * `m`: multiline (`PCRE_MULTILINE`) * `s`: dot matches newlines too (`PCRE_DOTALL`) diff --git a/reference/functions/storejson.markdown b/reference/functions/storejson.markdown index 7ad7875be..6be7f5e2e 100644 --- a/reference/functions/storejson.markdown +++ b/reference/functions/storejson.markdown @@ -17,6 +17,9 @@ tags: [reference, io functions, functions, storejson, json, inline_json, contain [%CFEngine_include_example(storejson.cf)%] -**History:** The [collecting function][Functions#collecting functions] behavior was added in 3.9. +**History:** + +* Introduced in CFEngine 3.6.0 +* The [collecting function][Functions#collecting functions] behavior was added in 3.9. **See also:** `readjson()`, `readyaml()`, `parsejson()`, `parseyaml()`, [about collecting functions][Functions#collecting functions], and `data` documentation. diff --git a/reference/functions/type.markdown b/reference/functions/type.markdown index 39dfe3ac0..4d8ed8812 100644 --- a/reference/functions/type.markdown +++ b/reference/functions/type.markdown @@ -7,14 +7,9 @@ tags: [reference, utility functions, functions, type] [%CFEngine_function_prototype(var, detail)%] -**Description:** Returns a variables type decription. +**Description:** Returns a variables type description. -**Return type:** `string` - -**Arguments:** - -* `var`: `string`, in the range `.*` -* `detail`: `boolean`, in the range: `true,false,yes,no,on,off` +[%CFEngine_function_attributes(var, detail)%] This function returns a variables type description as a string. The function expects a variable identifier as the first argument `var`. An optional second diff --git a/reference/functions/validjson.markdown b/reference/functions/validjson.markdown index 759ca70b5..fdcf229a8 100644 --- a/reference/functions/validjson.markdown +++ b/reference/functions/validjson.markdown @@ -5,12 +5,12 @@ published: true tags: [reference, functions, validjson, JSON, context] --- -[%CFEngine_function_prototype(data_container)%] +[%CFEngine_function_prototype(string)%] -**Description:** Validates a JSON container from `data_container` and returns +**Description:** Validates a JSON container from `string` and returns `true` if the contents are valid JSON. -[%CFEngine_function_attributes(data_container)%] +[%CFEngine_function_attributes(string)%] **Example:** diff --git a/reference/functions/variablesmatching.markdown b/reference/functions/variablesmatching.markdown index 7afda54b5..e9e9401ef 100644 --- a/reference/functions/variablesmatching.markdown +++ b/reference/functions/variablesmatching.markdown @@ -21,7 +21,7 @@ but not `inventory`, *both* are returned by variablesmatching(".*", "inventory", If you want logical AND semantics instead, you can make two calls to the function with one tag in each call and use the `intersection` function on the return values. -Variable tags are set using the [`meta`][Promise Types and Attributes#meta] attribute. +Variable tags are set using the [`meta`][Promise Types#meta] attribute. This function behaves exactly like `variablesmatching_as_data()` but returns just the list of all the variables. If you want their contents as well, see that diff --git a/reference/functions/variablesmatching_as_data.markdown b/reference/functions/variablesmatching_as_data.markdown index 4a34250c4..c16f6d99c 100644 --- a/reference/functions/variablesmatching_as_data.markdown +++ b/reference/functions/variablesmatching_as_data.markdown @@ -22,7 +22,7 @@ but not `inventory`, *both* are returned by variablesmatching_as_data(".*", "inv If you want logical AND semantics instead, you can make two calls to the function with one tag in each call and use the `intersection` function on the return values. -Variable tags are set using the [`meta`][Promise Types and Attributes#meta] attribute. +Variable tags are set using the [`meta`][Promise Types#meta] attribute. This function behaves exactly like `variablesmatching()` but returns a data container with the full contents of all the variables instead of just their diff --git a/reference/language-concepts/augments.markdown b/reference/language-concepts/augments.markdown index 38e1f89e2..2f2bb472f 100644 --- a/reference/language-concepts/augments.markdown +++ b/reference/language-concepts/augments.markdown @@ -9,14 +9,68 @@ tags: [manuals, language, syntax, concepts, augments] Augments files can be used to define variables and classes for use by **all** CFEngine components **before** any parsing or evaluation happen. Augments are fundamentally JSON data files, so you should view and edit them with a JSON-aware editor if -possible. This is a convenient way to override defaults defined in the -Masterfiles Policy Framework without modifying the shipped policy itself. +possible. This is a convenient way to override defaults defined in the default policy, +the Masterfiles Policy Framework (MPF), without modifying the shipped policy files. -There are two canonical augments files, `$(sys.workdir)/data/host_specific.json` -(typically `/var/cfengine/data/host_specific.json`), which is read first and -who's Variables take precedence, and `$(sys.policy_entry_dirname)/def.json` -(typically `/var/cfengine/inputs/def.json`) which may load additional Augments -as specified by the `augments` key. +## Using the MPF without maintaining your own patches to it ## + +As an example, you can add your own policy file to inputs and bundle name to the +bundle sequence, without editing `promises.cf`, by adding the Augments file below +(`/var/cfengine/masterfiles/def.json`): + +```json +{ + "inputs": ["services/my_policy_file.cf"], + "vars": + { + "control_common_bundlesequence_end": ["my_bundle_name"] + } +} +``` + +In this case, the contents of the policy file, `/var/cfengine/masterfiles/services/my_policy_file.cf`, could look something like this: + +```cf3 +bundle agent my_bundle_name +{ + files: + "/tmp/hello" + create => "true", + content => "cfengine"; +} +``` + +You can ensure the file is deleted and use the info log output to confirm that the policy is actually being run: + +``` +$ cf-agent -Kf update.cf && cf-agent -K +$ rm /tmp/hello ; cf-agent -KI + info: Created file '/tmp/hello', mode 0600 + info: Updated content of '/tmp/hello' with content 'cfengine' +``` + +In this example, `control_common_bundlesequence_end` is a special variable, handled by the Masterfiles Policy Framework (MPF). +To learn about more variables like this and ways to interact with the MPF without editing it, see the [MPF Reference documentation][Masterfiles Policy Framework]. +The rest of this documentation page below focuses on the specifics of how augments files work, independently of everything they can be used for in the MPF. + +## Augments Files ## + +There are two canonical augments files, `host_specific.json`, and `def.json` which may load additional Augments +as specified by the [_augments_ key][Augments#augments]. + +**Notes:** +* CFEngine variables are **not** expanded unless otherwise noted. + +### host_specific.json ### + +If `$(sys.workdir)/data/host_specific.json` (typically `/var/cfengine/data/host_specific.json`) is the first augments file that is loaded. Any variables defined as a result of processing this file +are automatically tagged with `source=cmdb`. Variables defined from this file can not be overridden by subsequently processed augments files. Policy always wins and thus _can_ overwrite the variable. + +**Notes:** + +* This file does not support the [_augments_ key][Augments#augments]. + +### def.json ### The file `def.json` is found based on the location of the policy entry (the first policy file read by the agent): @@ -25,26 +79,62 @@ The file `def.json` is found based on the location of the policy entry (the firs * with `-f /dirname/myfile.cf`, it's in `/dirname/def.json` * with `-f ./myfile.cf`, it's in `./def.json` -`def_preferred.json` will be used instead of `def.json` if it is present. This preferential loading can be disabled by providing the `--ignore-preferred-augments` option to the agent. +**Notes:** + +* `sys` variables are expanded in `def.json` and all subsequently loaded augments as specified by the `augments` key. +* `def_preferred.json` will be used instead of `def.json` if it is present. This preferential loading can be disabled by providing the `--ignore-preferred-augments` option to the agent. -Values will be expanded, so you can use the variables from -[Special Variables][]. +## Augments Keys ## An augments file can contain the following keys: -## inputs +### inputs ### + +This key is supported in `def.json`, `def_preferred.json`, and augments loaded by the [_augments_ key][Augments#augments]. -This key is supported in `def.json`, `def_preferred.json`, and augments loaded by the augments key. +Filenames entered here will appear in the `def.augments_inputs` variable. -Any filenames you put here will appear in the `def.augments_inputs` variable. -The standard set of masterfiles refers to this variable and will autoload those -files. +**Notes:** -## variables +* Files are loaded relative to `sys.policy_entry_dirname`. -This key is supported in both `host_specific.json`, `def.json`, `def_preferred.json`, and augments loaded by the augments key. +* The *inputs* key has precedence over the *vars* key. -Variables defined here can target a _namespace_ and or _bundle_ scope explicitly. When defined from `host_specific.json`, variables default to the `main` _bundle_ in the `data` __namespace_ (`$(data:main.MyVariable)`). +* If both the _inputs_ key and `vars.augments_inputs` are populated concurrently, + the variable `def.augments_inputs` will hold the value set by the *inputs* + key. The `def.augments_inputs` variable is part of the default inputs in the + `Masterfiles Policy Framework`. + +**Examples:** + +```json +{ + "inputs": [ "services/hello-world.cf", "example.cf", "/tmp/my_policy.cf" ], + "vars": { + "augments_inputs": [ "goodbye.cf" ] + } +} +``` + +The above Augments results in `$(sys.policy_entry_dirname)/services/hello-world.cf`, `$(sys.policy_entry_dirname)/example.cf` and `/tmp/my_policy.cf` being added to inputs. + + +```json +{ + "vars": { + "augments_inputs": [ "goodbye.cf" ] + } +} +``` + +The above Augments results in `$(sys.policy_entry_dirname)/goodbye.cf` being added to inputs. + +### variables ### + + +This key is supported in both `host_specific.json`, `def.json`, `def_preferred.json`, and augments loaded by the [_augments_ key][Augments#augments]. + +Variables defined here can target a _namespace_ and or _bundle_ scope explicitly. When defined from `host_specific.json`, variables default to the ```variables``` _bundle_ in the ```data``` _namespace_ (`$(data:variables.MyVariable)`). For example: @@ -148,17 +238,22 @@ bundle agent my_bundle } ``` +**Notes:** + +* ```vars``` and ```variables``` keys are allowed concurrently in the same file. +* If ```vars``` and ```variables``` keys in the same augments file define the same variable, the definition provided by the **```variables``` key wins**. + **History:** * Added in 3.18.0 -## vars +### vars ### This key is supported in both `host_sepecific.json`, `def.json`, and `def_preferred.json` and augments loaded by the augments key. Variables defined here can target a _namespace_ and or _bundle_ scope explicitly. When defined from `def.json`, variables default to the `def` -_bundle_ in the `default` __namespace_ (`$(default:def.MyVariable)`). +_bundle_ in the `default` _namespace_ (`$(default:def.MyVariable)`). Thus: @@ -182,7 +277,7 @@ and `MyNamespace:MyBundle.MyVariable` with the value `MyValue` in Again, note that this happens before policy is parsed or evaluated. You can see the list of variables thus defined in the output of `cf-promises ---show-vars` (see [Components and Common Control][]). They will be tagged with +--show-vars` (see [Components][]). They will be tagged with the tag `source=augments_file`. For instance, the above two variables (assuming you placed the data in `$(sys.inputdir)/def.json`) result in @@ -208,11 +303,16 @@ Variables of other types than string can be defined too, like in this example } ``` +**Notes:** + +* ```vars``` and ```variables``` keys are allowed concurrently in the same file. +* If ```vars``` and ```variables``` keys in the same augments file define the same variable, the definition provided by the **```variables``` key wins**. + **History:** * 3.18.0 gained ability to specify the _namespace_ and _bundle_ the variable should be defined in. -## classes +### classes ### This key is supported in both `host_sepecific.json`, `def.json`, `def_preferred.json`, and augments loaded by the augments key. @@ -299,6 +399,7 @@ for use. Thus: ``` results in + * `augments_class_from_rgex_my_always` being always defined. * `augments_class_from_regex_my_other_apache` will be defined if the classes @@ -341,7 +442,7 @@ results in * `myclass_defined_by_augments_in_def_json_3_18_0_v1` will be defined because the expression `cfengine.**` will match at least one defined class, `cfengine` You can see the list of classes thus defined through `def.json` in the output -of `cf-promises --show-classes` (see [Components and Common Control][]). They +of `cf-promises --show-classes` (see [Components][]). They will be tagged with the tags `source=augments_file`. For instance: ```console @@ -366,9 +467,9 @@ myclass_defined_by_augments_in_def_json_3_18_0_v1 optional,tags,sourc * Classes are defined as _soft_ classes instead of _hard_ classes. -## augments +### augments ### -This key is supported in `def.json`, `def_preferred.json`, and augments loaded by the augments key. +This key is supported in `def.json`, `def_preferred.json`, and augments loaded by the [_augments_ key][Augments#augments]. A list of file names that should be merged using `mergedata()` semantic diff --git a/reference/language-concepts/bodies.markdown b/reference/language-concepts/bodies.markdown index a00fef9bc..2bbf193f8 100644 --- a/reference/language-concepts/bodies.markdown +++ b/reference/language-concepts/bodies.markdown @@ -171,7 +171,7 @@ This promise bodies configures the `bundlesequence` to execute on a cf-agent. This promise body defines the clients allowed to connect to a cf-serverd. For more information, see the reference documentation about the [CFEngine -Agents][Components and Common Control] +Agents][Components] #### Default bodies diff --git a/reference/language-concepts/classes.markdown b/reference/language-concepts/classes.markdown index 0a6e6a876..2816e8bcc 100644 --- a/reference/language-concepts/classes.markdown +++ b/reference/language-concepts/classes.markdown @@ -68,7 +68,7 @@ Classes and variables have tags that describe their provenance (who created them) and purpose (why were they created). While you can provide your own tags for soft classes in policy with -the [`meta`][Promise Types and Attributes#meta] attribute, there are some tags applied to hard classes and +the [`meta`][Promise Types#meta] attribute, there are some tags applied to hard classes and other special cases. This list may change in future versions of CFEngine. @@ -112,6 +112,12 @@ day, the week of the year, etc. Time-varying classes (tagged with `time_based`) will change if you do this a few times over the course of a week. +**Notes:** + +* Hard classes can **not** be undefined. If you try to undefine or cancel a hard + class an error will be emitted, for example `error: You cannot cancel a + reserved hard class 'cfengine' in post-condition classes`. + * CFEngine-specific classes * `any`: this class is always set * `am_policy_hub`, `policy_server`: set when the file @@ -122,7 +128,15 @@ of a week. * `opt_dry_run`: set when the `--dry-run` option is given * `failsafe_fallback`: set when the base policy is invalid and the built-in `failsafe.cf` (see `bootstrap.c`) is invoked * (`community`, `community_edition`) and (`enterprise`, `enterprise_edition`): the two different CFEngine products, Community and Enterprise, can be distinguished by these mutually exclusive sets of hard classes - * `agent` *cf-agent*, `server` *cf-serverd*, `monitor` *cf-monitord*, `executor` *cf-execd*, `runagent` *cf-runagent*, `keygenerator` *cf-keygen*, `hub` *cf-hub*, `common` *cf-promises* and others: classes that identify the current component. `cf-promises` is a special case because it's not an agent in the CFEngine sense, so note that using `cf-promises --show-classes` will not show these classes because it can't. + * Component Specific Classes (each component has a class that is always considered defined by that component): + * `cf-agent` :: ```agent``` + * `cf-serverd` :: ```server``` + * `cf-monitord` :: ```monitor``` + * `cf-execd` :: ```executor``` + * `cf-runagent` :: ```runagent``` + * `cf-key` :: ```keygenerator``` + * `cf-hub` :: ```hub``` + * `cf-promises` :: ```common``` * Operating System Classes (note that the presence of these classes doesn't imply platform support) * Operating System Architecture - `arista`, `big_ip`, `debian`, `eos`, `fedora`, `Mandrake`, `Mandriva`, `oracle`, `redhat`, `slackware`, `smartmachine`, `smartos`, `solarisx86`, `sun4`, `SuSE`, `ubuntu`, `ultrix`, the always-favorite `unknown_ostype`, etc. * VM or hypervisor specific: `VMware`, `virt_guest_vz`, `virt_host_vz`, `virt_host_vz_vzps`, `xen`, `xen_dom0`, `xen_domu_hv`, `xen_domu_pv`, `oraclevmserver`, etc. @@ -165,7 +179,7 @@ of a week. * **See also:** `sys.fqhost`, `sys.uqhost`. - An arbitrary user-defined string (as specified in the `-D` command line option, or defined in a [`classes` promise][classes] promise or - [`classes` body][Promise Types and Attributes#classes], + [`classes` body][Promise Types#classes], `restart_class` in a `processes` promise, etc). - The IP address octets of any active interface (in the form `ipv4_192_0_0_1`, `ipv4_192_0_0`, @@ -281,28 +295,34 @@ evaluated. ## Making Decisions based on classes -The easiest way to limit the application of a promise to certain conditions is to use the following notation: +Class guards are the most common way to restrict a promise to a specific context. Once stated the restriction applies until a new context is specified. A new promise type automatically resets to an unrestricted context (the unrestricted context is typically referred to as `any`). + +This example illustrates how a class guard applies (to multiple promises) until a new context is specified. + +[%CFEngine_include_example(classes_context_applies_multiple_promises.cf)%] + +Another Example: ```cf3 - bundle agent greetings - { - reports: - Morning:: - "Good morning!"; +bundle agent greetings +{ + reports: + Morning:: + "Good morning!"; - Evening:: - "Good evening!"; + Evening:: + "Good evening!"; - "! any":: - "This report won't ever be seen."; + "! any":: + "This report won't ever be seen."; - # whitespace allowed only in 3.8 and later - Friday . Evening:: - "It's Friday evening, TGIF!"; + # whitespace allowed only in 3.8 and later + Friday . Evening:: + "It's Friday evening, TGIF!"; - "Monday . Evening":: - "It's Monday evening."; - } + "Monday . Evening":: + "It's Monday evening."; +} ``` In this example, the report "Good morning!" is only printed if the class @@ -316,27 +336,37 @@ The `"Monday . Evening"` context will only be true on Monday evenings. The `Friday . Evening` context will only be true on Friday evenings. See below for more on context operators. -Sometimes it's convenient to put class names in variables. This -example shows two ways to execute code conditionally based on such +### Variable class expressions + +Sometimes it's convenient to put class names in variables. Variables can be used +in quoted class expressions as well as the if and unless promise attributes. + +Variables used in quoted class expressions apply to all promises following the +quoted class expression until the end of the promise type or until a new class +expression is stated. + +**Note:** The double colon (`::`) at the end of the class expression should +remain _outside_ of the quoted string. + +This example shows two ways to execute code conditionally based on such variables: ```cf3 - bundle agent greetings - { - vars: - "myclassname" string => "Evening"; +bundle agent greetings +{ + vars: + "myclassname" string => "Evening"; - reports: - "$(myclassname)":: - "Good evening!"; + reports: + "$(myclassname)":: + "Good evening!"; + "What a wonderful sunset"; - "any":: - "Good evening too!" if => "$(myclassname)"; - } + any:: + "Good evening too!" if => "$(myclassname)"; +} ``` - - As you saw above, the class predicate `if` (`unless` is also available) can be used if variable class expressions are required. It is `AND`ed with the normal class expression, and is @@ -426,9 +456,10 @@ report "Good morning from Italy", this is further qualified by ensuring that the report is only generated if one of the known cities also has a class defined. -**Note:** Classes are automatically canonified when they are defined. Classes -are not automatically canonified when they are checked. +### Automatic canonification on class definition +Classes are automatically canonified when they are defined. Classes +are not automatically canonified when they are checked. This example shows how classes are automatically canonified when they are defined and that you must explicitly canonify when verifying classes. @@ -562,7 +593,7 @@ above and add [`and`][classes#and] or [`xor`][classes#xor] constraints to the single promise. Additionally classes can be defined or undefined as the result of a promise by -using a [classes body][Promise Types and Attributes#classes]. To set a class if +using a [classes body][Promise Types#classes]. To set a class if a promise is repaired, one might write: ```cf3 @@ -572,7 +603,7 @@ a promise is repaired, one might write: ``` These classes are `namespace` scoped by default. The -[`scope`][Promise Types and Attributes#scope] attribute can be used to make them +[`scope`][Promise Types#scope] attribute can be used to make them local to the bundle. It is recommended to use bundle scoped classes whenever possible. This example @@ -669,10 +700,10 @@ The standard library in the Masterfiles Policy Framework contains the [`feature`][lib/feature.cf] bundle which implements a useful model for defining classes for a period of time as well as canceling them on demand. -**See also:** [`persistance` classes attribute][classes#persistence], [`persist_time` in classes body][Promise Types and Attributes#persist_time], [`lib/event.cf`][lib/event.cf] in the MPF, [`lib/feature.cf`][lib/feature.cf] in the MPF +**See also:** [`persistance` classes attribute][classes#persistence], [`persist_time` in classes body][Promise Types#persist_time], [`lib/event.cf`][lib/event.cf] in the MPF, [`lib/feature.cf`][lib/feature.cf] in the MPF ## Canceling classes -You can cancel a class with a [`classes`][Promise Types and Attributes#classes] body. +You can cancel a class with a [`classes`][Promise Types#classes] body. See the `cancel_kept`, `cancel_notkept`, and `cancel_repaired` attributes. diff --git a/reference/language-concepts/modules.markdown b/reference/language-concepts/modules.markdown new file mode 100644 index 000000000..b29db4915 --- /dev/null +++ b/reference/language-concepts/modules.markdown @@ -0,0 +1,88 @@ +--- +layout: default +title: Modules +published: true +tags: [language, concepts, syntax, modules] +--- + +Modules allow users to extend the capabilities of CFEngine in a modular way, they can be easily added and upgraded independently of when you upgrade your CFEngine version. Several different types of modules are available. + +{% comment %} +## cfbs Modules + +cfbs (CFEngine Build System) Modules provide a way to share and consume CFEngine related assets. Policy, Modules and Data can all be distributed as cfbs modules. + +### Specification +{% endcomment %} + +## Promise Modules + +Promise modules allow for the implementation of [*custom* promise types][promise-type-custom], extending the CFEngine Language. They communicate with `cf-agent` using the [*Promise Module Protocol*][promise-type-custom-protocol]. + +**History:** + +* Introduced 3.17.0 + +## Package Modules + +[Package Modules][Package Modules] implement the logic behind *packages* type promises, superseding the *package\_method* based implementation. They interact with package managers like `yum`, `apt`, `msiexec`, and `pip` to determine which packages are currently installed or have updates available as well as installing, upgrading or un-installing packages. + +Package modules communicate with `cf-agent` via the [Package Module Protocol][package-modules-the-api]. + +**History:** + +* Introduced 3.7.0 + +## Variables and Classes Modules + +Variables and Classes Modules are the original way to extend CFEngine. The Variable and Class Module Protocol allows for *variables* and *classes* to be defined. The protocol can be interpreted by functions like [`usemodule()`][usemodule] and [`read_module_protocol()`][read_module_protocol] as well as output from [*commands* type promises][commands] with the [`module => "true"`][commands#module] attribute. + +The choice of interpretation can depend on many factors but a primary differentiate between functions and classes relate to CFEngine's evaluation details. Functions are evaluated early during policy execution unless they are explicitly guarded to delay execution. Commands promises are not executed until the bundle is actuated for it's three pass evaluation. + +Variables and Classes Modules are intended for use as system probes rather than additional configuration promises, especially now that promise modules are available. + +### Specification + +The protocol is *line based*. Lines that begin with `^` apply to all following lines. + +* **`^context=BundleName`:** Sets the bundle scope in which *variables* will be defined +* **`^meta=Tag1,Tag2`:** Sets a comma separated list of tags that are applied to defined *variables* and *classes* +* **`^persistence=X`:** Sets the number of minutes for which *classes* should persist +* **`+ClassName`:** Defines a namespace scoped class +* **`-ClassName`:** Undefines a class +* **`=VariableName=`:** Defines a string variable +* **`VariableName[KEY]=`:** Defines an associative array key value +* **`@VariableName=`:** Defines a list of strings +* **`%VariableName=`:** Must be valid JSON and defines a data container + +**Notes:** + +* It is not possible to define variables or classes in a namespace other than the default (`default`). +* If no context is provided, the context is the canonified leaf name of the module. For example, if the module is `/tmp/path/my-module.sh` the default context would be `my_module_sh` in the `default` namespace (`default:my_module_sh`). +* All *variables* and *classes* will be tagged with `source=module` in addition to any specified tags. +* All lines of output that do not match the module protocol are treated as *errors*. +* Variable names defined by the module protocol are limited to alphanumeric characters and `_`, `.`, `-`, `[`, `]`, `@`, and `/`. + +**Examples:** + +A Variables and Classes module written in shell: + +```sh +#!/bin/sh +/bin/echo "@mylist= { \"one\", \"two\", \"three\" }" +/bin/echo "=myscalar= scalar val" +/bin/echo "=myarray[key]= array key val" +/bin/echo "%mydata=[1,2,3]" +/bin/echo "+module_class" +/bin/echo "^persistence=10" +/bin/echo "+persistent_10_minute_class" +``` + +**History:** + +- Introduced in 3.0.0 +- `^context`, `^meta` Added in 3.6.0 +- `^persistence` Added in 3.8.0 +- `@` allowed in variables (intended for keys in classic array) 3.15.0, 3.12.3, 3.10.7 (2019) +- `/` allowed in variables (intended for keys in classic array) 3.14.0, 3.12.2, 3.10.6 (2019) + diff --git a/reference/package_modules.markdown b/reference/language-concepts/modules/package-module-api.markdown similarity index 92% rename from reference/package_modules.markdown rename to reference/language-concepts/modules/package-module-api.markdown index bdedc2d64..b824bbe95 100644 --- a/reference/package_modules.markdown +++ b/reference/language-concepts/modules/package-module-api.markdown @@ -3,33 +3,29 @@ layout: default title: Package Modules published: true sorting: 70 -tags: [reference, standard library] +tags: [reference, language concepts, modules, package module api] --- -See the [`packages` promises][packages] documentation for a -comprehensive reference on the body types and attributes used here. - - -## Package Module API - -This section describes how to create a package module for the CFEngine package -promise. Package modules are backends that enable the package promise to work +Package modules are back-ends that enable the package promise to work with different types of platform package managers. +See the [`packages` promises][packages] documentation for a +comprehensive reference on the body types and attributes used here. -### The CFEngine side +## The CFEngine side CFEngine never calls any package manager commands, it only ever calls the package module. The information that CFEngine deals in is: - * Which packages are currently installed: - * Name - * Version - * Architecture - * Which of the installed packages have updates available: - * Name - * Version - * Architecture +* Which packages are currently installed: + * Name + * Version + * Architecture + +* Which of the installed packages have updates available: + * Name + * Version + * Architecture These two lists are everything that CFEngine needs to know to decide whether its package promises are fulfilled or not. In addition to this it will carry out @@ -49,21 +45,19 @@ that the promise failed. In the same fashion, if we try to install an update, that update is expected to disappear from the list of available updates after the operation. - -### The package module +## The package module The package module itself is simply an executable, and can be in any executable format, whether than is Python, Perl, Shell script or native binary. -The package module resides in the /var/cfengine/modules/packages both on the hub +The package module resides in the `/var/cfengine/modules/packages` both on the hub and the clients, and out of the box CFEngine is set up to synchronize this directory among its clients. For this reason it is advised to avoid native binary formats for package modules, to reduce the burden of distribution to different platforms, but the API does not forbid it, and it may be useful in some circumstances. - -### The API +## The API The API consists of commands which are passed in the module arguments, combined with a simple text protocol that will be fed on the module's standard input, and @@ -78,8 +72,7 @@ CFEngine itself. The API commands are listed roughly in the order that they should be implemented, in order to facilitate a nice debugging cycle during development. - -#### options attribute +### options attribute All the API commands listed below, except `supports-api-version`, support the `options` attribute. This attribute will contain the contents from the `options` @@ -98,8 +91,7 @@ modules. The `options` attribute is valid in all of the commands except `supports-api-version`, even when the description reads "no input". - -#### supports-api-version +### supports-api-version The very first command that any package module must implement is `supports-api-version`. This command takes no input, and is expected to print @@ -111,14 +103,13 @@ This is the only command which does not support the `options` attribute. Example: -``` +```console $ ./package-module supports-api-version < /dev/null 1 $ ``` - -#### get-package-data +### get-package-data CFEngine uses this command to determine what kind of promise has been made. Currently two types are supported: "file" and "repo". @@ -158,7 +149,7 @@ discrepancies will be handled at the install stage instead. Example 1: -``` +```console $ ./package-module get-package-data < "myspace"; - } -``` +[%CFEngine_include_example(namespace_declaration.cf)%] -### Accessing syntax elements between namespaces and the default namespace +**Notes:** -To distinguish the bundle `mymethod` in the default namespace from one in -another namespace, you prefix the bundle name with the namespace, separated by -a colon. +- Multiple namespaces can be declared within the same file +- The same namespace can be declared in multiple files +- The same namespace can be declared in the same file multiple times -```cf3 - methods: +## Methods|usebundle - "namespace demo" usebundle => myspace:mymethod("arg1"); - "namespace demo" usebundle => mymethod("arg1","arg2"); -``` +Methods promises assume you are referring to a bundle in the same namespace as +the promiser. To refer to a bundle in another namespace you *must* specify the +namespace by prefixing the bundle name with the namespace followed by a colon +(`:`). -To distinguish a body from one in another namespace, you can prefix the body name with the namespace, separated by a colon. +[%CFEngine_include_example(namespace_methods-usebundle.cf)%] -```cf3 - files: - "/file" - create => "true", - perms => name1:settings; -``` +## Bodies -If you don't make any namespace declarations, you'll be in the -`default` namespace. Bundles, bodies, classes, and variables from the -`default` namespace can be accessed like any other: +Bodies are assumed to be within the same namespace as the promiser. To use a body from another namespace the namespace must be specified by prefixing the body name with the namespace followed by a colon (`:`). -```cf3 - files: - "/file" - create => "true", - perms => default:settings; -``` +A common mistake is forgetting to specify `default:` when using bodies from the standard library which resides in the `default` namespace. -If you use the standard library from your own namespace, remember to -specify this `default:` prefix. +[%CFEngine_include_example(namespace_bodies.cf)%] -To access classes, variables, or meta-data in bundles in a different namespace, use the -colon as a namespace prefix: +## Variables - $(namespace:bundle.variable) - $(namespace:bundle_meta.variable) +Variables (except for Special Variables) are assumed to be within the same scope +as the promiser but can also be referenced fully qualified with the namespace. -**Note** that this means that if you are in a namespace that's not `default`, you *must* qualify classes from `default` fully: +[%CFEngine_include_example(namespace_variable_references.cf)%] - default:myclass:: - "do something" if => "default:myotherclass"; +[Special variables][Special Variables] are always accessible without a namespace + prefix. For example, `this`, `mon`, `sys`, and `const` fall in this category. -### Namespacing of classes and variables created in policy +[%CFEngine_include_example(namespace_special_var_exception.cf)%] -In policy, you can't create classes outside your own namespace. So -the following, for example, will create the class `mynamespace:done` -if it runs in the namespace `mynamespace`. +**Notes:** -```cf3 - files: - "/file" - create => "true", - action => if_repaired("done"); -``` +- The [`variables` Augments key][Augments#variables] defines variables in the + `main` bundle of the `data` namespace by default but supports seeding variable + values in any specified namespace. -Similarly, variables you create in a namespaced bundle have to be -prefixed like `mynamespace:mybundle.myvar` from outside your -namespace, but can use `mybundle.myvar` inside the namespace and -`myvar` inside `mybundle`. +## Classes -As a workaround, you could have a helper bundle in another namespace -to create classes and variables as needed. +Promises can only define classes within the current namespace. Classes are +understood to refer to classes in the current namespace if a namespace is not +specified (except for Hard Classes). To refer to a +class in a different namespace prefix the class with the namespace suffixed by a +colon (`:`). -### Exceptions to namespacing rules +[%CFEngine_include_example(namespace_classes.cf)%] -Exceptions to the rules above: +[Hard classes][Classes and Decisions#Hard Classes] exist in all namespaces and +thus can be referred to from any namespace without qualification. -* All hard classes can be used as-is from any namespace, without a namespace - prefix. These are classes like `linux`. They will have the - tag `hardclass`. +[%CFEngine_include_example(namespace_hard_classes.cf)%] -* All special variable contexts, as documented in [Special Variables], - are always accessible without a namespace prefix. For - example, `this`, `mon`, `sys`, and `const` fall in this category. diff --git a/reference/language-concepts/normal-ordering.markdown b/reference/language-concepts/normal-ordering.markdown index 4d0d0ac87..d36e894e9 100644 --- a/reference/language-concepts/normal-ordering.markdown +++ b/reference/language-concepts/normal-ordering.markdown @@ -109,34 +109,35 @@ according to so-called `normal ordering` (essentially deletion first, followed by creation). The actual sequence continues for up to three iterations of the following, converging towards a final state: - meta - vars - defaults - classes - users - files - packages - guest_environments - methods - processes - services - commands - storage - databases - reports +1. [meta][meta] +2. [vars][vars] +3. [defaults][defaults] +4. [classes][classes] +5. [users][users] +6. [files][files] +7. [packages][packages] +8. [guest_environments][guest_environments] +9. [methods][methods] +10. [processes][processes] +11. [services][services] +12. [commands][commands] +13. [storage][storage] +14. [databases][databases] +15. [reports][reports] +16. [Custom promise types, in written order][custom] Within [`edit_line` bundles in files promises][edit_line], the normal ordering is: - meta - vars - defaults - classes - delete_lines - field_edits - insert_lines - replace_patterns - reports +1. [meta][meta] +2. [vars][vars] +3. [defaults][defaults] +4. [classes][classes] +5. [delete_lines][delete_lines] +6. [field_edits][field_edits] +7. [insert_lines][insert_lines] +8. [replace_patterns][replace_patterns] +9. [reports][reports] The order of promises within one of the above types follows their top-down ordering within the bundle itself. In vars this can be used to override the @@ -163,10 +164,10 @@ process with no additional freedoms. Within a server bundle, the normal ordering is: - vars - classes - roles - access +1. [vars][vars] +2. [classes][classes] +3. [roles][roles] +4. [access][access] ## Monitor normal ordering @@ -177,10 +178,10 @@ agent. Within a monitor bundle, the normal ordering is: - vars - classes - measurements - reports +1. [vars][vars] +2. [classes][classes] +3. [measurements][measurements] +4. [reports][reports]