Skip to content

Commit

Permalink
Remove unused / incomplete webhook code, swagger, and readme content. (…
Browse files Browse the repository at this point in the history 
…razee-io#1324)

* Remove unused / incomplete webhook code, swagger, and readme content.
razee-io#147

* linting
  • Loading branch information
@carrolp
carrolp authored May 24, 2023
1 parent a67b0a9 commit cba5a37
Show file tree
Hide file tree
Showing 8 changed files with 947 additions and 2,494 deletions.
244 changes: 0 additions & 244 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -307,250 +307,6 @@ kc_logs.sh razee razeedash-api 1m
Swagger UI is available and if started locally can be accessed via the following
URL: [http://localhost:3333/api-docs/](http://localhost:3333/api-docs/)

## Web hooks

Implemented web hooks so data in Razeedash can be augmented by third-party
services such as test suites and vulnerability scanners.

Components are:

- Web hook creation
- Web hook deletion
- Trigger by resource ID
- Trigger by Regex
- Callback response

### Web hook definition

`POST /v2/webhook/` will create a web hook and the header must contain the `razee-org-key`

JSON body for web hook triggered by cluster change:

```json
{
"cluster_id": "ID of the cluster",
"trigger": "cluster",
"kind": "Deployment",
"field": "searchableData.name",
"filter": "regex string example to match, `(watch-keeper)`",
"service_url": "URL of service to POST upon triggering"
}
```

**Note: field and filter are optional. If not defined the above will fire if a
`deployment` resource kind is changed**

JSON body for web hook triggered by image change:

```json
{
"trigger": "image",
"kind": "image",
"field": "name",
"filter": "regex string example, `(quay.io\\/mynamespace)`",
"service_url": "URL of service to POST upon triggering"
}
```

**Note:** field and filter are optional. If not defined the above will fire if
any new image is deployed on an organization's clusters

- kind: Kind would be the type of resource or in a special case, images
- trigger: cluster, image
- id: ID of the resource used in trigger OR
- field: field dot-notation into JSON to apply a filter
- filter: regex parameter to match if the trigger should fire or not if field
is defined
- service URL: URL to call if the web hook is triggered

### Web hook deletion

`DELETE /v2/webhook/:id` will delete the web hook. Current badge data will not
be affected by the deletion. If a callback for that webhook_id occurs then
the badge would be removed from the resource and 404 sent back to the calling
service.

### Trigger Logic

Trigger points will be added to Razeedash API for when:

- New image is deployed
- A resource kind is deployed or modified on a specific cluster

Trigger will also have to pass regex filter in order to fire the web hook. If
determine to call the web hook the service URL is called and relevant data
posted along with a callback URL. The called service can augment the data by
calling the callback URL.

Razeedash API will look up the resource to make sure it is still being used and
if not, return a 404 indicating to the calling service it should no longer
provide updates on this web hook. If found the badge information will added
to resource.

Calling the service by cluster trigger POSTs:

```json
{
"cluster_id": "ID of the cluster",
"cluster_metadata": "JSON array for the cluster metadata",
"resource_id": "ID of the resource to badge",
"resource_kind": "What kind of resource",
"resource": "JSON of the resource object",
"webhook_id": "ID of the webhook definition",
"callback_url": "URL to POST badge data"
}
```

Calling the service by image trigger POSTs:

```json
{
"image_id": "ID of the image",
"image_name": "Image name",
"webhook_id": "ID of the webhook definition",
"callback_url": "URL to POST badge data"
}
```

**Scenario:** Trigger by resource ID

User defines a web hook that if a deployment changes or something new is added
to call a web hook to run integration tests.

```json
{
"kind": "Deployment",
"trigger": "cluster",
"id": "fb56c61b676844d292f1f18e719c31f2",
"service_url": "https://my.testingservice.com/run_integration"
}
```

New deployment is rolled out to cluster designated as the "staging" cluster
environment. Web hook is called to service_url and the deployment.yaml, org_id,
cluster_id are all posted along with a a call back URL.

The testing service calls the callback with badge data:

- badge: URL of a running man
- description: "Running tests"
- link: (link to the live tests being run)
- status: info

Testing service runs integration tests on the staging cluster and and calls the
callback with badge data:

- badge: URL of a green circle
- description: "All tests completed successfully"
- link: (link to the test logs)
- status: info

**Scenario:** Trigger by regex

User defines a web hook that if a deployment changes or something new is added
to call a web hook to run integration tests.

```json
{
"kind": "image",
"trigger": "image",
"field": "name",
"filter": "(quay.io\/mynamespace)",
"service_url": "https://my.quayscanner.com/check"
}
```

When a new image is deployed, the name of the image is checked against the
filter, if defined, and then the service_url is called with image name, image
ID, org_id and callback URL.

The scanner service calls the callback with badge data:

- badge: URL image of binoculars
- description: "Looking for vulnerabilities"
- link: (link to service of that image being checked)
- status: info

Razeedash API checks to see if the image is still in use and returns a 201

The image is scanned and shown clean and will be rechecked by the service in 24
hours. In the meantime it calls the callback with badge data:

- badge: URL image of green circle
- description: "no vulnerabilities"
- link: (link to service of that image results)
- status: info

Razeedash API checks to see if the image is still in use and returns a 201

24 hours later a minor vulnerabilities is discovered and the service calls the
callback again with badge data:

- badge: URL image of yellow circle
- description: "minor vulnerabilities detected"
- link: (link to service of that image results)
- status: warning

Razeedash API checks to see if the image is still in use and returns a 201

24 hours later a major vulnerability is discovered and the service calls the
callback again with badge data:

- badge: URL image of red circle
- description: "major vulnerabilities detected"
- link: (link to service of that image results)
- status: error

Razeedash API checks to see if the image is still in use and finds it is not and
returns a 404 to the vulnerability service. The vulnerability service should
then stop reporting that image security issues from now on.

**Scenario:** Filter a specific resource

User defines a web hook that filters for a specific resource. In this case we
are looking for new or changed deployments where the field `metadata.name`
matches `watch-keeper` on a specific cluster to trigger the web hook.

```json
{
"kind": "Deployment",
"field": "metadata.name",
"filter": "(watch-keeper)",
"trigger": "cluster",
"id": "fb56c61b676844d292f1f18e719c31f2",
"service_url": "https://my.testingservice.com/run_integration"
}
```

New deployment is rolled out to cluster designated as the "staging" cluster
environment. Web hook is called to service_url and the deployment.yaml, org_id,
cluster_id are all posted along with a a call back URL.

### Callback response from remote service

When the remote service wants to add a badge as a result of the web hook call,
the POST to /v2/callback should have the following body:

Header should contain the `razee-org-key`

```json
{
"webhook_id": "id of the webhook from initial call to service",
"url": "URL of the badge image",
"description": "short description of badge",
"link": "URL link for details",
"status": "info | error | warning"
}
```

Razeedash API will accept the callback URL and make sure the webhook and
resource is still valid. If the resource is no longer in use or the webhook
was deleted then the callback response will return a 404. If valid it will add
the augmented data to the resource.

The resource will have a new attribute `badges`. The badge will replace any
existing badge with the same webhook_id or if it does not exist, add to the array.

## GraphQL for local development

```shell
Expand Down
119 changes: 6 additions & 113 deletions app/routes/v2/webhooks.js
View file
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
/**
* Copyright 2019 IBM Corp. All Rights Reserved.
* Copyright 2023 IBM Corp. All Rights Reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
Expand All @@ -14,115 +14,8 @@
* limitations under the License.
*/

const express = require('express');
const router = express.Router();
const asyncHandler = require('express-async-handler');
const jkValidate = require('json-key-validate');

const { WEBHOOK_TRIGGER_CLUSTER, WEBHOOK_TRIGGER_IMAGE, insertClusterBadge, insertImageBadge } = require('../../utils/webhook.js');

// Callback from triggered webhook
const addCallbackResult = async (req, res, next) => {
try {
const Webhooks = req.db.collection('webhooks');
const webhook = await Webhooks.findOne({ _id: req.params.webhook_id });
if (webhook) {
// Make sure webhook is still active
if (webhook.deleted == true) {
res.status(404).send('Web hook has been deleted');
} else {
// Validate badge properties
let badge = req.body;
badge.webhook_id = req.params.webhook_id;
let properties = ['webhook_id', 'url', 'description', 'link', 'status'];
if (webhook.trigger == WEBHOOK_TRIGGER_IMAGE) {
properties.push('image_id');
}
let isValid = jkValidate(badge, properties);
if (!isValid) {
res.status(400).send(`Missing properties, make sure the following fields are defined: ${JSON.stringify(properties)}`);
} else {
// determine resource to add badge
// if resource not currently used return 404
if (webhook.trigger == WEBHOOK_TRIGGER_CLUSTER) {
const cluster = await insertClusterBadge(webhook, badge, req);
if (cluster) {
res.status(201);
} else { // should never happen
res.log.error({ badge: badge, webhook: webhook }, 'cluster missing while processing badge ');
res.status(500);
}
} else if (webhook.trigger == WEBHOOK_TRIGGER_IMAGE) {
const image = await insertImageBadge(badge, req);
if (image) {
res.status(201);
} else { // should never happen
res.log.error({ badge: badge, webhook: webhook }, 'image missing while processing badge ');
res.status(500);
}
} else { // should never happen
res.log.error({ badge: badge, webhook: webhook }, 'Unknown webhook trigger defined in database');
res.status(500).send('unknown trigger');
}
}
}
} else {
res.status(404).send('Web hook not found');
}
} catch (err) {
req.log.error(err);
next(err);
}
};

const addWebhook = async (req, res, next) => {
try {
let webhook = req.body;
webhook.org_id = req.org._id;
const Webhooks = req.db.collection('webhooks');
if (webhook.trigger == WEBHOOK_TRIGGER_CLUSTER) {
const Clusters = req.db.collection('clusters');
const result = await Clusters.findOne({ cluster_id: webhook.cluster_id, org_id: req.org._id });
if ((!result) || (result && result.deleted == true)) {
res.status(404).send(`Cluster ${webhook.cluster_id} not found or has been deleted`);
} else {
await Webhooks.insertOne(webhook);
res.status(201);
}
} else {
await Webhooks.insertOne(webhook);
res.status(201);
}
} catch (err) {
req.log.error(err);
next(err);
}
};

// deleteWebhook - logical delete of webhook
const deleteWebhook = async (req, res, next) => {
try {
const Webhooks = req.db.collection('webhooks');
await Webhooks.updateOne(
{ _id: req.params.webhook_id },
{
$set: { deleted: true },
$currentDate: { lastModified: true }
});
res.status(204);
} catch (err) {
req.log.error(err.message);
next(err);
}
};

// POST /api/v2/webhooks/:id/callback
router.post('/:webhook_id/callback', asyncHandler(addCallbackResult));

// POST /api/v2/webhooks
router.post('/', asyncHandler(addWebhook));

// DELETE /api/v2/webhooks/:id
router.delete('/:webhook_id', asyncHandler(deleteWebhook));

module.exports = router;
/*
Webhook support was partially implemented for Razeedash-api/issues/62, but never exposed.
If the functionality is needed in the future, and assuming that requirements are similar,
revisit PRs for that issue and the PR that introduced this comment to potentially get started.
*/
Loading

0 comments on commit cba5a37

Please sign in to comment.