forked from openclarity/apiclarity
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Otel collector plugin (openclarity#255)
* First rev to build OpenTelemetry Collector exporter plugin. * Added build of otel-collector in docker container. * Updated path and example configs to fix errors. * Additional processing to use more defaults for spans without many attributes. * Fixes for linting older package versions. * Fix for attribute initialization. * Add config option to lookup IPs and convert to hostnames. Added more defaults to fill out spans with little information. * Added otel-collector plugin to license-check. * Cleanup of otel-collector docker build process. * Fix for new config "prefer_hostnames". * Add OTel collector with APIClarity to docker and release workflows. * Fixes to normalize otel-collector plugin docker build. * Fixes for docker tags. * Normalized OTel-collector plugin build to match other plugins. * Updated Makefile for otel-collector plugin licensei cache. * Added collector plugin to README as traffic source.
- Loading branch information
Showing
22 changed files
with
2,500 additions
and
9 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,31 @@ | ||
| FROM golang:1.18-alpine3.14 AS otelcol | ||
|
|
||
| RUN apk add --update --no-cache gcc g++ | ||
| ARG OTEL_COLLECTOR_VERSION=0.60.0 | ||
| ADD https://github.com/open-telemetry/opentelemetry-collector/releases/download/v${OTEL_COLLECTOR_VERSION}/ocb_${OTEL_COLLECTOR_VERSION}_linux_amd64 /usr/local/bin/ocb | ||
| RUN chmod 755 /usr/local/bin/ocb | ||
|
|
||
| WORKDIR /otelcol-build | ||
| ADD https://raw.githubusercontent.com/open-telemetry/opentelemetry-collector-releases/v${OTEL_COLLECTOR_VERSION}/distributions/otelcol/manifest.yaml ./ | ||
| COPY otel-collector/scripts/modify_otelcol_builder.awk /usr/local/bin/modify_otelcol_builder.awk | ||
| RUN /usr/local/bin/modify_otelcol_builder.awk <./manifest.yaml >./otelcol-builder-${OTEL_COLLECTOR_VERSION}.yaml | ||
|
|
||
| COPY api ./api/ | ||
| COPY otel-collector/apiclarityexporter ./otel-collector/apiclarityexporter/ | ||
| RUN /usr/local/bin/ocb --config ./otelcol-builder-${OTEL_COLLECTOR_VERSION}.yaml | ||
| # Note that this shouldn't be necessary, but in some cases the file seems to be | ||
| # copied with the execute bit lost (see #1317) | ||
| RUN chmod 755 ./otelcol | ||
|
|
||
| FROM alpine:3.14 as certs | ||
| WORKDIR / | ||
| RUN apk --update add ca-certificates | ||
|
|
||
| ARG USER_UID=10001 | ||
| USER ${USER_UID} | ||
|
|
||
| COPY --from=otelcol /otelcol-build/otelcol / | ||
| COPY otel-collector/configs/otel-collector-config.yaml /etc/otelcol/config.yaml | ||
| ENTRYPOINT ["/otelcol"] | ||
| CMD ["--config", "/etc/otelcol/config.yaml"] | ||
| EXPOSE 4317 8888 55678 55679 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -3,9 +3,10 @@ DOCKER_REGISTRY ?= ghcr.io/openclarity | |
| VERSION ?= $(shell git rev-parse HEAD) | ||
| DOCKER_TAG ?= ${VERSION} | ||
| TYK_VERSION ?= v3.2.2 | ||
| OTEL_COLLECTOR_VERSION ?= 0.60.0 | ||
|
|
||
| .PHONY: docker | ||
| docker: docker-kong docker-tyk docker-taper | ||
| docker: docker-kong docker-tyk docker-taper docker-otelcollector | ||
|
|
||
| .PHONY: docker-kong | ||
| docker-kong: | ||
|
|
@@ -27,8 +28,15 @@ docker-tyk: | |
| go mod edit -replace github.com/openclarity/apiclarity/plugins/[email protected]=./../../../common && \ | ||
| cd -) | ||
|
|
||
| .PHONY: docker-otelcollector | ||
| docker-otelcollector: Dockerfile.otel-collector | ||
| @DOCKER_BUILDKIT=1 docker build \ | ||
| --file Dockerfile.otel-collector \ | ||
| --build-arg OTEL_COLLECTOR_VERSION=${OTEL_COLLECTOR_VERSION} \ | ||
| -t otel-apiclarityexporter:${DOCKER_TAG} . | ||
|
|
||
| .PHONY: push-docker | ||
| push-docker: push-docker-kong push-docker-tyk push-docker-taper | ||
| push-docker: push-docker-kong push-docker-tyk push-docker-taper push-docker-otelcollector | ||
|
|
||
| .PHONY: push-docker-kong | ||
| push-docker-kong: docker-kong | ||
|
|
@@ -44,3 +52,8 @@ push-docker-tyk: docker-tyk | |
| push-docker-taper: docker-taper | ||
| @echo "Publishing passive taper Docker image ..." | ||
| docker push ${DOCKER_REGISTRY}/passive-taper:${DOCKER_TAG} | ||
|
|
||
| .PHONY: push-docker-otelcollector | ||
| push-docker-otelcollector: docker-otelcollector | ||
| @echo "Publishing otel-apiclarityexporter Docker image ..." | ||
| docker push ${DOCKER_REGISTRY}/otel-apiclarityexporter:${DOCKER_TAG} | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,69 @@ | ||
| # APIClarity HTTP Exporter | ||
|
|
||
| | Status | | | ||
| | ------------------------ | --------------------- | | ||
| | Stability | traces [stable] | | ||
| | Supported pipeline types | traces | | ||
| | Distributions | [contrib] | | ||
|
|
||
| Exports traces and/or metrics via HTTP to an [APIClarity]( | ||
| https://github.com/openclarity/apiclarity/blob/master/plugins/api/swagger.yaml) | ||
| endpoint for analysis. | ||
|
|
||
| The following settings are required: | ||
|
|
||
| - `endpoint` (no default): The target base URL to send data to (e.g.: https://example.com:4318). | ||
| The trace signal will be added to this base URL, i.e. "/api/telemetry" will be appended. | ||
|
|
||
| The following settings can be optionally configured: | ||
|
|
||
| - `tls`: see [TLS Configuration Settings](../../config/configtls/README.md) for the full set of available options. | ||
| - `timeout` (default = 30s): HTTP request time limit. For details see https://golang.org/pkg/net/http/#Client | ||
| - `read_buffer_size` (default = 0): ReadBufferSize for HTTP client. | ||
| - `write_buffer_size` (default = 512 * 1024): WriteBufferSize for HTTP client. | ||
|
|
||
| Example: | ||
|
|
||
| ```yaml | ||
| exporters: | ||
| apiclarity: | ||
| endpoint: https://example.com:4318/ | ||
| ``` | ||
| The full list of settings exposed for this exporter are documented [here](./config.go) | ||
| with detailed sample configurations [here](./testdata/config.yaml). | ||
| [contrib]: https://github.com/open-telemetry/opentelemetry-collector-releases/tree/main/distributions/otelcol-contrib | ||
| ## Using the plugin | ||
| In order to use the APIClarity Exporter, you will need to build your own OpenTelemetry Collector with the Exporter included. The instructions to [build a custom collector are here.](https://opentelemetry.io/docs/collector/custom-collector/) | ||
| An example builder-config.yaml file including the exporter could be: | ||
| ```yaml | ||
| dist: | ||
| name: otelcol-api | ||
| description: "OTel Collector distribution with APIClarity support" | ||
| output_path: ./otelcol-api | ||
|
|
||
| exporters: | ||
| - gomod: "github.com/openclarity/apiclarity/plugins/otel-collector/apiclarityexporter v0.0.0" | ||
| - gomod: | ||
| "github.com/open-telemetry/opentelemetry-collector-contrib/exporter/jaegerexporter | ||
| v0.53.0" | ||
| - import: go.opentelemetry.io/collector/exporter/loggingexporter | ||
| gomod: go.opentelemetry.io/collector v0.53.0 | ||
|
|
||
| replaces: | ||
| - github.com/openclarity/apiclarity/plugins/otel-collector/apiclarityexporter v0.0.0 => github.com/openclarity/apiclarity/plugins/otel-collector/apiclarityexporter v0.0.0-20220915093602-8a11adcdb9e1 | ||
| - github.com/openclarity/apiclarity/plugins/api v0.0.0 => github.com/openclarity/apiclarity/plugins/api v0.0.0-20220915093602-8a11adcdb9e1 | ||
|
|
||
| receivers: | ||
| - import: go.opentelemetry.io/collector/receiver/otlpreceiver | ||
| gomod: go.opentelemetry.io/collector v0.53.0 | ||
|
|
||
| processors: | ||
| - import: go.opentelemetry.io/collector/processor/batchprocessor | ||
| gomod: go.opentelemetry.io/collector v0.53.0 | ||
| ``` |
Oops, something went wrong.