Skip to main content

Configure Masking Rules

These instructions apply to the Docker Compose client. If you are using the K8S operator, please refer to the alternative instructions provided.

Step 1: Open a Shell in the OTel Container​

Under Docker Compose Client​

  1. Open a shell in the OTel container:

    docker exec -it datorios-roei-cluster-1-otel-collector-worker-1 bash

    Note: Replace roei-cluster-1 with your cluster's name where applicable.

Under K8S Operator​

  1. Find the OTel pod:

    kubectl get pods | grep -i otel

    Example output:

    flink-otel-collector-xxxxxxxxx-xxxxx 1/1 Running 0 13h

  2. Open a shell in the OTel container:

    kubectl exec -it flink-otel-collector-xxxxxxxxx-xxxxx -- /bin/bash

Step 2: Edit the OTel Configuration File​

  1. Update the container to include a text editor:

    apt update && apt install vim -y

  2. Open the configuration file:

    vi /otel-collector/worker.conf

  3. Add the following sections to the file:

    receivers:
      ...
    processors:
      transform:
        error_mode: propagate
        log_statements:
        - context: log
          statements:
          - replace_pattern(body, "\\b4[0-9]{12}(?:[0-9]{3})?\\b", "$$1", SHA1, "credit_card_%s") # Visa
    exporters:
      ...
    services:
      pipelines:
        logs:
          ...
          processors: [transform]
          ...

  4. Save and exit the file:

    • Press ESC, type :wq, and press Enter to save and exit Vim.
    • Close the worker shell if needed.

Step 3: Apply the New Configuration​

  1. Restart the configuration process: 
    kill -HUP 1

Notes​

Under Docker Compose Client​

  • These instructions are for a cluster named roei-cluster-1. Replace the name where applicable.
  • If the OTel collector scale is greater than 1, repeat these steps for each worker individually by replacing worker-1 with worker-<index>.
  • Configuration is not persistent: Restarting the Flink cluster will revert to the default configuration.

Under K8S Operator​

  • Follow similar steps for each worker if the scale of the OTel collector is greater than 1. Replace worker-1 with worker-<index> where applicable.
  • Configuration is not persistent: Restarting the Flink cluster will revert to the default configuration.