Skip to main content

Configure Collection Verbosity

Under Docker Compose Client​

Viewing the Current Configuration​

To view the current configuration file in all Task Manager instances, use the following command:

for name in `docker ps --format "{{.Names}}" | grep datorios-roei-cluster-1-taskmanager`; do
    echo $name
    docker exec $name cat /datorios/probes.config
    echo
done

This command performs the following steps:

  1. Lists all Docker container names matching the pattern datorios-roei-cluster-1-taskmanager.
  2. Prints the container name for each Task Manager instance.
  3. Executes the cat /datorios/probes.config command inside each container to display the configuration file contents.
  4. Adds an empty line for better readability between outputs.

Changing the Config File in All Task Manager Instances​

To modify the configuration file in all Task Manager instances (e.g., disabling state probes), use the following command:

for name in `docker ps --format "{{.Names}}" | grep datorios-roei-cluster-1-taskmanager`; do
    docker exec $name sed -i 's/STATES=true/STATES=false/g' /datorios/probes.config
done

This command performs the following steps:

  1. Lists all Docker container names matching the pattern datorios-roei-cluster-1-taskmanager.
  2. Executes the sed command inside each container to update the probes.config file, changing the value of STATES from true to false.

Verifying the Applied Config in Task Manager Logs​

To verify that the configuration has been applied by checking the Task Manager logs, use the following command:

for name in `docker ps --format "{{.Names}}" | grep datorios-roei-cluster-1-taskmanager`; do
    echo $name
    docker logs $name 2>&1 | grep Reloading
    echo
done

This command performs the following steps:

  1. Lists all Docker container names matching the pattern datorios-roei-cluster-1-taskmanager.
  2. Prints the container name for each Task Manager instance.
  3. Searches the logs of each container for entries containing the word "Reloading", indicating the configuration has been reloaded.
  4. Adds an empty line for better readability between outputs.

Under K8S Operator​

Adding a Lifecycle Hook for Configuration​

To modify the configuration for Task Manager instances in Kubernetes, update the deployment-flink.yaml file with a postStart lifecycle hook under taskManager:

lifecycle:
  postStart:
    exec:
      command:
        - /bin/sh
        - -c
        - sed -i 's/WINDOWS=true/WINDOWS=false/g' /datorios/probes.config

Full Section Example:​

taskManager:
  resource:
    memory: "2048m"
    cpu: 1
  podTemplate:
    spec:
      containers:
        - name: flink-main-container
          image: public.ecr.aws/v3p0m7u7/metro-flink-public:1.17.2
          lifecycle:
            postStart:
              exec:
                command:
                  - /bin/sh
                  - -c
                  - sed -i 's/WINDOWS=true/WINDOWS=false/g' /datorios/probes.config

Notes:

  • This hook will apply the configuration when the Task Manager launches. No further runtime changes are needed.
  • You do not need to delete the deployment; reapplying it will recreate the configuration.

Probes You Can Enable/Disable​

There are three types of probes you can configure in the probes.config file:

  1. WINDOWS
  2. STATES
  3. RECORDS

Set each probe to true or false to enable or disable its collection.

These commands and configurations allow you to effectively manage and verify the verbosity settings for your Task Manager instances.