Basic troubleshooting steps for the Logging Operator
This document (000020946) is provided subject to the disclaimer at the end of this document.
Environment
Rancher 2.7.x
Rancher 2.8.x
Logging Operator V2
Situation
Resolution
1. On the Rancher menu, go to Workload, Pods.
Check the rancher-logging-root-fluentd-configcheck pod. Its status should be `Completed`.
If it is in a different state, check its logs to understand the error on your configuration.
You may do so by clicking on the three dots menu and 'View Logs'.
2. Check the fluentd logs.
a) When using logging-operator version +up3.17.10 and earlier, they are not written to standard output. Executing on the container rancher-logging-root-fluentd-0, navigate to /fluentd/out, and check the log file there:
$ kubectl -n cattle-logging-system exec -it rancher-logging-root-fluentd-0 -- sh $ cat /fluentd/out/log
b) When using logging-operator version +up4.4.0 and later, the fluentd logs are written to standard output, in the container versus a file, i.e. they are written to the fluentd Pod logs (and excluded from collection by the "fluentbit.io/exclude: 'true'" annotation on the fluentd Pod). To check the fluentd logs you can therefore check the Pod logs:
$ kubectl logs rancher-logging-root-fluentd-0 -n cattle-logging-system
If there are any errors, you can try to fix them or open a case if you need more help.
3. If there are no errors in the two first steps, a simple way to check if the logs are being scrapped is to send them to a file. An output YAML would look like this:
apiVersion: logging.banzaicloud.io/v1beta1 kind: Output metadata: name: fileoutput namespace: default spec: file: path: /tmp/${tag}Note the ${tag} on the path, required by Fluentd.
In this case, the logs will be available on the /tmp folder of the Pod rancher-logging-root-fluentd-0
If logs are not written to the /tmp folder, open a case with us to help you.
4. If logs are flowing to the file, you need to check your Outputs.
Tweaking the buffer values might be a solution.
Disclaimer
This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.
- Document ID:000020946
- Creation Date: 20-Jan-2023
- Modified Date:04-Sep-2024
-
- SUSE Rancher
For questions or concerns with the SUSE Knowledgebase please contact: tidfeedback[at]suse.com