Fluentd/EFK Integration with Enterprise PKS
search cancel

Fluentd/EFK Integration with Enterprise PKS

book

Article ID: 316815

calendar_today

Updated On:

Products

VMware

Issue/Introduction

Symptoms:
  • EFK Integration with Enterprise PKS is not working; data is not flowing into Elastic search cluster and so no data is available in Kibana UI.
  • You see messages similar to the following in the fluentd pod logs:

=======
2019-06-12 11:45:50 +0000 [warn]: /var/log/containers/es-cluster-0_kube-logging_increase-vm-max-map-bec6d8ecad62b38684d904405986b97ad8e9fa392b289304540a34a584b6c821.log unreadable. It is excluded and would be examined next time.
2019-06-12 11:45:50 +0000 [warn]: /var/log/containers/kubernetes-dashboard-5f4b59b97f-b2dvv_kube-system_kubernetes-dashboard-fd206f70baa36db15330004376b24bb17b0eb488fafa6b2bd9797af3ef8d5d56.log unreadable. It is excluded and would be examined next time.
2019-06-12 11:45:50 +0000 [warn]: /var/log/containers/coffee-rc-jvzlk_default_coffee-223ebab59b8c7e7064e6795c374f523d625845aaad6507500efa9ca03fba33e7.log unreadable. It is excluded and would be examined next time.
2019-06-12 11:45:50 +0000 [warn]: /var/log/containers/es-cluster-0_kube-logging_fix-permissions-d11b748f6806147465307bd57ca5c5e992adc5ba37c9b2aca133bb4f2c7a1c8c.log unreadable. It is excluded and would be examined next time.
2019-06-12 11:45:50 +0000 [warn]: /var/log/containers/event-controller-6c77ddd949-7dd2t_pks-system_ghostunnel-fce7ba5edbc8dca58e3bc679fdbe0c655c519b92853931469829d4b9de304317.log unreadable. It is excluded and would be examined next time.
2019-06-12 11:45:50 +0000 [warn]: /var/log/containers/event-controller-6c77ddd949-7dd2t_pks-system_concat-keystore-40c172fd5463b8cdf167eb461d47f85c7f6280bf9b987eb26f41382ee27d973c.log unreadable. It is excluded and would be examined next time.
2019-06-12 11:45:50 +0000 [warn]: /var/log/containers/tea-rc-s7vqg_default_tea-d5bbc16bc7ea5f649b2ad386efefd75d9665bf3b6b661721987706e1ccd22761.log unreadable. It is excluded and would be examined next time.

==============

Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.


Environment

VMware PKS 1.x

Resolution

  • To ensure that the fluentd pod is run as the root user, add the following line in the fluentd deployment file under the containers.env.name/value  spec.container section:

=======

         - name: FLUENT_UID
            value: "0"
=======

  • To change the volume mount to point to the default docker directory in Enterprise PKS, ensure the volume mounts and volume section in the fluentd deployment file look similar to the following:

========

        volumeMounts:
        - name: varlog
          mountPath: /var/log
        - name: varlibdockercontainers
          mountPath: /var/vcap/store/docker/docker/containers
          readOnly: true
      terminationGracePeriodSeconds: 30
      volumes:
      - name: varlog
        hostPath:
          path: /var/log
      - name: varlibdockercontainers
        hostPath:
          path: /var/vcap/store/docker/docker/containers

 

========


Additional Information

To be alerted when this article is updated, click the subscribe button [cid:[email protected]] . For more information on KB subscription features, see the Knowledge Base Article FAQs: How to Subscribe to VMware Knowledge Base Articles (76417).