Monitor Che server

The Che server exposes JVM metrics such as memory usage and class loading on port 8087 on the /metrics endpoint. Monitoring these metrics helps administrators identify performance bottlenecks and plan server capacity.

Enable Che server metrics

Che exposes the JVM metrics on port 8087 of the che-host Service. Configure this behavior to support performance monitoring and capacity planning.

Prerequisites
  • An active kubectl session with administrative permissions to the destination Kubernetes cluster. See Overview of kubectl.

Procedure
  1. Configure the CheCluster Custom Resource. See Using the CLI to configure the CheCluster Custom Resource.

    spec:
      components:
        metrics:
          enable: <boolean> (1)
    1 true to enable, false to disable.
Verification
  • Verify the metrics endpoint is accessible:

    $ kubectl get service che-host -n eclipse-che -o jsonpath='{.spec.ports[?(@.port==8087)]}'

Verify Che server metrics with Prometheus

Verify that Che Server JVM metrics are available in Prometheus. The che-operator automatically creates and reconciles the required Prometheus resources (ServiceMonitor, Role, and RoleBinding) and configures namespace labeling.

Verification
  1. In the Administrator view of the OpenShift web console, go to ObserveMetrics.

  2. Run a PromQL query to confirm that the metrics are available. For example, enter process_uptime_seconds{job="che-host"} and click Run queries.

To troubleshoot missing metrics, view the Prometheus container logs for possible RBAC-related errors:

  1. Get the name of the Prometheus pod:

    $ oc get pods -l app.kubernetes.io/name=prometheus -n openshift-monitoring -o=jsonpath='{.items[*].metadata.name}'
  2. Print the last 20 lines of the Prometheus container logs from the Prometheus pod from the previous step:

    $ oc logs --tail=20 <prometheus_pod_name> -c prometheus -n openshift-monitoring

View Che Server from an Kubernetes web console dashboard

View Che Server JVM metrics on a custom dashboard in the Administrator perspective of the Kubernetes web console. This dashboard helps you identify performance bottlenecks and monitor server health.

Prerequisites
Procedure
  • Create a ConfigMap for the dashboard definition in the openshift-config-managed namespace and apply the necessary label.

    1. $ oc create configmap grafana-dashboard-devspaces-server \
        --from-literal=devspaces-server-dashboard.json="$(curl https://raw.githubusercontent.com/eclipse-che/che-server/main/docs/grafana/openshift-console-dashboard.json)" \
        -n openshift-config-managed
    2. $ oc label configmap grafana-dashboard-devspaces-server console.openshift.io/dashboard=true -n openshift-config-managed
      The dashboard definition is based on Grafana 6.x dashboards. Not all Grafana 6.x dashboard features are supported in the OpenShift web console.
Verification steps
  1. In the Administrator view of the OpenShift web console, go to ObserveDashboards.

  2. Go to DashboardChe Server JVM and verify that the dashboard panels contain data.

    The *JVM quick facts* panel
    Figure 1. Quick Facts
    The *JVM Memory* panel
    Figure 2. JVM Memory
    The *JVM Misc* panel
    Figure 3. JVM Misc
    The *JVM Memory Pools (heap)* panel
    Figure 4. JVM Memory Pools (heap)
    The *JVM Memory Pools (non-heap)* panel
    Figure 5. JVM Memory Pools (Non-Heap)
    The *JVM garbage collection* panel
    Figure 6. Garbage Collection
    The *JVM class loading* panel
    Figure 7. Class loading
    The *JVM buffer pools* panel
    Figure 8. Buffer Pools