Logging and Telemetry

We use Azure’s Application Insights for Application Performance Monitoring. This ingests logs, telemetry, metrics and much more, and provides solutions for querying, charts, dashboards, transaction tracing and other APM goodies.

It should be noted that this is provided Digital Studio Ops and not Cloud Platform.

Using Application Insights

Access to App Insights is managed by the Digital Studio Ops team via Azure Active Directory. They provide a self-service mechanism for requesting access. Check their Slack channel #ask-digital-studio-ops for examples and more information.

App Insights provides many features and it’s worth reading their documentation to get an overview of what’s available.

We generally use Log Analytics to query the logs, telemetry and metrics published by Kotlin apps.

Publishing to Application Insights

Logging

We use slf4j for our logging facade. There are many examples of how to create an instance of a logger.

Note that by default the Kotlin template sets your application logging level to DEBUG and that debug logs are ingested into App Insights. Therefore be careful not to include PII in your debug logs, and be aware that excessive debug logging will end up in App Insights.

Telemetry / Structured Logging

Microsoft provides a client for publishing telemetry to App Insights. This is available in your application as a Spring Bean and there are many examples of using the TelemetryClient.

The ability to pass key/value pairs to the telemetry client makes it easy to add structured logging to your application; the key/value pairs can be queried in Log Analytics.

Exceptions, JVM Metrics, HTTP Requests and more

Spring Actuator is provided by our Kotlin library and this publishes a whole load of other stuff to App Insights. See the various tables in Log Analytics and try querying them to see what’s available.

Application Configuration

Logback

We use logback for our logging framework and the Kotlin template provides a configuration xml with sensible defaults. This should work out of the box but be aware this is where to tweak your logging output.

Note that Spring provides a mechanism to override the logging levels in its configuration properties. This is useful when running locally but we don’t want logging levels defined in 2 places - stick to logback for permanent logging level changes.

Sending logs to Application Insights

The Kotlin template has an App Insights agent added to its Dockerfile and a connection secret mapped by its helm values file. This means that when a new project is bootstrapped it will send logs, telemetry and metrics to App Insights; you don’t need to do anything.

Note that when you created your namespace in Cloud Platform the application_insights_instance chosen determines which App Insights instance your logs are sent to. Be careful this is accurate or your production logs could end up in dev.

Sampling in Application Insights

App Insights supports sampling to allow fine-grained control over how much data is ingested. We have a huge footprint in App Insights and sampling is useful for limiting “noise”, e.g. we only ingest 10% of /health requests by default. Be aware that your project has configuration files applicationinsights.json and applicationinsights.dev.json to control sampling and might be needed to reduce your App Insights footprint.