Today, we will review one of the most common dotnet-monitor use cases: running as a side card in a Kubernetes cluster. We recommend having some background in Kubernetes and reviewing our previous articles if you haven't already done so:
In the image above, we are displaying a pod with two containers: one is the application we want to monitor, and the other is the dotnet-monitor. Both are using a shared volume for communication. To get diagnostic data, a user can access the dotnet-monitor API container through port forwarding or set up collection rules to upload the artifact to external storage.
Pre-requisites
Storage
Log into Azure by running az login, and create a storage account and container using the following commands:
az group create -l eastus -n MyResourceGroup
az storage account create --name 24f12674a14143d --resource-group MyResourceGroup --location eastus --sku Standard_ZRS --encryption-services blob
$assignee = az ad signed-in-user show --query objectId -o tsv
az role assignment create --role "Storage Blob Data Contributor" --assignee $assignee --scope "/subscriptions/e759b3f9-6ac3-4f9d-b479-1ba4471235cd/resourceGroups/MyResourceGroup/providers/Microsoft.Storage/storageAccounts/24f12674a14143d"
az storage container create --account-name 24f12674a14143d --name mycontainer --auth-mode login
Obtain the account keys by executing the following command:
az storage account keys list -g MyResourceGroup -n 24f12674a14143d
Container Image
At the project level, we need a Dockerfile to containerize our application. It should include the following content:
FROM mcr.microsoft.com/dotnet/aspnet:6.0 AS base
WORKDIR /app
FROM mcr.microsoft.com/dotnet/sdk:6.0 AS build
COPY ["DotNetMonitorSandBox/DotNetMonitorSandBox.csproj", "DotNetMonitorSandBox/"]
RUN dotnet restore "DotNetMonitorSandBox/DotNetMonitorSandBox.csproj"
COPY . .
WORKDIR "/DotNetMonitorSandBox"
RUN dotnet build "DotNetMonitorSandBox.csproj" -c Release -o /app/build
FROM build AS publish
RUN dotnet publish "DotNetMonitorSandBox.csproj" -c Release -o /app/publish
FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "DotNetMonitorSandBox.dll"]
At the solution level, execute the following command to create the container image:
docker build -t raulnq/dotnetmonitorsandbox:1.0 -f .\DotNetMonitorSandBox\Dockerfile .
Configuration
We will configure dotnet-monitor using a JSON file, which will be mounted in the pod from a config map. The configmap.yaml file will contain the following content:
apiVersion: v1
kind: ConfigMap
metadata:
name: dotnet-monitor-configmap
data:
settings.json: |
{
"Egress": {
"AzureBlobStorage": {
"monitorBlob": {
"accountUri": "https://24f12674a14143d.blob.core.windows.net/",
"containerName": "mycontainer",
"blobPrefix": "artifacts",
"accountKey": "{account-key}"
}
}
},
"DiagnosticPort": {
"ConnectionMode": "Listen",
"EndpointName": "/diag/dotnet-monitor.sock"
},
"CollectionRules": {
"LargeGCHeapSize": {
"Trigger": {
"Type": "EventCounter",
"Settings": {
"ProviderName": "System.Runtime",
"CounterName": "gc-heap-size",
"GreaterThan": 10
}
},
"Actions": [
{
"Type": "CollectGCDump",
"Settings": {
"Egress": "monitorBlob"
}
}
],
"Limits": {
"ActionCount": 2,
"ActionCountSlidingWindowDuration": "1:00:00"
}
}
}
}
In the JSON content, there are three sections of interest (refer to previous articles for details):
Egress: In this section, we define the target Azure Blob Storage to which the diagnostic artifacts will be sent.
DiagnosticPort: We use the listen mode and define the Unix Domain Socket used by the dotnet-monitor.
CollectionRules: In this section, a rule is defined to collect a GCDump when the size of the heap exceeds 10MB.
Deployment
The next step involves defining a deployment.yaml file as follows:
apiVersion: apps/v1
kind: Deployment
metadata:
name: dotnet-monitor-deployment
spec:
replicas: 1
selector:
matchLabels:
app: dotnet-monitor-app
template:
metadata:
labels:
app: dotnet-monitor-app
spec:
restartPolicy: Always
containers:
- name: app
image: raulnq/dotnetmonitorsandbox:1.0
imagePullPolicy: IfNotPresent
env:
- name: DOTNET_DiagnosticPorts
value: /diag/dotnet-monitor.sock,suspend
volumeMounts:
- mountPath: /diag
name: diagvol
resources:
requests:
cpu: 250m
memory: 512Mi
limits:
cpu: 250m
memory: 512Mi
- name: monitor
image: mcr.microsoft.com/dotnet/monitor:8
args: [ "collect", "--no-auth" ]
imagePullPolicy: IfNotPresent
env:
- name: DOTNETMONITOR_Urls
value: http://localhost:52323
volumeMounts:
- mountPath: /diag
name: diagvol
- mountPath: /etc/dotnet-monitor/settings.json
name: configvol
subPath: settings.json
resources:
requests:
cpu: 50m
memory: 32Mi
limits:
cpu: 250m
memory: 256Mi
volumes:
- name: diagvol
emptyDir: {}
- name: configvol
configMap:
name: dotnet-monitor-configmap
The deployment above consists of a couple of volumes and two containers:
volumes
containers
app: Our primary application, in which we establish an environment variable that corresponds to the diagnostic port utilized by dotnet-monitor. In addition to this, we are mounting the emptyDir volume that will be used for communication with the monitor container.
monitor: In this container, we are mounting the same emptyDir volume and the setting file.
Run the following commands to deploy our application to the Kubernetes cluster:
kubectl apply -f .\configmap.yaml
kubectl apply -f .\deployment.yaml
To test dotnet-monitor, we will perform port forwarding for the main application. First, execute the command kubectl get pods to obtain the pod name, and then proceed with the following command:
kubectl port-forward {pod-name} 8081:80
We will simulate a memory leak by repeatedly accessing the endpoint http://localhost:8081/memory-leak. Over time, we will begin to see GC dumps appear in our Azure Blob Storage.
In conclusion, using dotnet-monitor as a sidecar in Kubernetes enables efficient diagnostics and monitoring of .NET applications, simplifying the process of collecting diagnostic data in a containerized environment. Thank you, and happy coding.
