Health checks (probes) are a powerful feature of Kubernetes and make sure your container is healthy. A separate endpoint, next to your regular endpoint, tells using an HTTP status code if your service is healthy or not. You decide the implementation of the check that is executed. Let’s see what needs to be done implementing a .NET 6 gRPC health check.
There is a lot of documentation from Microsoft regarding health checks in .NET Core:
https://docs.microsoft.com/en-us/aspnet/core/host-and-deploy/health-checks?view=aspnetcore-6.0
This blog post shows the steps that need to be taken and shows how to configure it in your Docker container and Kubernetes. Which is missing in the Microsoft documentation.
The proto file
The proto file for gRPC health check can be found here: https://github.com/grpc/grpc/blob/master/doc/health-checking.md
We are adding the following line to decide in which namespace the code is generated:option csharp_namespace = "MySolution.Service.API";
This results in the following full proto file:
syntax = "proto3";
option csharp_namespace = "MySolution.Service.API";
package grpc.health.v1;
service Health {
rpc Check(HealthCheckRequest) returns (HealthCheckResponse);
rpc Watch(HealthCheckRequest) returns (stream HealthCheckResponse);
}
message HealthCheckRequest {
string service = 1;
}
message HealthCheckResponse {
enum ServingStatus {
UNKNOWN = 0;
SERVING = 1;
NOT_SERVING = 2;
SERVICE_UNKNOWN = 3; // Used only by the Watch method.
}
ServingStatus status = 1;
}
Implement the HealthService in c#
The base class is generated based on the proto.
public class HealthService : Health.HealthBase
{
private readonly HealthCheckService healthCheckService;
private readonly ILogger<HealthService> logger;
public HealthService(HealthCheckService healthCheckService, ILogger<HealthService> logger)
{
this.healthCheckService = healthCheckService;
this.logger = logger;
}
public async override Task<HealthCheckResponse> Check(HealthCheckRequest request, ServerCallContext context)
{
logger.LogDebug("Starting HealthCheck");
var result = await healthCheckService.CheckHealthAsync(context.CancellationToken);
var status = result.Status == HealthStatus.Healthy ? ServingStatus.Serving : ServingStatus.NotServing;
return new HealthCheckResponse
{
Status = status
};
}
}
Create a Health check
There are ready-to-use health checks available. Like a health check for SQL Azure.
Let’s create a custom health check. In this sample, it’s a health check on the availability of Keycloak. This can be checked using a HTTP-get call.
public class KeycloakConnectionHealthCheck : IHealthCheck
{
private readonly IHttpClientFactory httpClientFactory;
private readonly IOptions<KeyCloakSettings> configuration;
private readonly ILogger<KeycloakConnectionHealthCheck> logger;
public KeycloakConnectionHealthCheck(IHttpClientFactory httpClientFactory, IOptions<KeyCloakSettings> configuration, ILogger<KeycloakConnectionHealthCheck> logger)
{
this.httpClientFactory = httpClientFactory;
this.configuration = configuration;
this.logger = logger;
}
public async Task<HealthCheckResult> CheckHealthAsync(HealthCheckContext context, CancellationToken cancellationToken = default)
{
logger.LogDebug("Starting CheckHealthAsync");
try
{
var httpRequestMessage = new HttpRequestMessage(
HttpMethod.Get,
configuration.Value.Uri)
{ };
var httpClient = httpClientFactory.CreateClient("keycloak");
var httpResponseMessage = await httpClient.SendAsync(httpRequestMessage);
if (httpResponseMessage.StatusCode != System.Net.HttpStatusCode.OK)
return HealthCheckResult.Unhealthy();
return HealthCheckResult.Healthy();
}
catch (Exception ex)
{
logger.LogError(ex, "Exception occured during CheckHealthAsync");
return HealthCheckResult.Unhealthy();
}
}
}
Register the Health check
We want to access the health service on endpoint /health:
builder.Services.AddHealthChecks()
.AddCheck<KeycloakConnectionHealthCheck>("keycloak");
app.MapGrpcService<HealthService>();
app.MapHealthChecks("/health");
Because the health check in the example is using the httpclientFactory the following registration is needed as well:
builder.Services.AddHttpClient();
The service with health checks is ready to use now. When hosting in Kubernetes we need to do some things more.
Hosting in Kubernetes
Kubernetes offers several types of health checks like livenessProbe and readinessProbe. A gRPC endpoint is supported by Kubernetes from version 1.23 or higher. (see the documentation)
We are running an older version of Kubernetes. Therefore we need to apply the following 2 steps:
1. Extend the Dockerfile.
It should include a little command to check gRPC endpoints. Called grpc_health_probe-linux-amd64:
Add the following for example right above the last line (ENTRYPOINT….)
RUN apt-get update && apt-get install -y wget
RUN GRPC_HEALTH_PROBE_VERSION=v0.4.6 && \
wget -qO/bin/grpc_health_probe https://github.com/grpc-ecosystem/grpc-health-probe/releases/download/${GRPC_HEALTH_PROBE_VERSION}/grpc_health_probe-linux-amd64 && \
chmod +x /bin/grpc_health_probe
2. Configure the Kubernetes deployment.yaml
The command should be called from the probes configured in Kubernetes. Like:
livenessProbe:
exec:
command: [“/bin/grpc_health_probe”, “-addr=:80”]
Closing words
We have executed all steps to configure health checks in our .NET 6 gRPC service. And to configure it right in Kubernetes, so the health endpoint is actually called.
Pascal Naber 