Retrying transient HTTP errors with Polly

There are many ways to achieve fault-tolerance in distributed systems. One of the most basic strategies is trivial retry logic, which triggers on certain error conditions. Let’s take a look at how we can implement this for HTTP requests using Polly.

Install Polly via Package Manager.

PM> Install-Package Polly

Now we’ll write a simple class which will retry an HTTP request on specific error conditions.

public class TransientHttpFailureRetryPolicy
{
    private const int MaxRetryCount = 5;

    private const double DurationBetweenRetries = 250;

    private readonly AsyncRetryPolicy<HttpResponseMessage> policy;

    public TransientHttpFailureRetryPolicy()
    {
        this.policy = Policy
            .Handle<HttpRequestException>()
            .Or<TaskCanceledException>()
            .OrResult<HttpResponseMessage>(response => 
                response.StatusCode == HttpStatusCode.RequestTimeout ||
                response.StatusCode == HttpStatusCode.BadGateway ||
                response.StatusCode == HttpStatusCode.GatewayTimeout ||
                response.StatusCode == HttpStatusCode.ServiceUnavailable
            )
            .WaitAndRetryAsync(
                MaxRetryCount,
                retryCount => TimeSpan.FromMilliseconds(DurationBetweenRetries * Math.Pow(2, retryCount - 1))
            );
    }

    public async Task<HttpResponseMessage> ExecuteAsync(Func<Task<HttpResponseMessage>> action)
    {
        return await policy.ExecuteAsync(action);
    }
}

The above class will retry HTTP requests if one of the following conditions is met:

HttpRequestException is thrown during invocation of HTTP request.
TaskCanceledException is thrown during invocation of HTTP request.
Any of the following HTTP status codes returned in the response - 408, 502, 503, 504.

If an HTTP request triggers retry behavior, the policy will execute up to 4 retries with each retry delayed longer than the previous. This strategy, known as exponential backoff, prevents the caller from overwhelming the target server and gives it time to adequately meet demand. If at any point during retrying the request comes back with a response that doesn’t meet retry conditions, the policy will return it back to the caller.

Using the above class is also very trivial as shown in the following example.

var retryPolicy = new TransientHttpFailureRetryPolicy();
var url = $"https://localhost:5001/stock/aapl/quote";

using (var response = await retryPolicy.ExecuteAsync(() => httpClient.SendAsync(new HttpRequestMessage
{
    Method = HttpMethod.Get,
    RequestUri = new Uri(url)
})))
{
    if (response.IsSuccessStatusCode)
    {
        var body = response.Content;

        // Process response body here
    }
}

We can accomplish quite a lot with such a basic retry policy and significantly reduce errors between components in distributed systems.

Happy coding!

Menu

Socials

Retrying transient HTTP errors with Polly