Failed to get user avatar from URL with Azure AD

We are using OpenID Connect combined with Azure AD. Nearly all works fine but we get this error message in the log file:

WARN o.x.c.o.a.i.OIDCUserManager - Failed to get user avatar from URL https://graph.microsoft.com/v1.0/me/photo/$value: IOException: Server returned HTTP response code: 401 for URL: https://graph.microsoft.com/v1.0/me/photo/$value

This happens to user with and without a microsoft user avatar.