Multiple authentication services using IdentityServer4 with .NET Core 2.0

Reading Time: 7 minutes

Implementing authentication server using IdentityServer4 is pretty straightforward even if you have never done it. No big deal, install the required packages, copy-paste-change some code from the docs and you’re done. What if I want it to work in a less standard way? By less standard I mean providing scalable OAuth authentication service in microservice architecture without the use of ANY (conventional) database. Easy? Let’s have a look!

Let me give you some context

Let’s imagine some e-commerce business. Let’s say it’s an online electronic store owned by e-entrepreneur – Jack Shopensky.

Jack is mostly happy about his software, the website has small amount of bugs, is easy to use and (what’s most important) makes money for itself.

The only problem Jack has is that every Black Friday or Cyber Monday, when he sets up a big sale on the latest gaming consoles, the website really slows down.

Sometimes it crashes.

And it’s unacceptable for Jack. The customers are angry, Jack makes less money.

So he asks Ted van Digitalius – his programmer – what should he do? Ted is a very wise and experienced programmer.

What does he tells Jack?

“Move it to the cloud! Scale up! Sky is the limit! JUST DO IT!”. And so he does.


After a very fruitful development process and deployment Jack gets the bug report from another angry customer:

This is very weird. I go to your website, add a new playstation to the cart go to checkout and it says my cart is empty! So I go back and add it again and it says I have two items in my cart, but when I go to checkout again – it’s still empty! Please fix.

What could be wrong here? How can Ted help Jack to profit and his customer to be happy again?

Unauthorized or OK – make a decision!

I mentioned in some posts before that I’m working on a project using Microsoft Service Fabric. Using this tool I can deploy multiple services, on multiple nodes that are basically abstracted away infrastructure (physical machines or VMs). That way one application built with multiple different services can have instances/replicas on many nodes within one cluster. It gives me some amount of scalability and responsivness.

Let’s imagine an application consisting of:

  • A bunch of small, stateless web APIs (could be a back-end for the e-shop from the beginning)
  • Stateless authentication service (used by APIs to authenticate users)
  • Data store for authorization codes, refresh tokens or other grants ( This is used solely by authentication service – I want to use cache-like stateful service for it)

Each node would hold one instance of each API, authentication service and one replica of data storing stateful service.

What is that stateful service?

It’s one of the features of Service Fabric allowing me to hold data in a kind of persistent (or not) state, replicated along all replicas of the service and partitioned if needed.

Service Fabric does some magic to guarantee that my state will not be lost accidentally. <wishful-thinking>And I like to think developers did a decent job on it.</wishful-thinking>

In that stateful service I want to store my grants (like authorization code or refresh token). If somehow Service Fabric fucks up, well I don’t care – worst case scenario user will have to login again.

Back to my problem!

After implementing everything and testing it locally I was happy. Everything worked like a charm.

I loged in with authenticating service, got access token. Used access token to get resources from APIs. Everything was just fine.

Alright, so if everything works as expected why not to deploy it to the actual Azure environment? Let’s see how it plays out. What could go wrong?

Everything deployed, services all green – life is good.

So I open up postman. Using authorization feature I try to log in.

It worked.

I got the access and refresh token. Now I can send the request to the API with the appropriate authorization header.


Um, wut?

Okay, let’s pretend it didn’t happen. Again.


Right on! Don’t know what happened before, probably some error in the Matrix or what.

Okay, just to be sure, one more time.

401. Hm. Again. 401. Again. 401. 200. 401.

Great. What could that be?

What actually happened?

First of all, why it worked locally and not on Azure? Interesting, right?

You see, when I test stateless services locally I set them to single instance, because Service Fabric emulator does not handle well more than one stateless instance. That means even if I emulate 5-node environment I still need to have only one instance of a service. Could that cause some problems?

Well… Wait, what’s the difference? Those are stateless services, so why does it even matter? Why the world hates me now all of the sudden?


 Stuff that went wrong

Remember when I told you locally I have only one instance of the authentication service? Yeah. It looked something like that.

One postman, one auth service and one API. Everything in order. But what happens when real world scenario kicks in?

I know this diagram is as readable as handwriting of your doctor, let me explain what happens here:

  1. I use postman to obtain the authorization code from auth service (load balancer choosed instance #1)
  2. In the meantime API instances pulls IdentityServer configuration – the way of choosing the auth service instance is accidental.
  3. I get the auth code and send another request to obtain tokens. (Load balancer choosed auth service instance #2, but grants are stored in a stateful service common for all the nodes in the cluster)
  4. I use the obtained access token to get data from API (load balancer choosed API instance #1 – NOTE: API instance #1 took configuration from auth service instance #5)
  5. I get error 401 Unauthorized.


First of all, my stateful service storing auth codes and refresh tokens has a few replicas on a few separate nodes. But fortunetally those are REPLICAS, so the state is being replicated among them all. It doesn’t matter which replica auth service would use, the state is the same everywhere.

The obvious point of failure is that I login with #1 instance of auth service and API uses #5 instance. But what exactly is the culprit here?

Signing key.

Yep. My auth service was generating RSA key on the fly to sign the tokens it produces. What if 5 instances of the auth service works in parallel? Each instance simply has a different signing key, so if API does not pulls IdentityServer configuration from the same auth service I used to login, I would get error 401: you shall not pass.

Alright, solution. Pretty simple. I created Azure Key Vault, where I put the self-signed key. On auth service instance start up it gets pulled and used to sign JWT tokens.

Result? Now all works as expected, it doesn’t matter which instance is used by any of the service! Whoo!

Where’s the code, mister?

Alright, for future reference here’s how to do it. First in the code of my IdentityServer. To use the external key that’s not even on the running machine I need to implement ISigningCredentialStore interface. This is the place where I get signing key from Azure Key Vault and returns it in a form acceptable for IdentityServer.

namespace Auth.OAuthServer.Stores	 	 
 public class DeveloperSigningCredentialStore : ISigningCredentialStore	 	 
   public async Task&lt;SigningCredentials&gt; GetSigningCredentialsAsync()	 	 
     var keyVaultClient = new KeyVaultClient(KeyVaultAuthentication.GetToken);	 	 
     KeyBundle singleKey = await keyVaultClient.GetKeyAsync(Config.CertVaultKeyUrl);	 	 
     string kid = singleKey.Key.Kid.Substring(singleKey.Key.Kid.LastIndexOf('/') + 1);	 	 
     string jsonString = JsonConvert.SerializeObject(singleKey.Key);	 	 
     var jwk = new JsonWebKey(jsonString)	 	 
      Kid = kid	 	 
     var parameters = new RSAParameters	 	 
      Exponent = Base64UrlEncoder.DecodeBytes(jwk.E),	 	 
      Modulus = Base64UrlEncoder.DecodeBytes(jwk.N),	 	 
     var securityKey = new RsaSecurityKey(parameters)	 	 
      KeyId = jwk.Kid	 	 
     return new SigningCredentials(securityKey, SecurityAlgorithms.RsaSha256);	 	 
public class KeyVaultAuthentication	 	 
  public static async Task<string> GetToken(string authority, string resource, string scope)	 	 
    var authContext = new AuthenticationContext(authority);	 	 
    ClientCredential clientCred = new ClientCredential(Config.AdAppId, Config.AdAppSecret);	 	 
    AuthenticationResult result = await authContext.AcquireTokenAsync(resource, clientCred);
    if (result == null)	 	 
      throw new InvalidOperationException("Failed to obtain the JWT token");	 	 
    return result.AccessToken;	 	 

Here’s what this code does in sequence:

  1. Creates Azure KeyVault client and authenticates with it.
  2. Gets the signing key.
  3. Kid is the azure key vault url striped down to just plain ID
  4. Creates the RSA security key using the values took from a key stored in Azure Key Vault
  5. Returns ready to use signing credentials

Okay, another thing is I need to use the signing key properly when creating and signing a token. This is not a required step for a default implementations and keys, but here I want to have a full control over token creation. In order to do that I need to inherit the class used by IdentityServer to create Token.

public class DeveloperTokenCreationService : DefaultTokenCreationService	 	 
  private readonly ISigningCredentialStore _publicKeyProvider;
  public DeveloperTokenCreationService(ISystemClock clock, IKeyMaterialService keys, ILogger&lt;DefaultTokenCreationService&gt; logger, ISigningCredentialStore publicKeyProvider) : base(clock, keys, logger)	 	 
    _publicKeyProvider = publicKeyProvider;	 	 
  protected override async Task<string> CreateJwtAsync(JwtSecurityToken jwt)	 	 
    byte[] rawDataBytes = System.Text.Encoding.UTF8.GetBytes($"{jwt.EncodedHeader}.{jwt.EncodedPayload}");	 	 
    var keyVaultClient = new KeyVaultClient(KeyVaultAuthentication.GetToken);	 	 
    var hasher = new SHA256CryptoServiceProvider();	 	 
    byte[] digest = hasher.ComputeHash(rawDataBytes);	 	 
    KeyOperationResult signature = await keyVaultClient.SignAsync(	 	 
    JsonWebKeySignatureAlgorithm.RS256, digest, CancellationToken.None);	
    return $"{jwt.EncodedHeader}.{jwt.EncodedPayload}.{Base64UrlEncoder.Encode(signature.Result)}";	 	 
  protected override async Task<JwtHeade> CreateHeaderAsync(Token token)	 	 
    var credentials = await _publicKeyProvider.GetSigningCredentialsAsync();	 	 
    var header = new JwtHeader(credentials);	
    return header;	 	 

Here’s what it does:

  1. Converts JWT security token header and payload to byte array.
  2. Gets the signing key from Azure Key Vault again (this is a place for refactor to avoid double pulling of the signing key and double Key Vault Client creation – omitting it for simplicity for now)
  3. Computes the SHA256 hash of the token bytes
  4. Signs the token with extracted signing key
  5. Returns signed JWT Token

Alright, that basically it. Now to glue it all together I need some changes in Startup.cs

public void ConfigureServices(IServiceCollection services)	 	 
    services.AddSingleton<ISigningCredentialStore, DeveloperSigningCredentialStore>();	 	 
    services.AddSingleton<ITokenCreationService, DeveloperTokenCreationService>();	 	 


Now I inject the newly created functions and I’m ready to go! One more thing left to do.

APIs using my authentication service need to have the same signing key to be able to validate tokens correctly based on IdentityServer configuration. That’s a simple one. In the client API Startup.cs

  .AddAuthentication(options =&gt;	 	 
    options.DefaultAuthenticateScheme = JwtBearerDefaults.AuthenticationScheme;	 	 
    options.DefaultChallengeScheme = JwtBearerDefaults.AuthenticationScheme;	 	 
    options.DefaultForbidScheme = JwtBearerDefaults.AuthenticationScheme;	 	 
  .AddJwtBearer(JwtBearerDefaults.AuthenticationScheme, "humanaction.auth", options =&gt;	 	 
    options.Authority = Configuration.GetConnectionString(AuthServiceUrlSettingsKey);	 	 
    options.Audience = "specific.api";	 	 
    options.TokenValidationParameters = new TokenValidationParameters	 	 
    IssuerSigningKey = GetSecurityKeyForAuthService().Result	 	 
    options.RequireHttpsMetadata = false;	 	 


As you can see I;m using the Microsoft official extension for authentication instead of the IdentityServer one. That’s because with Microsofts solution gives me more felxibility with seting TokenValidationParameters where IdentityServer extension is pretty fixed. The second one uses the microsoft extension anyway under the hood.

One really important thing there is IssuerSigningKey. Here I set exactly the same signing key the authentication service uses to sign JWT tokens. Getting it is analogical to what happened in authentication service, so I won’t duplicate code here. This function is a good candidate for an internal nuget package.


So I did it, it works now. Yay! It wasn’t really complicated, was it? But I hope it’ll save you time for figuring out the problem if you occur this kind of messed up architectural idea in the future. So what have I achieved this time?

  • I have a multiple instances of authentication service running in the cloud
  • If I want I can scale it to the infinity and nothing bad happens
  • I can scale authentication service independently of it’s grant store, because of the stateful service

Just for the end a few things to remember. If you want to implement multiple IdentityServers as one logical being in some kind of cluster, have a look at this checklist:

  • Use the same signing key for every IdentityServer!
  • Use Microsoft extension AddJwtBearer instead of using the default one for IdentityServer – it gives more options when defining TokenValidationParameters like IssuerSigningCertificate, multiple valid issuers etc.
  • Even if you use multiple services or databases of any kind holding the grants, make sure they all have the same state at all times to avoid weird behavior

Leave a Reply

Your email address will not be published. Required fields are marked *