In the simplest case, a certificate allows you to establish protected connection between client and server. But this is not all it is capable of. For example, I saw an online course on Pluralsight called Microservices Security. And there was one thing mentioned there, which is called Mutual Transport Layer Security. It not only allows client to make sure that it is interacting with the correct server, but also allows the server to authenticate the client.
This is why developers must know how to work with certificates. And it is for this reason that I decided to write this article. I want it to be a place where one can find basic knowledge about certificates. I don't think that experts can find something interesting here, but I hope that it will be useful for beginners and those who want to refresh their knowledge.
This article will contain the following sections:
- What is a certificate and why do we need them?
- How to create a self-signed certificate for testing on your computer?
- How to use certificates with ASP.NET Core on the server side and on the client side?
Why do we need certificates?
All icons were created by Vitaly Gorbachev at Flaticon
Unfortunately, since the channel is public, anyone can read and even change the messages that Alice and Bob send to each other:
This situation is called "Man in the Middle".
How can Alice and Bob protect themselves from this danger? Encryption comes to the rescue. The most ancient and widespread encryption systems are systems with a symmetric key. In this case, Alice and Bob must have exactly the same keys (which is why they are called symmetric), which are not known to anyone else. Then, using any symmetric encryption system, they can exchange messages over a public communication channel without fear that a hacker will be able to read the messages or change them.
But a hacker can still repeat one or more messages that he saw earlier. In some cases, this can pose a serious danger (imagine that a hacker can repeat a request to transfer money from one account to another). But this problem is effectively solved in all modern communication systems. (For example, you can add a sequence number to each message. If the number in the message on the receiving side is not equal to the expected number, such a message is discarded).
What should we do? This is where asymmetric encryption or public key encryption comes to the rescue. Its main idea is as follows. Let's say Alice wants to send a message to Bob. Now Bob generates not one, but two keys - public and private. The public key is not a secret. Bob can give it to anyone who wants to talk to him. But he keeps the private key secret and does not show it to anyone, even Alice. The trick is that if a message is encrypted with a public key, it can only be decrypted using the private key. Conversely, a message encrypted with a private key can only be decrypted using the public key.
Now it is clear how Alice and Bob should act. Each of them generates its own public and private keys. Then they exchange their public keys over the communication channel. Since public keys are not a secret, they can be transmitted over public channels. But Alice and Bob keep their private keys secret. Let's say Bob wants to send his message to Alice. He encrypts it with her public key and sends an encrypted message over the channel. Only the person who has the private key can decrypt this message (this means that only Alice can do this). The hacker can't decrypt it.
It looks like our problem has been solved. But this is not so simple. The hacker who controls the communication channel has something to tell us. The problem is again in the key distribution mechanism, but now these are public keys. Let's see what can happen.
Suppose that Alice has generated a pair of public and private keys. Now she wants to give her public key to Bob. She sends this key over the communication channel. At this point, the hacker intercepts this key and does not allow Bob to get it. Instead, the hacker generates his own pair of public and private keys. He then sends his public key to Bob, saying that it is Alice's public key. The hacker keeps Alice's real public key for himself:
What can we do to avoid such a situation? And here we come close to certificates. Imagine that Alice distributes through a public channel not just her public key, but a key with a label where it is written that the key belongs to Alice. This label also contains the signature of some respected person whom Alice and Bob trust:
You can assume that the certificate is a key with such a label. But how does it work in the digital world?
In the digital world, everything can be represented as a sequence of bits (zeros and ones). The same applies to keys. What should we do to create a digital signature for such a sequence of bits? This signature must have the following properties:
- It should be short. Imagine that you want to create a digital signature for a movie file. Such a file can take up tens of gigabytes on the disk. If our signature is of the same size, it will be difficult to transfer it along with the file.
- It should be impossible (or very difficult in practice) to fake it. Otherwise, the hacker could still force Bob to accept his own key instead of Alice's key.
How do we create such a signature? We can do this as follows. First, we will calculate the so-called hash for our sequence of bits. You send your sequence of bits to the input of some function (it is called a hash function), and this function returns you another sequence of bits, but already very short. This output sequence is called a hash. All modern hash functions have the following properties:
- For an input sequence of any length, they generate a hash of the same length. Usually this length does not exceed several tens of bytes. Remember that our signature must be short. This property of the hash makes it convenient to use in the signature.
- If you only know the hash, you will not be able to get the input sequence for which this hash was created. This means that you cannot recover the input sequence from the hash.
- If you have a hash for some sequence of bits, you cannot specify another sequence of bits with the same hash. Indeed, there are a lot of different files with a length of 1 GB. But for any of them, you can calculate a hash of, say, 32 bytes. There are far fewer different sequences of 32 bytes in length than there are different files of 1 GB in length. This means that there must be two different files with a length of 1 GB with the same hash. And yet, if you know one of these files and its hash, you will not be able to specify another file that gives the same hash.
But enough about hashes. Unfortunately, the hash itself is not suitable for the role of a signature. Yes, it is short. But anyone can calculate it. A hacker can calculate a hash for his public key, nothing prevents him from doing this. How can we make the hash resistant to forgery? And here again, public-key encryption comes to the rescue.
Remember, I said that Alice and Bob should trust the signature on the key label. Let's say Alice and Bob trust the signature of Very Important Person. How can Very Important Person sign a key? To do this, he generates his own pair of public and private keys. He passes his public key to Alice and Bob, and keeps the private key secret. When he needs to sign Alice's public key, he does it as follows. First, he calculates the hash of Alice's key, and then encrypts it with his private key. A hash encrypted with the private key of Very Important Person (it is usually called a certificate authority) is a signature. Since no one knows the private key of Very Important Person, no one can forge his signature.
Now we understand how to create a signature. But we also need to know how we can verify it, how to make sure that the signature was not forged. Let's say Bob has some key. The label says that this is Alice's public key. In addition, there is a signature of Very Important Person. But how to check it? First of all, Bob calculates the hash of the received public key. Remember that everyone can do it. Bob then decrypts the signature using the public key of Very Important Person. As I said before, a signature is just an encrypted hash. After that, Bob compares two hashes: the one that he calculated, and the one that he received from the decrypted signature. If they are equal, then everything is fine, and Bob can be sure that this is Alice's key. But if the hashes are different, then the key cannot be trusted. Since the hacker can't create the correct signature, he can't force Bob to trust the wrong key.
So, a certificate is just a key and a label for it. However, in practice, a lot of additional information is added to the certificate:
- Who owns the key. In our case, this is Alice.
- From what date and until what date the key is valid.
- Who signed the key. In our case, this is Very Important Person. This information is necessary, because in reality, different certificate authorities can sign the key.
- What algorithm is used to calculate the hash and create the signature.
- ... and any additional information.
A hash and signature are created for all this data, so a hacker can't fake any of it.
But there is still a gap in our strict scheme. I hope you have already understood what I mean. How do Alice and Bob get the public key of Very Important person? If a hacker can replace this key with his own key, our entire system will be destroyed.
Well, of course, the public key of Very Important Person is distributed with a certificate, but now signed by Very-Very Important Person. Hmm... But how is the public key of Very-Very Important Person distributed? With a certificate, of course. Well, you know... there are certificates all the way down.
But jokes aside. Indeed, Alice's certificate can be signed with the certificate of Very Important Person. And his certificate can be signed with the certificate of Very-Very Important Person. This is called a chain of trust. But this chain is not endless. It usually ends with a root certificate. This certificate is not signed by anyone, to be more precise, it is signed by itself (self-signed certificate). Usually, root certificates belong to very reliable companies, whose job is to sign other certificates with their root certificates.
Previously, companies took money for signing certificates. But now we have services like Let's Encrypt, which do it for free. I think that many large companies have realized that it is better to provide certificates for free and make the Internet a more secure space than to have a lot of poorly protected sites, each of which can be used as a platform for attacks on these large companies. Something like this happened with antiviruses. Twenty years ago, we had to pay for them. Now a person can easily find a free high-quality antivirus for installation on a personal computer.
But let's go back to our certificates. We still have one last question. Why do we trust root certificates? What prevents a hacker from replacing them? The reason is how they get to Alice and Bob's computers. You see, they are not delivered via the open communication channel, but are delivered together with the operating system. Recently, some browsers have started to be installed with their own set of trusted certificates.
That's all. That's all I wanted to say about certificates. There are many interesting things connected with them, such as mechanisms for deprecation and revocation of certificates, but we will not talk about this here. Let's move on to practical things.
Creation of certificates
Let's get started. Everything we need is already in .NET Core. Let's create a console application and use some useful namespaces:
using System.Security;
using System.Security.Cryptography;
using System.Security.Cryptography.X509Certificates;
Now we need to create a pair of public and private keys. Secure distribution of the public key is the work of the certificate:// Generate private-public key pair
var rsaKey = RSA.Create(2048);
Then we need to create a certificate request:// Describe certificate
string subject = "CN=localhost";
// Create certificate request
var certificateRequest = new CertificateRequest(
subject,
rsaKey,
HashAlgorithmName.SHA256,
RSASignaturePadding.Pkcs1
);
The certificate request contains information about who this certificate was issued for (the subject variable). If we want the certificate to be used by a web server available at www.example.com, then the variable subject should be equal to CN=www.example.com. In our case, we want to test our web server on localhost. This is why the value of the subject variable is equal to CN=localhost.Next, we pass our key pair to the certificate request and specify the algorithms that should be used to calculate the hash and signature.
Now we need to provide some additional information about which certificate we need. Let's indicate that we don't want to sign other certificates with this one:
certificateRequest.CertificateExtensions.Add(
new X509BasicConstraintsExtension(
certificateAuthority: false,
hasPathLengthConstraint: false,
pathLengthConstraint: 0,
critical: true
)
);
Then there is something interesting. You see, a certificate is just an encryption key store. These keys can be used for various purposes. We have already seen that they can be used for digital signature and session key encryption. But there are other uses for it. Now we must specify how our certificate can be used:certificateRequest.CertificateExtensions.Add(
new X509KeyUsageExtension(
keyUsages:
X509KeyUsageFlags.DigitalSignature
| X509KeyUsageFlags.KeyEncipherment,
critical: false
)
);
Next we provide a public key for identification:
certificateRequest.CertificateExtensions.Add(
new X509SubjectKeyIdentifierExtension(
key: certificateRequest.PublicKey,
critical: false
)
);
And here comes a little bit of black magic. As I have already told you, if you want to use the certificate for protection of www.example.com site, it's subject field must contain CN=www.example.com. But it is not enough for Chrome browsers. They require that the Subject Alternative Name field must contain DNS Name=www.example.com. In our case, it must contain DNS Name=localhost. Otherwise Chrome will not trust such a certificate. Unfortunately I have not found a convenient way to set value of Subject Alternative Name field for our certificate. But the following piece of code sets it to DNS Name=localhost:certificateRequest.CertificateExtensions.Add(
new X509Extension(
new AsnEncodedData(
"Subject Alternative Name",
new byte[] { 48, 11, 130, 9, 108, 111, 99, 97, 108, 104, 111, 115, 116 }
),
false
)
);
That's it. Our certificate request is ready. Now we can create the certificate itself:var expireAt = DateTimeOffset.Now.AddYears(5);
var certificate = certificateRequest.CreateSelfSigned(DateTimeOffset.Now, expireAt);
Here we say that the certificate will be valid for five years from the current moment.Now we have a certificate. But it exists only in the computer's memory so far. To be able to install it in our system, we need to write it to a file in the PFX format. But there is one obstacle here. The file we want to get must contain both public and private keys, because the server must perform both encryption and decryption. But for security reasons, our certificate cannot be used to export the private key. We can create a certificate ready for export as follows:
// Export certificate with private key
var exportableCertificate = new X509Certificate2(
certificate.Export(X509ContentType.Cert),
(string)null,
X509KeyStorageFlags.Exportable | X509KeyStorageFlags.PersistKeySet
).CopyWithPrivateKey(rsaKey);
For convenience, we can add a description:exportableCertificate.FriendlyName = "Ivan Yakimov Test-only Certificate For Client Authorization";
Now we can export the certificate to a file. Since this file also contains a private key, it is reasonable to protect it with a password. In this case, even if the file is stolen, the criminal will not be able to use it:// Create password for certificate protection
var passwordForCertificateProtection = new SecureString();
foreach (var @char in "p@ssw0rd")
{
passwordForCertificateProtection.AppendChar(@char);
}
// Export certificate to a file.
File.WriteAllBytes(
"certificateForServerAuthorization.pfx",
exportableCertificate.Export(
X509ContentType.Pfx,
passwordForCertificateProtection
)
);
So, we have a certificate file that can be used to protect the Web server. But you can also create a certificate to authenticate clients of this server. The creation process is almost the same as for the server certificate, but the subject field can contain anything, and we no longer need the Subject Alternative Name field:// Generate private-public key pair
var rsaKey = RSA.Create(2048);
// Describe certificate
string subject = "CN=Ivan Yakimov";
// Create certificate request
var certificateRequest = new CertificateRequest(
subject,
rsaKey,
HashAlgorithmName.SHA256,
RSASignaturePadding.Pkcs1
);
certificateRequest.CertificateExtensions.Add(
new X509BasicConstraintsExtension(
certificateAuthority: false,
hasPathLengthConstraint: false,
pathLengthConstraint: 0,
critical: true
)
);
certificateRequest.CertificateExtensions.Add(
new X509KeyUsageExtension(
keyUsages:
X509KeyUsageFlags.DigitalSignature
| X509KeyUsageFlags.KeyEncipherment,
critical: false
)
);
certificateRequest.CertificateExtensions.Add(
new X509SubjectKeyIdentifierExtension(
key: certificateRequest.PublicKey,
critical: false
)
);
var expireAt = DateTimeOffset.Now.AddYears(5);
var certificate = certificateRequest.CreateSelfSigned(DateTimeOffset.Now, expireAt);
// Export certificate with private key
var exportableCertificate = new X509Certificate2(
certificate.Export(X509ContentType.Cert),
(string)null,
X509KeyStorageFlags.Exportable | X509KeyStorageFlags.PersistKeySet
).CopyWithPrivateKey(rsaKey);
exportableCertificate.FriendlyName = "Ivan Yakimov Test-only Certificate For Client Authorization";
// Create password for certificate protection
var passwordForCertificateProtection = new SecureString();
foreach (var @char in "p@ssw0rd")
{
passwordForCertificateProtection.AppendChar(@char);
}
// Export certificate to a file.
File.WriteAllBytes(
"certificateForClientAuthorization.pfx",
exportableCertificate.Export(
X509ContentType.Pfx,
passwordForCertificateProtection
)
);
Now we can install the certificate we created into the system. To do this in Windows, double-click on the PFX certificate file. The wizard window opens. Specify that you want to install the certificate only for the current user, and not for the entire machine:This is the end of the certificate import configuration. Then you can click only "Next", "Finish" and "Ok".
Now our certificate is present in the Trusted Root Certification Authorities storage. You can open it by clicking the Manage User Certificates link in the Control Panel:
Before proceeding to using these certificates in the .NET code, I want to show you another way to create self-signed certificates. If you don't want to write the certificate creation program, but you have PowerShell, you can create a certificate using it.
Here is the code that generates a certificate to protect the server:
$certificate = New-SelfSignedCertificate `
-Subject localhost `
-DnsName localhost `
-KeyAlgorithm RSA `
-KeyLength 2048 `
-NotBefore (Get-Date) `
-NotAfter (Get-Date).AddYears(5) `
-FriendlyName "Ivan Yakimov Test-only Certificate For Server Authorization" `
-HashAlgorithm SHA256 `
-KeyUsage DigitalSignature, KeyEncipherment, DataEncipherment `
-TextExtension @("2.5.29.37={text}1.3.6.1.5.5.7.3.1")
$pfxPassword = ConvertTo-SecureString `
-String "p@ssw0rd" `
-Force `
-AsPlainText
Export-PfxCertificate `
-Cert $certificate `
-FilePath "certificateForServerAuthorization.pfx" `
-Password $pfxPassword
New-SelfSignedCertificate and Export-PfxCertificate command are from the pki module. I hope that by now you can already understand the meaning of the various parameters here.And here is the code for creating a certificate for client authentication:
$certificate = New-SelfSignedCertificate `
-Type Custom `
-Subject "Ivan Yakimov" `
-TextExtension @("2.5.29.37={text}1.3.6.1.5.5.7.3.2") `
-FriendlyName "Ivan Yakimov Test-only Certificate For Client Authorization" `
-KeyUsage DigitalSignature `
-KeyAlgorithm RSA `
-KeyLength 2048
$pfxPassword = ConvertTo-SecureString `
-String "p@ssw0rd" `
-Force `
-AsPlainText
Export-PfxCertificate `
-Cert $certificate `
-FilePath "certificateForClientAuthorization.pfx" `
-Password $pfxPassword
Now let's see how we can use these certificates.How to use certificates in .NET code
The first option is to get a certificate from a PFX file. You can use this option if you have a certificate file that you have installed in the trusted certificate store. In this case, you can get a certificate as follows:
var certificate = new X509Certificate2(
"certificateForServerAuthorization.pfx",
"p@ssw0rd"
);
Here certificateForServerAuthorization.pfx is the path to the certificate file, and p@ssw0rd is the password that you used to protect it.But you may not always have access to the certificate file. In this case, you can take the certificate directly from the storage:
var store = new X509Store(StoreName.Root, StoreLocation.CurrentUser);
store.Open(OpenFlags.ReadOnly);
var certificate = store.Certificates.OfType<X509Certificate2>()
.First(c => c.FriendlyName == "Ivan Yakimov Test-only Certificate For Server Authorization");
The value StoreLocation.CurrentUser means that we want to work with the certificate store of the current user, and not the entire computer. The value StoreName.Root means, that we must look for the certificate in the Trusted Root Certification Authorities storage. Here, for simplicity, I'm looking for a certificate by name, but you can specify any suitable criterion.Now we have a certificate. Let's make our server to use it. To do this, we need to change the code of the Program.cs file:
public class Program
{
public static void Main(string[] args)
{
CreateHostBuilder(args).Build().Run();
}
public static IHostBuilder CreateHostBuilder(string[] args)
{
var store = new X509Store(StoreName.Root, StoreLocation.CurrentUser);
store.Open(OpenFlags.ReadOnly);
var certificate = store.Certificates.OfType<X509Certificate2>()
.First(c => c.FriendlyName == "Ivan Yakimov Test-only Certificate For Server Authorization");
return Host.CreateDefaultBuilder(args)
.ConfigureWebHostDefaults(webBuilder =>
{
webBuilder
.UseKestrel(options =>
{
options.Listen(System.Net.IPAddress.Loopback, 44321, listenOptions =>
{
var connectionOptions = new HttpsConnectionAdapterOptions();
connectionOptions.ServerCertificate = certificate;
listenOptions.UseHttps(connectionOptions);
});
})
.UseStartup<Startup>();
});
}
}
As you can see, all the magic happens inside the UseKestrel method. Here we specify which port we want to use and which certificate we want to apply.Now the browser considers our site protected:
var client = new HttpClient()
{
BaseAddress = new Uri("https://localhost:44321")
};
var result = await client.GetAsync("data");
var content = await result.Content.ReadAsStringAsync();
Console.WriteLine(content);
In fact, the standard HttpClient verifies the server certificate and will not establish a connection if it cannot verify its authenticity. But what if we want to do some additional checks? For example, you may want to check who signed the server certificate. Or you want to check some non-standard field of this certificate. This can be done. We just need to define the method that will be called after the system performs the standard certificate verification:var handler = new HttpClientHandler()
{
ServerCertificateCustomValidationCallback = (request, certificate, chain, errors) => {
if (errors != SslPolicyErrors.None) return false;
return true;
}
};
var client = new HttpClient(handler)
{
BaseAddress = new Uri("https://localhost:44321")
};
You assign this method to the ServerCertificateCustomValidationCallback property of HttpClientHandler instance. The instance must be passed to the HttpClient's constructor.Let's take a closer look at this verification method. As I said before, it is called after, and not instead of the standard check. The results of this check can be obtained from the last parameter of this method (errors). If this value is not equal to SslPolicyErrors.No, the standard verification failed, and you can't trust such a certificate. This method also allows you to get information about:
- The request (request).
- Server certificate (certificate).
- Chain of trust for this certificate (chain). Here you can find the detailed reason why the standard check failed, if you are interested in this information.
So, now we know how to protect our server with a certificate. But the certificate can also be used to authenticate the client. In this case, the server will only serve requests from those clients that provide the "correct" certificate. A certificate is considered correct if it passes the standard verification, and also meets any additional conditions requested by the server.
Let's see how to make the server require a certificate from the client. To do this, you only need a small code change:
return Host.CreateDefaultBuilder(args)
.UseSerilog()
.ConfigureWebHostDefaults(webBuilder =>
{
webBuilder
.UseKestrel(options =>
{
options.Listen(System.Net.IPAddress.Loopback, 44321, listenOptions =>
{
var connectionOptions = new HttpsConnectionAdapterOptions();
connectionOptions.ServerCertificate = certificate;
connectionOptions.ClientCertificateMode = ClientCertificateMode.RequireCertificate;
connectionOptions.ClientCertificateValidation = (certificate, chain, errors) =>
{
if (errors != SslPolicyErrors.None) return false;
// Here is your code...
return true;
};
listenOptions.UseHttps(connectionOptions);
});
})
.UseStartup<Startup>();
});
As you can see, we have additionally set only two properties of the HttpsConnectionAdapterOptions object. Using the ClientCertificateMode property, we determine that the client certificate is mandatory, and using the ClientCertificateValidation property, we set our custom function for additional certificate verification.If you open such a site in a browser, it will ask you which client certificate you want to use:
var handler = new HttpClientHandler()
{
ServerCertificateCustomValidationCallback = (request, certificate, chain, errors) => {
if (errors != SslPolicyErrors.None) return false;
// Here is your code...
return true;
}
};
handler.ClientCertificates.Add(certificate);
var client = new HttpClient(handler)
{
BaseAddress = new Uri("https://localhost:44321")
};
You just add the certificate into the ClientCertificates collection of the HttpClientHandler object.Conclusion
Appendix
- Develop Locally with HTTPS, Self-Signed Certificates and ASP.NET Core
- X.509 своими силами в .Net Core
- All icons were created by Vitaly Gorbachev at Flaticon
The source code for this article can be found at GitHub.
No comments:
Post a Comment