vcsjones.dev Open in urlscan Pro
2600:1f14:1e1:d100:1330:a688:a8de:268  Public Scan

Form analysis
0 forms found in the DOM

Text Content

Random Thoughts
About Contact Projects RSS
Warranty void.
 * Dec 30, 2021
   
   
   .NET'S CRYPTOGRAPHIC ONE-SHOTS
   
   Over the past few releases in .NET, formerly .NET Core, there has been
   progress on making cryptographic primitives like AES, SHA, etc. better for
   developers to use.
   
   “Better” is an interesting point of conversation with cryptographic API
   design. To a developer, better may mean more throughput, less allocations, or
   a simply less cumbersome API. To a framework or library author, it means
   thinking about how developers will use, or mis-use, an API.
   
   Let’s look at AES encryption as it was in the .NET Framework days:
   
   using System.Security.Cryptography;
   
   byte[] data = default;
   
   using (Aes aes = Aes.Create())
   {
       byte[] key = default; // Get a key from somewhere
       byte[] iv = default; // Get a unique IV from somewhere
   
       using ICryptoTransform transform = aes.CreateEncryptor(key, iv);
       byte[] encrypted = transform.TransformFinalBlock(data, 0, data.Length);
   }
   
   
   This may only be a dozen or so lines of a code, but not all of it is straight
   forward. What is a “transform”? What is a “block”? What does “final” mean?
   
   The APIs expose quite a bit of functionality, and are confusingly named.
   TransformFinalBlock for example, despite having “Block” in it’s name, is
   almost always going to be capable of encrypting more than one block. It also
   means the data doesn’t need to be block aligned, so it handles padding
   appropriately. Since none of that may be understood, developers often times
   work around perceived problems, like handling individual blocks. While this
   API design offers the most flexibility for developers, it also offers the
   most complexity.
   
   This complexity exists for a small group of people. Most developers have some
   small amount of data they want to encrypt, and when they don’t, they have a
   Stream.
   
   Complex APIs that are prone to misuse are problematic in a security context.
   A misused cryptographic API almost always harms intended goal of the
   cryptographic API, whether that be secrecy, integrity, etc.
   
   For .NET, this became a concern when AES GCM and CCM were exposed in .NET
   Core 3.1. The AesGcm and AesCcm classes do not follow the design of AES
   prior. In addition to not inheriting from Aes or SymmetricAlgorithm, they
   also do not expose any block functionality. This was discussed at length, and
   instead these types offer simple Encrypt or Decrypt APIs that takes data and
   return data, or write to an existing buffer.
   
   This, while less flexible, resolves many concerns about misusing the AES GCM
   cipher mode. Primarily among those concerns was releasing unauthenticated
   data. Streaming decryption is, put simply, difficult to do safely.
   
   “Streaming” decryption doesn’t necessarily mean the use of System.IO.Stream.
   Rather, it means processing a block of plaintext or ciphertext a block at a
   time and doing something with it in the middle of encrypting or decrypting.
   This is often perceived as desirable when handling large amounts of data.
   After all, if I have a 12 gigabyte file, I can’t just put that in a byte
   array and encrypt it. Rather, processing it in chunks lets me handle it in
   memory.
   
   In pseudo code, let’s say I wanted to decrypt a file and send it over the
   network:
   
   # NOTE: This is an example of doing things improperly
   encryptedFileStream = getFile()
   stream = getStream()
   
   loop {
       data = encryptedFileStream.read(128 / 8) # One AES block size
   
       if (data.length == 128 / 8) {
           stream.write(decrypt(data))
       }
       else {
           stream.write(decryptFinal(data))
           stream.close()
           break
       }
   }
   
   
   Recall though that AES GCM is authenticated. That is, AES GCM can tell if
   your ciphertext has been modified while in storage or in transit. An
   important detail of this though is that GCM cannot authenticate until it has
   processed the entire ciphertext (when decryptFinal is called.)
   
   This is breaks down because as we are decrypting, we’re sending (releasing)
   the plaintext before AES GCM has been able to authenticate the entire cipher
   text. If the person, tool, whatever on the other end of the network is
   processing that decrypted data in real time, then they have processed
   unauthentic data and it’s too late to go back and tell them “Never mind, that
   data I send you a few seconds ago might have been tampered with.”
   
   There are correct ways to do this, but are also still difficult to do
   correctly. You could break the file up in to small chunks and treat them as
   individual ciphertexts. However then you need to worry about many nonces,
   ensuring chunks are processed in the right order, a chunk isn’t missing, or
   replayed, etc.
   
   Before long you’ve invented a cryptographic protocol. This is largely why
   many folks will recommend using something that is well understood and robust
   rather than trying to build it yourself. Though not a primitive cipher, this
   kind of problem falls in the “roll your own cryptography” bucket.
   
   It’s rather easy to accidentally roll your own cryptography, especially so
   when working with “streaming” data.
   
   In .NET then, AES GCM and CCM do not support encrypting individual blocks.
   This still does not solve the issue of ensuring chunks are handled
   appropriately when handling large amounts of data. For that, higher level
   tools are still recommended. However it removes the temptation for streaming
   AES GCM, for which any attempt to use is almost always incorrect. Since it is
   very difficult to use correctly with no practical use cases, it isn’t
   offered.
   
   
   TOWARD BETTER APIS
   
   Simple APIs are important to making them misuse resistent, and .NET has
   gotten better at that over the past few releases.
   
   Like AesGcm, the SymmetricAlgorithm and its derivatives like Aes, TripleDES,
   etc. all offer similar one-shot APIs starting in .NET 6 in the form of
   EncryptCbc, EncryptEcb or DecryptCbc and DecryptEcb.
   
   using System.Security.Cryptography;
   
   byte[] data = default;
   
   using (Aes aes = Aes.Create())
   {
       byte[] key = default; // Get a key from somewhere
       byte[] iv = default; // Get a unique IV from somewhere
   
       aes.Key = key;
   
       // Encrypt all the data at once
       byte[] encrypted = aes.EncryptCbc(data, iv);
   }
   
   
   There is no ICryptoTransform that needs to be reasoned about or disposed, and
   there is no need to worry about blocks, padding, etc. Where possible, one
   shots have been added in most places, and made static where possible.
   
   For hashing, prior to .NET 5 it would look something like this:
   
   // Prior to .NET 5
   using System.Security.Cryptography;
   
   byte[] data = default; // Some data
   
   using (SHA256 hash = SHA256.Create())
   {
       byte[] digest = hash.ComputeHash(data);
   }
   
   
   Rather than creating an instance of a hash algorithm, HashData now exists:
   
   // Starting in .NET 5
   using System.Security.Cryptography;
   
   byte[] data = default; // Some data
   byte[] digest = SHA256.HashData(data);
   
   
   This is much easier to reason about. The method is static, there is no
   stateful hash object that needs to be instantiated, no need to remember to
   dispose of it, and no need to worry about thread safety. Not only are the one
   shots easier to use, they almost always offer better performance, either in
   throughput or reduced allocations. These one shots are not simple wrappers
   around HashAlgorithm.Create() and then hashing something. They internally do
   not allocate on the managed heap at all. Everyone benefits here: the APIs are
   simpler, and developers get better performance.
   
   For .NET 6, the one shot hashing APIs were brought to the HMAC classes as
   well, offering the same improved APIs and better performance.
   
   Also for .NET 6 PBKDF2 got the same treatment with Rfc2898DeriveBytes.Pbkdf2.
   
   using System.Security.Cryptography;
   
   byte[] salt = RandomNumberGenerator.GetBytes(32);
   byte[] prk = Rfc2898DeriveBytes.Pbkdf2(
       userPassword,
       salt,
       iterations: 200_000,
       HashAlgorithmName.SHA256,
       outputLength: 32);
   
   
   All of these APIs also offer modern amenities, like working
   ReadOnlySpan<byte> for input data and being able to write to a Span<byte> for
   output data.
   
   I’m happy with .NETs move toward easier to use APIs for cryptographic
   primitives. I still largely believe many developers should use higher level
   concepts rather than these basic building blocks. However, for those that
   need the building blocks, they are getting better.

 * Feb 24, 2020
   
   
   DOS AND DON'TS OF STACKALLOC
   
   In .NET Core 2.1 a small but well-received feature was the ability to
   “safely” allocate a segment of data on the stack, using stackalloc, when used
   with Span<T>.
   
   Before Span<T>, stackalloc required being in an unsafe context:
   
   unsafe {
       byte* data = stackalloc byte[256];
   }
   
   
   The use of unsafe, along with the little number of APIs in .NET that could
   work with pointers, was enough to deter a lot of people from using it. As a
   result, it remained a relatively niche feature. The introduction of Span<T>
   now means this can be done without being in an unsafe context now:
   
   Span<byte> data = stackalloc byte[256];
   
   
   The .NET team has also been working diligently to add Span<T> APIs where it
   makes sense. There is now more appeal and possibility to use stackalloc.
   
   stackalloc is desirable in some performance sensitive areas. It can be used
   in places where small arrays were used, with the advantage that it does not
   allocate on the heap - and thus does not apply pressure to the garbage
   collector. stackalloc is not a general purpose drop-in for arrays. They are
   limited in a number of ways that other posts explain well enough, and require
   the use of Span<T>.
   
   Recently I vented a bit about stackalloc on Twitter, as one does on Twitter,
   specifically the community’s fast embrace of it, without discussing or
   well-documenting some of stackalloc’s sharp edges. I’m going to expand on
   that here, and make an argument for stackalloc still being unsafe and
   requiring some thought about being used.
   
   
   DON’T: USE VARIABLE ALLOCATION LENGTHS
   
   A large risk with using stackalloc is running out of stack space. If you’ve
   ever written a method that is recursive and went too deep, you’ll eventually
   receive a StackOverflowException. The StackOverflowException is a bit special
   in that it is one of the exceptions that cannot be caught. When a stack
   overflow occurs, the process immediately exits. Allocating too much with
   stackalloc has the same effect - it causes a stack overflow and the process
   immediately terminates.
   
   This is particularly worrisome when the allocation’s length is determined by
   user input:
   
   Span<char> buffer = stackalloc char[userInput.Length]; //DON'T
   
   
   This allows users to take down your process, an effective denial-of-service.
   
   Using a constant also reduces the risk of arithmetic or overflow mistakes.
   Currently, .NET Core’s CLR interprets that amount to be allocated as an
   unsigned integer. This means that an arithmetic over or under flow may result
   in a stack overflow.
   
   Span<byte> b = stackalloc byte[(userInput % 64) - 1]; //DON'T
   
   
   
   DO: USE A CONSTANT FOR ALLOCATION SIZE
   
   Instead, it’s better to use a constant value for stackalloc, always. It
   immediately resolves any ambiguities about how much is allocated on the
   stack.
   
   Span<char> buffer = stackalloc char[256]; //better
   
   
   Once you have an allocated buffer, you can use Span’s Slice funtionality to
   adjust it to the correct size:
   
   Span<char> buffer = stackalloc char[256];
   Span<char> input = buffer.Slice(0, userInput.Length);
   
   
   
   DON’T: USE STACKALLOC IN NON-CONSTANT LOOPS
   
   Even if you allocate a fixed length amount of data on the stack, doing so in
   a loop can be dangerous as well, especially if the number of the iterations
   the loop makes is driven by user input:
   
   for (int i = 0; i < userInput; i++) { // DON'T
       Span<char> buffer = stackalloc char[256];
   }
   
   
   This also can cause a denial of service, since this allows someone to control
   the number of stack allocations, though not the length of the allocation.
   
   
   DO: ALLOCATE OUTSIDE OF LOOPS
   
   Span<char> buffer = stackalloc char[256]; //better
   for (int i = 0; i < userInput; i++) {
       //Do something with buffer
   }
   
   
   Allocating outside of the loop is the best solution. This is not only safer,
   but also better for performance.
   
   
   DON’T: ALLOCATE A LOT ON THE STACK
   
   It’s tempting to allocate as much as nearly possible on the stack:
   
   Span<byte> data = stackalloc byte[8000 * 1024]; // DON'T
   
   
   You may find that this runs fine on Linux, but fails on Windows with a stack
   overflow. Different operating systems, architectures, and environments, have
   different stacks limits. Linux typically allows for a larger stack than
   Windows by default, and other hosting scenarios such as in an IIS worker
   process come with even lower limits. An embedded environment may have a stack
   of only a few kilobytes.
   
   
   DO: CONSERVATIVELY USE THE STACK
   
   The stack should be used for small allocations only. How much depends on the
   size of each element being allocated. It’s also desirable to not allocate
   many large structs, either.
   
   I won’t prescribe anything specific, but anything larger than a kilobyte is a
   point of concern. You can allocate on the heap depending on how much you
   need. A typical pattern might be:
   
   const int MaxStackSize = 256;
   Span<byte> buffer =
       userInput > MaxStackSize
         ? new byte[userInput]
         : stackalloc byte[MaxStackSize];
   
   Span<byte> data = buffer.Slice(0, userInput);
   
   
   This will allocate on the stack for small amounts, still in a constant
   amount, or if too large, will use a heap-allocated array. This pattern may
   also make it easier to use ArrayPool, if you choose, which also does not
   guarantee that the returned array is exactly the requested size:
   
   const int MaxStackSize = 256;
   byte[]? rentedFromPool = null;
   Span<byte> buffer =
       userInput > MaxStackSize
       ? (rentedFromPool = ArrayPool<byte>.Shared.Rent(userInput))
       : stackalloc byte[MaxStackSize];
   
   // Use data
   Span<byte> data = buffer.Slice(0, userInput);
   
   // Return from pool, if we rented
   if (rentedFromPool is object) {
       // DO: if using ArrayPool, think carefully about clearing
       // or not clearing the array.
       ArrayPool<byte>.Shared.Return(rentedFromPool, clearArray: true);
   }
   
   
   
   DON’T: ASSUME STACK ALLOCATIONS ARE ZERO INITIALIZED
   
   Most normal uses of stackalloc result in zero-initialized data. This behavior
   is however not guaranteed, and can change depending if the application is
   built for Debug or Release, and other contents of the method. Therefore,
   don’t assume that any of the elements in a stackalloced Span<T> are
   initialized to something by default. For example:
   
   Span<byte> buffer = stackalloc byte[sizeof(int)];
   byte lo = 1;
   byte hi = 1;
   buffer[0] = lo;
   buffer[1] = hi;
   // DONT: depend on elements at 2 and 3 being zero-initialized
   int result = BinaryPrimitives.ReadInt32LittleEndian(buffer);
   
   
   In this case, we might expect the result to be 257, every time. However if
   the stackalloc does not zero initialize the buffer, then the contents of the
   upper-half of the integer will not be as expected.
   
   This behavior will not always be observed. In Debug builds, it’s likely that
   you will see that stackalloc zero-initializes its contents every time,
   whereas in Release builds, you may find that the contents of a stackalloc are
   uninitialized.
   
   Starting in .NET 5, developers can opt to explicitly skip zero-initializing
   stackalloc contents with the SkipLocalsInit attribute. Without it, whether or
   not stackalloc is default initialized is up to Roslyn.
   
   
   DO: INITIALIZE IF REQUIRED
   
   Any item read from a stackalloced buffer should be explicitly assigned, or
   use Clear to explicitly clear the entire Span<T> and initialize it to
   defaults.
   
   Span<byte> buffer = stackalloc byte[sizeof(int)];
   buffer.Clear(); //explicit zero initialize
   byte lo = 1;
   byte hi = 1;
   buffer[0] = lo;
   buffer[1] = hi;
   int result = BinaryPrimitives.ReadInt32LittleEndian(buffer);
   
   
   Though not explicitly covered in this post, the same advice applies to arrays
   rented from the ArrayPool.
   
   
   SUMMARY
   
   In summary, stackalloc needs to be used with care. Failing to do so can
   result in process termination, which is a denial-of-service: your program or
   web server aren’t running any more.

 * Oct 7, 2019
   
   
   IMPORT AND EXPORT RSA KEY FORMATS IN .NET CORE 3
   
   .NET Core 3.0 introduced over a dozen new APIs for importing and exporting
   RSA keys in different formats. Many of them are a variant of another with a
   slightly different API, but they are extremely useful for working with
   private and public keys from other systems that work with encoding keys.
   
   RSA keys can be encoded in a variety of different ways, depending on if the
   key is public or private or protected with a password. Different programs
   will import or export RSA keys in a different format, etc.
   
   Often times RSA keys can be described as “PEM” encoded, but that is already
   ambiguous as to how the key is actually encoded. PEM takes the form of:
   
   -----BEGIN LABEL-----
   content
   -----END LABEL-----
   
   
   The content between the labels is base64 encoded. The one that is probably
   the most often seen is BEGIN RSA PRIVATE KEY, which is frequently used in web
   servers like nginx, apache, etc:
   
   -----BEGIN RSA PRIVATE KEY-----
   MII...
   -----END RSA PRIVATE KEY-----
   
   
   The base64-encoded text is an RSAPrivateKey from the PKCS#1 spec, which is
   just an ASN.1 SEQUENCE of integers that make up the RSA key. The
   corresponding .NET Core 3 API for this is ImportRSAPrivateKey, or one of its
   overloads. If your key is “PEM” encoded, you need to find the base64 text
   between the label BEGIN and END headers, base64 decode it, and pass to
   ImportRSAPrivateKey. There is currently an API proposal to make reading PEM
   files easier. If your private key is DER encoded, then that just means you
   can read the content directly as bytes in to ImportRSAPrivateKey.
   
   Here is an example:
   
   var privateKey = "MII..."; //Get just the base64 content.
   var privateKeyBytes = Convert.FromBase64String(privateKey);
   using var rsa = RSA.Create();
   rsa.ImportRSAPrivateKey(privateKeyBytes, out _);
   
   
   When using openssl, the openssl rsa commands typically output RSAPrivateKey
   PKCS#1 private keys, for example openssl genrsa.
   
   A different format for a private key is PKCS#8. Unlike the RSAPrivateKey from
   PKCS#1, a PKCS#8 encoded key can represent other kinds of keys than RSA. As
   such, the PEM label for a PKCS#8 key is “BEGIN PRIVATE KEY” (note the lack of
   “RSA” there). The key itself contains an AlgorithmIdentifer of what kind of
   key it is.
   
   PKCS#8 keys can also be encrypted protected, too. In that case, the PEM label
   will be “BEGIN ENCRYPTED PRIVATE KEY”.
   
   .NET Core 3 has APIs for both of these. Unencrypted PKCS#8 keys can be
   imported with ImportPkcs8PrivateKey, and encrypted PKCS#8 keys can be
   imported with ImportEncryptedPkcs8PrivateKey. Their usage is similar to
   ImportRSAPrivateKey.
   
   Public keys have similar behavior. A PEM encoded key that has the label
   “BEGIN RSA PUBLIC KEY” should use ImportRSAPublicKey. Also like private keys,
   the public key has a format that self-describes the algorithm of the key
   called a Subject Public Key Info (SPKI) which is used heavily in X509 and
   many other standards. The PEM header for this is “BEGIN PUBLIC KEY”, and
   ImportSubjectPublicKeyInfo is the correct way to import these.
   
   All of these APIs have export versions of themselves as well, so if you are
   trying to export a key from .NET Core 3 to a particular format, you’ll need
   to use the correct export API.
   
   To summarize each PEM label and API pairing:
   
   1. “BEGIN RSA PRIVATE KEY” => RSA.ImportRSAPrivateKey
   2. “BEGIN PRIVATE KEY” => RSA.ImportPkcs8PrivateKey
   3. “BEGIN ENCRYPTED PRIVATE KEY” => RSA.ImportEncryptedPkcs8PrivateKey
   4. “BEGIN RSA PUBLIC KEY” => RSA.ImportRSAPublicKey
   5. “BEGIN PUBLIC KEY” => RSA.ImportSubjectPublicKeyInfo
   
   One gotcha with openssl is to pay attention to the output of the key format.
   A common enough task from openssl is “Given this PEM-encoded RSA private key,
   give me a PEM encoded public-key” and is often enough done like this:
   
   openssl rsa -in key.pem -pubout
   
   
   Even if key.pem is a PKCS#1 RSAPrivateKey (“BEGIN RSA PRIVATE KEY”), the
   -pubout option will output a SPKI (“BEGIN PUBLIC KEY”), not an RSAPublicKey
   (“BEGIN RSA PUBLIC KEY”). For that, you would need to use -RSAPublicKey_out
   instead of -pubout. The openssl pkey commands will also typically give you
   PKCS#8 or SPKI formatted keys.

 * Jul 18, 2019
   
   
   SOMETIMES VALID RSA SIGNATURES IN .NET
   
   One of the nice things about .NET Core being open source is following along
   with some of the issues that people report. I tend to keep an eye on
   System.Security tagged issues, since those tend to be at the intersection of
   things that interest me and things I can maybe help with.
   
   A user filed an issue where .NET Framework considered a CMS valid, and .NET
   Core did not. This didn’t entirely surprise me. In the .NET Framework, the
   SignedCms class is heavily backed by Windows’ handling of CMS/PKCS#7. In .NET
   Core, the implementation is managed (sans the cryptography). The managed
   implementation adheres somewhat strictly to the CMS specification. As other
   issues have noticed, Windows’, thus .NET Framework’s, implementation was a
   little more relaxed in some ways.
   
   This turned out not to be one of those cases. The CMS part was actually
   working just fine. What was failing was RSA itself. The core of the issue was
   that different implementations of RSA disagreed on the RSA signature’s
   validity.
   
   That seems pretty strange!
   
   When I talk about different implementations on Windows, I am usually
   referring to CAPI vs CNG, or RSACryptoServiceProvider and RSACng,
   respectively. For now, I’m keeping this post to the .NET Framework. We’ll
   bring .NET Core in to the discussion later.
   
   There are two implementations because, well, Windows has two of them. CNG, or
   “Cryptography API: Next Generation” is the newer of the two and is intended
   to be future of cryptographic primitives on Windows. It shipped in Windows
   Vista, and offers functionality that CAPI cannot do. An example of that is
   PSS RSA signatures.
   
   .NET Framework exposes these implementations as RSACryptoServiceProvider and
   RSACng. They should be interchangable, and CNG implementations should be used
   going forward. However, there is one corner case where the old, CAPI
   implementation considers a signature valid while the CNG one does not.
   
   The issue can be demonstrated like so:
   
   byte[] n = new byte[] { ... };
   byte[] e = new byte[] { ... };
   byte[] signature = new byte[] { ... };
   var digest = new byte[] {
       0x68, 0xB4, 0xF9, 0x26, 0x34, 0x31, 0x25, 0xDD,
       0x26, 0x50, 0x13, 0x68, 0xC1, 0x99, 0x26, 0x71,
       0x19, 0xA2, 0xDE, 0x81, 
   };
   using (var rsa = new RSACng())
   {
       rsa.ImportParameters(new RSAParameters {
           Modulus = n,
           Exponent = e
       });
       var valid = rsa.VerifyHash(digest, signature, HashAlgorithmName.SHA1,
                                  RSASignaturePadding.Pkcs1);
       Console.WriteLine(valid);
   }
   using (var rsa = new RSACryptoServiceProvider())
   {
       rsa.ImportParameters(new RSAParameters {
           Modulus = n,
           Exponent = e
       });
       var valid = rsa.VerifyHash(digest, signature, HashAlgorithmName.SHA1,
                                  RSASignaturePadding.Pkcs1);
       Console.WriteLine(valid);
   }
   
   
   Note: to avoid bloating this blog post with large signatures and RSA keys, I
   omitted them. However the full example with public keys is available on
   GitHub here.
   
   When used with one of the curious signatures that exhibits this behavior,
   such as the one in the GitHub link, the first result will be false, and the
   second will be true.
   
   Nothing jumped out at me as being problematic. The signature padding is PKCS,
   the public exponent is the very typical 67,537, and the RSA key is sensible
   in size.
   
   To make it stranger, this signature came off the timestamp of Firefox’s own
   signed installer. So why are the results different?
   
   Jeremy Barton from Microsoft on .NET Core made the observation that the
   padding in the RSA signature itself is incorrect, but in a way that CAPI
   tollerates and CNG does not, at least by default. Let’s look at the raw
   signature. To do that, we need the public key and signature on disk, and we
   can poke at them with OpenSSL.
   
   Using the command:
   
   openssl rsautl -verify -in sig.bin -inkey key.der \
       -pubin -hexdump -raw -keyform der
   
   
   We get the following output:
   
   0000 - 00 01 ff ff ff ff ff ff-ff ff ff ff ff ff ff ff
   0010 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff
   0020 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff
   0030 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff
   0040 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff
   0050 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff
   0060 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff
   0070 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff
   0080 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff
   0090 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff
   00a0 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff
   00b0 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff
   00c0 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff
   00d0 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff
   00e0 - ff ff ff ff ff ff ff ff-ff ff ff 00 68 b4 f9 26
   00f0 - 34 31 25 dd 26 50 13 68-c1 99 26 71 19 a2 de 81
   
   
   This is a PKCS#1 v1.5 padded signature, as indicated by by starting with 00
   01. The digest at the end can be seen, 68 b4 f9 26 ... 19 a2 de 81 which
   matches the digest above, so we know that the signature is for the right
   digest.
   
   What is not correct in this signature is how the digest is encoded. The
   signature contains the bare digest. It should be encoded as an ASN.1 sequence
   along with the AlgorithmIdentifer of the digest:
   
   DigestInfo ::= SEQUENCE {
   	digestAlgorithm AlgorithmIdentifier,
   	digest OCTET STRING
   }
   
   
   This goes back all the way to a document (warning: link is to an ftp:// site)
   written in 1993 by RSA labratories explaining how PKCS#1 v1.5 works,and was
   standardized in to an RFC in 1998.
   
   The RSA signature we have only contains the raw digest. It is not part of a
   DigestInfo. If the digest were properly encoded, it would look something like
   this:
   
   0000 - 00 01 ff ff ff ff ff ff-ff ff ff ff ff ff ff ff
   0010 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff
   0020 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff
   0030 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff
   0040 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff
   0050 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff
   0060 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff
   0070 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff
   0080 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff
   0090 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff
   00a0 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff
   00b0 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff
   00c0 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff
   00d0 - ff ff ff ff ff ff ff ff-ff ff ff ff 00 30 21 30
   00e0 - 09 06 05 2b 0e 03 02 1a-05 00 04 14 68 b4 f9 26
   00f0 - 34 31 25 dd 26 50 13 68-c1 99 26 71 19 a2 de 81
   
   
   The signature now includes DigestInfo along with the OID 1.3.14.3.2.26 to
   indicate that the digest is SHA1.
   
   At this point we know what the difference is, and the original specification
   in part 10.1.2 makes it fairly clear that the “data” should be a digest and
   should be encoded as DigestInfo, not a bare digest.
   
   The source of this signature is from Verisign's timestamp authority at
   http://timestamp.verisign.com/ scripts/ timstamp.dll. After checking with
   someone at DigiCert (now running this TSA), it was launched in May 1995.
   
   I suspect that the TSA is old enough that the implementation was made before
   the specification was complete or simply got the specification wrong and no
   one noticed. Bringing this back to CNG and CAPI, CNG can validate this
   signatures, but you must explicitly tell CNG that the signature does not have
   an object identifier. BCRYPT_PKCS1_PADDING_INFO’s documentation has the
   detail there, but gist of it is
   
   > If there is no OID in the signature, then verification fails unless this
   > member is NULL.
   
   This would be used with {B,N}CryptVerifySignature. To bring this back around
   to the .NET Framework, how do we use RSACng and give null in for the padding
   algorithm? The short answer is: you cannot. If you try, you will get an
   explicit ArgumentException saying that the hash algorithm name cannot be
   null.
   
   For .NET Framework, this solution “keep using RSACryptoServiceProvider”. If
   you need to validate these signatures, chances are you do not need to use
   CNG’s newer capabilities like PSS since these malformed signatures appear to
   be coming from old systems. Higher level things like SignedCms and SignedXml
   use RSACryptoServiceProvider by default, so they will continue to work.
   
   To bring in .NET Core, the situation is a little more difficult. If you are
   using SignedCms like so:
   
   var signedCms = new SignedCms();
   signedCms.Decode(File.ReadAllBytes("cms-with-sig.bin"));
   signedCms.CheckSignature(true);
   
   
   This will start throwing when you migrate to .NET Core. .NET Core will use
   CNG when run on Windows to validate RSA signatures for SignedCms and
   SignedXml. This is currently not configurable, either. When used with
   SignedCms, it ultimately calls the X509Certificate2.GetRSAPublicKey()
   extension method, and that will always return an implementation based on CNG.
   
   If you are using SignedCms on .NET Core and need to validate a CMS that is
   signed with these problematic signatures, you are currently out of luck using
   in-the-box components. As far as other platforms go, both macOS and Linux
   environments for .NET Core will agree with CNG - that the signature is
   invalid.
   
   The good news is, these signatures are not easy to come by. So far, only the
   old Verisign timestamp authority is known to have produced signatures like
   this.

 * Feb 1, 2019
   
   
   C# READONLYSPAN AND STATIC DATA
   
   Since C# 7 there have been a lot of point releases that contain all kinds of
   goodies. Many of them are performance focused, such as safe stack allocations
   using Span<T>, or interoperability with improvements to fixed.
   
   One that I love, but is not documented well, is some special treatment that
   ReadOnlySpan<byte> gets when its contents are known at compile time.
   
   Here’s an example of a lookup table I used to aide with hex encoding that
   uses a byte[]:
   
   private static byte[] LookupTable => new byte[] {
       (byte)'0', (byte)'1', (byte)'2', (byte)'3', (byte)'4',
       (byte)'5', (byte)'6', (byte)'7', (byte)'8', (byte)'9',
       (byte)'A', (byte)'B', (byte)'C', (byte)'D', (byte)'E',
       (byte)'F',
   };
   
   
   This binary data has to get stored somewhere in our produced library. If we
   use dumpbin we can see it in the .text section of the binary.
   
   dumpbin /RAWDATA /SECTION:.text mylib.dll
   
   
   Right at the bottom, we see:
   
   00402A40: 30 31 32 33 34 35 36 37 38 39 41 42 43 44 45 46  0123456789ABCDEF
   
   
   I won’t go into the a lot of the details on how this data is compiled into
   the .text section, but at this point we need to get that data into the array
   somehow.
   
   If we look at the jit assembly of LookupTable, we see:
   
   sub rsp, 0x28
   vzeroupper
   mov rcx, 0x7ffc4638746a
   mov edx, 0x10
   call 0x7ffc49b52630
   mov rdx, 0x1b51450099c
   lea rcx, [rax+0x10]
   vmovdqu xmm0, [rdx]
   vmovdqu [rcx], xmm0
   add rsp, 0x28
   ret
   
   
   Where 0x7ffc49b52630 is InitializeArray.
   
   With an array, our property leans on InitializeArray, the source of which is
   in the CoreCLR. For little-endian platforms, it boils down to a memcpy from a
   runtime field handle.
   
   Indeed, with a debugger we finally see:
   
   00007ffd`b18b701a e831a40e00       call    coreclr!memcpy (00007ffd`b19a1450)
   
   
   Dumping @rdx L10 yields:
   
   000001f0`4c552a90  30 31 32 33 34 35 36 37-38 39 41 42 43 44 45 46  0123456789ABCDEF
   
   
   So that was a very long-winded way of saying that when using arrays,
   initializing a field or variable with bytes results in memcpy from the image
   into the array, which results in more data on the heap.
   
   Now, starting in 7.3, we can avoid that memcpy when using ReadOnlySpan<byte>.
   
   private static ReadOnlySpan<byte> LookupTable => new byte[] {
       (byte)'0', (byte)'1', (byte)'2', (byte)'3', (byte)'4',
       (byte)'5', (byte)'6', (byte)'7', (byte)'8', (byte)'9',
       (byte)'A', (byte)'B', (byte)'C', (byte)'D', (byte)'E',
       (byte)'F',
   };
   
   
   Looking at the jit assembly:
   
   mov eax, 0x10
   xor edx, edx
   mov r8, 0x1b5144c0968
   mov [rcx], rdx
   mov [rcx+0x8], r8
   mov [rcx+0x10], eax
   mov rax, rcx
   ret
   
   
   We see that there is mov r8, 0x1b5144c0968. The contents of 0x1b5144c0968
   are:
   
   000001b5`144c0968  30 31 32 33 34 35 36 37-38 39 41 42 43 44 45 46  0123456789ABCDEF
   
   
   So we see that the method is now returning the data directly and omitting the
   memcpy entirely, so our ReadOnlySpan<byte> is pointing directly to the .text
   section.
   
   This works for property getters as shown above, but also as the return of a
   method:
   
   ReadOnlySpan<byte> GetBytes() {
       return new byte[] { ... };
   }
   
   
   Which works similar to the getter of the property. In addition, this also
   works for locals in a method body as well:
   
   void Write200Ok(Stream s) {
       ReadOnlySpan<byte> data = new byte[] {
           (byte)'H', (byte)'T', (byte)'T', (byte)'P',
           (byte)'/', (byte)'1', (byte)'.', (byte)'1',
           (byte)' ', (byte)'2', (byte)'0', (byte)'0',
           (byte)' ', (byte)'O', (byte)'K'
       };
       s.Write(data);
   }
   
   
   Which also produces a reasonable JIT disassembly:
   
   sub     rsp, 0x38
   xor     eax, eax
   mov     qword ptr [rsp+0x28], rax
   mov     qword ptr [rsp+0x30], rax
   mov     rcx, 0x1e595b42ade
   mov     eax, 0x0F
   lea     r8, [rsp+0x28]
   mov     qword ptr [r8], rcx
   mov     dword ptr [r8+8], eax
   mov     rcx, rdx
   lea     rdx, [rsp+0x28]
   cmp     dword ptr [rcx], ecx
   call    0x7ff89ede10c8 (Stream.Write(System.ReadOnlySpan`1<Byte>), mdToken: 0000000006000001)
   add     rsp, 0x38
   ret
   
   
   Here we see mov rcx, 0x1e595b42ade which moves the address of the static data
   directly in to the register with no additional work to create a byte array.
   
   These optimizations currently only works with ReadOnlySpan<byte> right now.
   Other types will continue to use InitializeArray due to needing to handle
   different platforms and how they handle endianness.

Previous Page: 1 of 18 Next