How to safely exfiltrate secrets from Azure Pipelines

Azure Pipelines does what it can to prevent accidental logging of secrets. As long as you tell it a particular value is a secret (through marking a variable as a secret, or using a logging command), any time the secret may appear in the logs Azure Pipelines will automatically redact it. But when you actually want or need to export a secret from Azure Pipelines in a secure way, this post tells you one way to do it.

Why would you exfiltrate secrets from Azure Pipelines

Exporting secrets is typically a bad thing. Secrets are meant to be hard to share, and a secret that is exposed to Azure Pipelines but not to other users should typically remain a secret known just to your build agent and pipeline. But on very rare occasions I’ve found myself needing to recover or export a secret that Azure Pipelines was entitled to but I wasn’t, although I owned the pipeline.

In such cases, it’s still important to respect the secret and keep it from prying eyes.

Encrypting the secret for export

Merely logging the secret may fail due to the automatic redaction. While you may be able to defeat this through some trivial encoding tricks on the secret (e.g. base64, hex), doing so also exposes your secrets to the bad guys. The high-level strategy will be to simply log the secret in some encrypted form so that though anyone will be able to observe the encrypted secret, only you will be able to decrypt it.

In my technique I’m using .NET encryption APIs that are only implemented when running on Windows, so if your secret is only available on linux/mac agents, you’ll have to switch to a Windows agent first if you’re going to use the particular code in this blog post.

In studying up for how to do this, I started out with the following blog post, which seemed applicable because I had access to the secret in my pipeline from a powershell script:

But this post kept symmetric and asymmetric encryption as separate functions that couldn’t interoperate. Symmetric encryption wasn’t appropriate because that means the pipeline would also have the key to decrypt the secret in order to encrypt it. And if the pipeline YAML contained that key, then anyone else would be able to decrypt the secret too. Asymmetric encryption only worked in a trivial test case because it has a limitation on the length of the secret being encrypted. So I had to combine the two.

I rewrote just about all of the powershell scripts from the above blog post then in order to provide simple cmdlets I could use within the pipeline and from my own machine. The scripts first create a new symmetric key and encrypt the secret with that. The symmetric key is then asymmetrically encrypted, and the encrypted key, the symmetric IV and ciphertext is then logged. On my end, that process is then reversed using my private key in order to recover the original secret.

With the code included at the bottom of the post, I could now take the following steps:

Create a new RSA key and print the public key for checking into the pipeline:

$key = New-AsymmetricKey
[Convert]::tobase64string($key.PublicKey)     

It is critical that I keep that particular Powershell window open until the conclusion of this, because the $key is needed later to decrypt the logged, encrypted secret. I could also have serialized and deserialized the key later if needed, but keeping the window open is easier.

Then I added the EncryptionTools.ps1 file to the repo, imported it into a powershell context, and encrypted the secret. In my case the secret was a text access token. The last step prints out a JSON blob containing the ciphertext and other details I need:

. "$(System.DefaultWorkingDirectory)/azure-pipelines/EncryptionTools.ps1"
$PublicKey = 'UlNBMQAMAAADAAAA...UA7VWDL2d'
$PublicKey = [Convert]::FromBase64String($PublicKey)
$secret = [Text.Encoding]::UTF8.GetBytes($accessToken)
Encrypt-Data -Data $secret -PublicKey $PublicKey | ConvertTo-Json

After running the pipeline, I got JSON like this in my Azure Pipeline console log:

{
  "EncryptedKey": "TjE9Av7Z...mfCz",
  "IV": "2BtjqsymGwFoEuAIyAL3lQ==",
  "Ciphertext": "uPm3lx...Bge5OCHiI="
}

I then imported the logged JSON and decrypted the secret like this:

$packet = '{ "EncryptedKey": ... }' | ConvertFrom-Json
$secret = Decrypt-Data -EncryptedPacket $packet -PrivateKey $key.PrivateKey
$accessToken = [Text.Encoding]::UTF8.GetString($secret)

I now have the access token from Azure Pipelines, in such a way that it would be impossible for anyone except me to decrypt it. In my case, I could now run my local REST API tests far more quickly than I could do by running the pipeline over and over again.

The code that makes the above Powershell script work is below: