Skip to main content

Interana Version 2 Guides

You are viewing the documentation for Interana version 2. For documentation on the most recent version of Interana, go to docs.interana.com.

 

Interana Docs

Use Azure KeyVault in the Interana import pipeline

  1. Last updated
  2. Save as PDF

When using Azure KeyVault, the Interana import pipeline connects to and downloads data from Azure Containers in the same way as any Azure pipeline. But in this case, the import pipeline must connect to KeyVault to retrieve the credentials to be able to connect to the Azure Container.

First, Interana connects to Azure AD (Active Directory) to retrieve an access token. Interana uses this access token to authenticate with the KeyVault and retrieve the credentials. Once Interana has the credentials, the import pipeline works the same as a normal Azure pipeline.

Pipeline Parameters

When creating an Azure KeyVault pipeline, you'll still use the data source type as azure_blob and two of the same parameters, storage_account and container.

Common Parameters

The data source type is azure_blob.

Pipeline parameters

These parameters are required, as with any Azure pipeline.

Parameter Required? Description
storage_account Yes The name of the Azure Storage account.
container Yes The name of the container to search and download files from.

Azure KeyVault parameters

We added the following parameters to support Azure KeyVault.

Parameter Required? Description
certificate Yes The name of private key, as stored in the secret table.
client_id Yes The ID of the application connecting to the blob store.
thumbprint Yes The thumbprint of the private key stored in the secret table and used to authenticate with Azure AD.
tenant Yes The instance of the Azure AD endpoint used to authenticate with.
vault Yes The name of KeyVault to retrieve the credentials from.
secret_id Yes The ID of the Azure Blob credentials stored in the KeyVault.
secret_version Yes The version of the secret, which is the "friendly" name of the secret. This is the name from the Get Secret API.
api_version No The parameter for the get secret request; default is 2016-10-01, which is the current version of the API.
resource No The resource we will authenticate to Azure AD to use. We recommend that you set this to https://vault.azure.net.

Certificate parameters

Rather than including the certificate file on every import node, we created a new table, secret, to store the contents of the certificate. The table only has the following columns:

Parameter Required? Description
name Yes The name provided by the user to identify the certificate.
secret Yes The contents of the certificate.

Add a certificate to the table

Use the following command to add a new secret:

./insert_cert_to_db.py --add <certificate name> <path to certificate>

For example:

./insert_cert_to_db.py --add my_secret my_certificate.pem

Remove a certificate from the table

Use the following command to remove a secret:

./insert_cert_to_db.py --remove <certificate name>

For example:

./insert_cert_to_db.py --remove my_secret

Example of setting up an Azure KeyVault pipeline

Here is an example of how to set up an Azure KeyVault pipeline.

First, use the insert_cert_to_db.py script to add the certificate (my_cert.pem) to the secret table:

./insert_cert_to_db.py --add test_cert my_cert.pem

Next, use a config file to create a table and pipeline:

ia table create config-file azure_keyvault_test_table.txt

With the table and pipeline created, we can now create a job:

ia job create azure_test_pipeline onetime 2014-01-01 2014-01-02

Sample Config File

{
    "table": {
        "name": "azure_test",
        "type": "Event",
        "time_column": "time",
        "time_column_format": "milliseconds",
        "shard_keys": ["shard_key"]
    },
 
    "ingest": [
        {
            "name": "azure_test_pipeline",
            "data_source_type": "azure_blob",
            "table_name": "azure_test",
 
            "data_source_parameters": {
                "file_pattern": "{year:04d}/{month:02d}/{day:02d}/",
                "storage_account": "interanadevfiles",
                "container": "integration-test-files",
                "certificate": "test_cert",
                "client_id": "7h35und4y5-1u5h-41vv4y5-5501f411c3",
                "thumbprint": "A4B0E7EDE645A3C7C248FE7C61703EA043D3D0CD",
                "tenant": "https://login.online.com/",
                "vault": "firstvault2",
                "secret_id": "2447209f9a998230343ea48b14f2f34620f8",
                "secret_version": "dev-storagekey"
            }
        }
    ]
}