You will connect Elementary Cloud to Bigquery for syncing the Elementary schema (created by the Elementary dbt package).

​
Create service account:

  1. In the Cloud Console, go to: IAM & Admin > Service Accounts

  2. Click on ‘CREATE SERVICE ACCOUNT’

Create service account
  1. Fill in the service account name (‘elementary’) and account description (‘Elementary Data’) and click ‘CREATE AND CONTINUE’:
Create and Continue

  1. Now we need to configure the relevant permissions for this new service account. Select the following role: BigQuery Job User (you will need to grant read access to specific datasets later).

  2. The last step is optional, skip it and press done.

  1. Press on the dots icon to the right of your screen for your new service account and select ‘Manage keys’:
Manage keys
  1. Press on ‘ADD KEY’ and select ‘Create new key’:
Add Key
  1. Use the ‘JSON’ option radio button and press ‘CREATE’:
JSON
  1. This will automatically generate and download a JSON file with your private key information for this service account. This JSON file provides the credentials to programmatically connect and work with your BigQuery environment.

​
Grant service user access to specific datasets

In order for a service user to work with Elementary cloud, it requires the following permissions:

  • Role “BigQuery Data Viewer” on your Elementary dataset.
  • Role “BigQuery Metadata Viewer” on your dbt datasets.

To grant a role on a specific dataset, follow these steps:

  1. Go to your project in BigQuery console

  2. In the “Explorer” tab, find your desired dataset.

  3. Click on the three dots icon next to the dataset name, then Share.

Dataset share
  1. Click the “ADD PRINCIPAL” button on the top right corner.
Add principal
  1. Fill out the form:
    • In the “New principals” textbox, write the email address of your user.
    • In the “Select a role” dropdown menu, choose the desired role (BigQuery Data Viewer for your Elementary dataset, BigQuery Metadata Viewer for your dbt dataset).
    • Click “Save”.
Grant access

Make sure to grant the correct access to your Elementary dataset and your dbt dataset.

​
Permissions and security

Elementary cloud doesn’t need permissions to your tables and schemas. It is recommended to create a read only user for the elementary schema only. For more details, refer to security and privacy.

​
Fill the connection form

Provide the following fields:

  • Service account file: The service account file you generated for Elementary. For more information, refer to Create BigQuery service account.
  • Project: The name of your BigQuery project.
  • Elementary dataset: The name of your Elementary dataset. Usually [dataset name]_elementary.
  • Location: Use this field to configure the location of BigQuery datasets as per the BigQuery documentation.

​
Uploading columns data

In order to let Elementary Cloud access metadata about the columns in your data warehouse, you need to run the following command:

dbt run-operation elementary.upload_dbt_columns

You may choose to run this command in one of the following ways:

  1. Manually whenever there is a schema change in your dataset - Not recommended.
  2. As part of your dbt pipeline - Recommended if you have one main pipeline.
  3. As an independent pipeline - Recommended if you have a lot of small pipelines running in different times.

​
Add the Elementary IP to allowlist

Elementary IP for allowlist: 3.126.156.226

​
Need help with onboarding?

We can provide support on Slack or hop on an onboarding call.