|
1 | 1 | ---
|
2 |
| -title: BigQuery Setup |
| 2 | +title: BigQuery Data Graph Setup |
3 | 3 | beta: true
|
4 | 4 | plan: unify
|
5 |
| -hidden: true |
6 | 5 | redirect_from:
|
7 | 6 | - '/unify/linked-profiles/setup-guides/BigQuery-setup'
|
8 | 7 | ---
|
9 | 8 |
|
10 | 9 | > info ""
|
11 |
| -> At this time, you can only use BigQuery with Linked Events. |
| 10 | +> BigQuery for Data Graph is in beta and Segment is actively working on this feature. Some functionality may change before it becomes generally available. This feature is governed by Segment’s [First Access and Beta Preview Terms](https://www.twilio.com/en-us/legal/tos){:target="_blank"}. |
12 | 11 |
|
13 |
| -On this page, you'll learn how to connect your BigQuery data warehouse to Segment. |
| 12 | +Set up your BigQuery data warehouse to Segment for the [Data Graph](/docs/unify/data-graph/data-graph/). |
14 | 13 |
|
15 | 14 |
|
16 |
| -## Set up BigQuery |
17 |
| - |
| 15 | +## Step 1: Roles and permissions |
18 | 16 | > warning ""
|
19 |
| -> You need to be an account admin to set up the Segment BigQuery connector as well as write permissions for the `__segment_reverse_etl` dataset. |
20 |
| -
|
21 |
| -To set up the Segment BigQuery connector: |
| 17 | +> You need to be an account admin to set up the Segment BigQuery connector as well as write permissions for the `__segment_reverse_etl` dataset. |
22 | 18 |
|
23 |
| -1. Navigate to **IAM & Admin > Service Accounts** in BigQuery. |
| 19 | +To set the roles and permissions: |
| 20 | +1. Navigate to **IAM & Admin > Service Accounts** in BigQuery. |
24 | 21 | 2. Click **+ Create Service Account** to create a new service account.
|
25 |
| -3. Enter your **Service account name** and a description of what the account will do. |
| 22 | +3. Enter your Service account name and a description of what the account will do. |
26 | 23 | 4. Click **Create and Continue**.
|
27 |
| -5. In the **Grant this service account access to project** section, select the [*BigQuery User*](https://cloud.google.com/bigquery/docs/access-control#bigquery.user){:target="_blank"} role to add. |
28 |
| -6. Click **+ Add another role** and add the *BigQuery Job User* role. |
29 |
| -7. Click **+ Add another role** and add the [*BigQuery Metadata Viewer*](https://cloud.google.com/bigquery/docs/access-control#bigquery.metadataViewer){:target="_blank"} role. |
30 |
| -8. Click **Continue**, then click **Done**. |
31 |
| -9. Search for the service account you've just created. |
32 |
| -11. From your service account, click the three dots under **Actions** and select **Manage keys**. |
33 |
| -12. Click **Add Key > Create new key**. |
34 |
| -13. In the pop-up window, select **JSON** for the key type, and click **Create**. |
35 |
| -14. Copy all the content within the file you've created and downloaded. |
36 |
| -15. Navigate to Segment and paste all the credentials you've just copied into the **Enter your credentials** section as you connect your warehouse destination. |
37 |
| - |
38 |
| -## Grant access to datasets and tables for enrichment |
39 |
| - |
40 |
| -Grant access to datasets and tables so that Segment can list datasets, tables, and columns, and create Linked Events. |
41 |
| - |
42 |
| -Grant |
43 |
| -- [`BigQuery Data Viewer`](https://cloud.google.com/bigquery/docs/access-control#bigquery.dataViewer){:target="_blank"} role <br> |
44 |
| -OR |
45 |
| -- Permissions: |
46 |
| - - `bigquery.datasets.get` |
47 |
| - - `bigquery.tables.list` |
48 |
| - - `bigquery.tables.get` |
49 |
| - - `bigquery.tables.getData` |
50 |
| - |
51 |
| -These can be scoped to projects or [datasets](https://cloud.google.com/bigquery/docs/control-access-to-resources-iam#grant_access_to_a_dataset){:target="_blank"}. |
| 24 | +5. Click **+ Add another role** and add the *[BigQuery User](https://cloud.google.com/bigquery/docs/access-control#bigquery.user){:target="_blank"}* role. |
| 25 | +6. Click **Continue**, then click **Done**. |
| 26 | +7. Search for the service account you just created. |
| 27 | +8. From your service account, click the three dots under **Actions** and select **Manage keys**. |
| 28 | +9. Navigate to **Add Key > Create new key**. |
| 29 | +10. In the pop-up window, select **JSON** for the key type, and click **Create**. The file will download. |
| 30 | +11. Copy all the content in the JSON file you created in the previous step, and save it for Step 5. |
52 | 31 |
|
53 |
| -> info "" |
54 |
| -> To create Linked Events on your listed tables, Segment needs `bigquery.tables.get` and `bigquery.tables.getData` at dataset level. However, you can still scope `bigquery.tables.get` and `bigquery.tables.getData` to specific tables. See BigQuery's [docs](https://cloud.google.com/bigquery/docs/control-access-to-resources-iam#grant_access_to_a_table_or_view){:target="_blank"} for more info. |
| 32 | + |
| 33 | +## Step 2: Grant read-only access for the Data Graph |
| 34 | +Grant the [BigQuery Data Viewer](https://cloud.google.com/bigquery/docs/access-control#bigquery.dataViewer){:target="_blank"} role to the service account at the project level. Make sure to grant read-only access to the Profiles Sync project in case you have a separate project. |
| 35 | + |
| 36 | +To grant read-only access for the Data Graph: |
| 37 | +1. Navigate to **IAM & Admin > IAM** in BigQuery. |
| 38 | +2. Search for the service account you just created. |
| 39 | +3. From your service account, click the **Edit principals pencil**. |
| 40 | +4. Click **ADD ANOTHER ROLE**. |
| 41 | +5. Select the **BigQuery Data Viewer role**. |
| 42 | +6. Click **Save**. |
| 43 | + |
| 44 | +## *(Optional)* Step 3: Restrict read-only access |
| 45 | +If you want to restrict access to specific datasets, grant the BigQuery Data Viewer role on datasets to the service account. Make sure to grant read-only access to the Profiles Sync dataset. |
| 46 | + |
| 47 | +To restrict read-only access: |
| 48 | +1. In the Explorer pane in BigQuery, expand your project and select a dataset. |
| 49 | +2. Navigate to **Sharing > Permissions**. |
| 50 | +3. Click **Add Principal**. |
| 51 | +4. Enter your service account in the New principals section. |
| 52 | +5. Select the **BigQuery Data Viewer** role in the **Select a role** section. |
| 53 | +6. Click **Save**. |
| 54 | + |
| 55 | +You can also run the following command: |
| 56 | + |
| 57 | +``` |
| 58 | +GRANT `roles/bigquery.dataViewer` ON SCHEMA `YOUR_DATASET_NAME` TO "serviceAccount:<YOUR SERVICE ACCOUNT EMAIL>"; |
| 59 | +``` |
| 60 | + |
| 61 | +## Step 4: Validate permissions |
| 62 | +1. Navigate to **IAM & Admin > Service Accounts** in BigQuery. |
| 63 | +2. Search for the service account you’ve just created. |
| 64 | +3. From your service account, click the three dots under **Actions** and select **Manage permissions**. |
| 65 | +4. Click **View Access** and click **Continue**. |
| 66 | +5. Select a box with List resources within resource(s) matching your query. |
| 67 | +6. Click **Analyze**, then click **Run query**. |
| 68 | + |
| 69 | +## Step 5: Connect your warehouse to Segment |
| 70 | +1. Navigate to **Unify > Data Graph** in Segment. This should be a Unify space with Profiles Sync already set up. |
| 71 | +2. Click **Connect warehouse**. |
| 72 | +3. Select *BigQuery* as your warehouse type. |
| 73 | +4. Enter your warehouse credentials. Segment requires the following settings to connect to your BigQuery warehouse: |
| 74 | + * **Service Account Credentials:** JSON credentials for a GCP Service Account that has BigQuery read/write access. This is the credential created in Step 1. |
| 75 | + * **Data Location:** This specifies the primary data location. This can be either region or multi-region. |
| 76 | +5. Test your connection, then click **Save**. |
| 77 | + |
| 78 | +## Update user access for Segment Reverse ETL dataset |
| 79 | +If you ran Segment Reverse ETL in the project you are configuring as the Segment connection project, a Segment-managed dataset is already created and you need to provide the new Segment user access to the existing dataset. |
| 80 | + |
| 81 | +If you run into an error on the Segment app indicating that the user doesn’t have sufficient privileges on an existing `__segment_reverse_etl` dataset, grant the [BigQuery Data Editor](https://cloud.google.com/bigquery/docs/access-control#bigquery.dataEditor){:target="_blank"} role on the `__segment_reverse_etl` dataset to the service account . Note that the `__segment_reverse_etl` dataset is hidden in the console. Run the following SQL command: |
| 82 | + |
| 83 | +``` |
| 84 | +GRANT `roles/bigquery.dataEditor` ON SCHEMA `__segment_reverse_etl` TO "serviceAccount:<YOUR SERVICE ACCOUNT EMAIL>"; |
| 85 | +``` |
0 commit comments