Skip to main content

BigQuery Writer initial setup

Initial BigQuery setup

You must perform the following one-time tasks before you can use BigQuery Writer in an application.

Get the project ID for BigQuery

If you have not used BigQuery before, see BigQuery > Documentation > Guides > What is BigQuery? > Get started with BigQuery.

BigQuery_projectID.png

Above, the project is rlproject2 and the BigQuery project ID is rlproject2-358421. You will need your project ID to configure BigQuery Writer to write to the tables in the project's datasets.

Create a service account with the necessary BigQuery permissions

Before you can use BigQuery Writer in a Striim application, you must create a service account (see Service accounts overview). The service account must have the following roles (see BigQuery > Documentation > Guides > Introduction to IAM > BigQuery predefined Cloud IAM roles):

  • bigquery.dataEditor for the target project or dataset

  • bigquery.jobUser for the target project

  • bigquery.resourceAdmin

Alternatively, instead of using those predefined roles, you may create a custom role with the following permissions for the target tables (see BigQuery > Documentation > Guides > Introduction to IAM > BigQuery custom roles):

  • bigquery.datasets.get

  • bigquery.jobs.create

  • bigquery.jobs.get

  • bigquery.jobs.list

  • bigquery.jobs.listAll

  • bigquery.tables.create

  • bigquery.tables.delete

  • bigquery.tables.get

  • bigquery.tables.getData

  • bigquery.tables.list

  • bigquery.tables.update

  • bigquery.tables.updateData

  • bigquery.tables.updateTag

Download the service account key

Download the service account's key file (see iAM > Documentation > Guides > Create and delete service account keys > Create a service account key) and copy it to the same location on each Striim server that will run BigQuery Writer, or to a network location accessible by all servers. When creating an application, specify the path to and name of this file in BigQuery Writer's Service Account Key property.

Configure Striim to work with BigQuery

All clients and drivers required by BigQuery are bundled with Striim.  No additional setup is required.