Skip to main content

Google Search Reader

Note

This adapter is in preview and is available on Striim Developer only. See Striim Developer for more information.

The Google Search Reader helps you integrate and analyze Google Search data in a structured way for more efficient reporting and decision-making. This reader seamlessly integrates Google search data with other data warehouses that aid in making data-driven decisions based on the performance of various search queries, keywords, and pages. This data can help you refine content strategies and marketing campaigns.

Feature summary

Feature

Supported?

Notes

Objects

Standard objects

✓

Custom objects

✓

Authentication

Basic authentication

Username and password

OAuth authentication

Manual configuration based

Custom authentication methods

✓

API key-based authentication

Operations

Automated mode

✓

Initial load

✓

Pull-based incremental load

✓

Push-based incremental load

Automated pipeline

Governance

Connection profile

Sherlock AI

Sentinel AI

Schema handling

Initial schema creation

✓

Works with supported targets

Schema evolution

Setup

Wizard template

Flow Designer

✓

Striim TQL

✓

Runtime

Resilience/recovery

✓

Supported only for Incremental load.

Recovers the data based on the provided Incremental Load Marker.

Parallel execution

Metrics

✓

Standard metrics

Supported authentication method

The Google Search Reader supports connecting to Google Search using API key-based authentication. To set up and fetch authorization credentials:

Configuring a custom search engine site

  1. Sign in to the Google Custom Search Engine site.

  2. Click New Search Engine and add a site to focus the search on.

  3. Click Edit Search Engine, select your search engine in the menu, and click Setup.

  4. In the Details section, click Search Engine Id to retrieve the search engine ID used as the CustomSearchId in the connection string.

  5. In the Sites to Search section, you can select the Search the Entire Web but Emphasize Included Sites option in the menu.

Getting the API key

To authenticate requests to the Custom Search API,

  1. Log in to the Google API console and select a project.

  2. In the sidebar under APIs & Services, choose Library and then select the Custom Search API. Click Enable.

  3. In the sidebar, choose APIs & Services > Credentials.

  4. Choose Create Credentials > API Key.

  5. Select the API Key to restrict the key to your platform.

Supported objects

The Google Search Reader can read from the following supported objects in Google:

  • ImageSearch

  • WebSearch

Google Search Reader properties

Property

Type

Default value

Notes

API key

Password

Google API key generated for authentication.

Connection pool size

Integer

20

Specifies the maximum number of active connections.

Custom search ID

String

Unique ID associated with the search engine created.

The Search Engine ID (CX) is visible in the settings of your Custom Search Engine.

Exclude tables

String

A list of tables excluded from read operations. Typically used to create a list of exceptions when the Tables property includes wildcards. Misconfiguration of the Tables and Exclude Tables properties can cause "Invalid table names" errors.

Incremental load marker

String

The incremental load marker is a unique incremental column in each object used for incremental load. When no marker is specified, tables are resynced at each polling interval.

Specify the name of the column that contains the start position value. This column must meet the following criteria:

  • It should have an integer or timestamp data type (for example, a creation timestamp or an employee ID).

  • The values must be unique and continuously increasing to ensure proper incremental reading.

Migrate schema

Boolean

False

Only available in Initial Load or Automated mode. Set to True to enable initial schema migration, which propagates the object schema from the source to the target.

Mode

Select list:

  • Automated mode

  • Initial load

  • Incremental load

Automated

Automated mode applies incremental updates to objects that support incremental load and performs full resyncs for objects that do not support incremental load.

Polling interval

Integer

5m

Specifies an interval as an integer followed by a unit. Supported units are days (d), hours (h), minutes (m), or seconds (s). The reader polls the source at the specified interval.

Refresh token

Password

An OAuth 2.0 refresh token.Use the value generated while creating the token.

Search term

String

Search expression for the data you want to extract.

Format: <TableName1>=<searchTerm1>;<TableName2>=<searchTerm2>

For example: WebSearch=striim

This expression extracts data that contains the search phrase "striim."

Start Position

String

%=-1

Value of the incremental load marker that defines the initial reading position.

Tables

String

A semicolon-delimited (;) list of objects to read from the source. Supports the % wildcard. Misconfiguration of the Tables and Exclude Tables properties can cause "Invalid table names" errors. Do not modify this property when recovery is enabled for the application.

Thread pool count

Integer

10

The number of parallel running threads. The default value of zero specifies single-threaded operation.

When the value of the thread pool counter is higher than the connection pool size, large data ingestion operations can cause the app to halt. Since best performance is achieved when using one thread for each table being synced, increasing the size of the connection pool to match the number of threads in use is a performance best practice.