Database Storage API
The connector can leverage the Database Storage API, which allows higher throughput than the standard API. This enables the connector to handle large result sets more efficiently. For more information about the API, see "BigQuery Storage API Overview" in the Google Database documentation: https://cloud.google.com/bigquery/docs/reference/storage/.
If this feature is enabled, the connector checks the number of rows in an incoming result set table and the number of pages needed to retrieve all the results. If the number of rows and pages exceeds the defined threshold, the connector switches to using the Database Storage API. If the connector encounters any issues initializing the storage API for retrieval, it falls back to using the standard API, unless this is a permissions issue. To ensure best performance, do not use this feature with a named destination dataset or table.
You can customize the thresholds for using the Database Storage API. For information about the configuration options used to determine when the API is used, see the following:
- EnableHighThroughputAPI
- HighThroughputActivationRatio
- HighThroughputMinTableSize
- UnsupportedHTAPIFallback
In order to take advantage of this feature, the Database project you are querying must have the Database Storage API enabled. For more information, see "Enabling the API" in the Google Database documentation: https://cloud.google.com/bigquery/docs/reference/storage/#enabling_the_api.
Important:
Pricing for the Database Storage API is different than pricing for the standard API. For more information, see "Database Storage API Pricing" in the Google Database documentation: https://cloud.google.com/bigquery/pricing#storage-api.
Using the large result set support feature can reduce some of the performance gains
from the Database Storage API. If you enable the Database Storage API, we
recommend setting AllowLargeResults to 0
The INTERVAL data type is not supported on the Read API. When retrieving data from a column of the INTERVAL type, the connector returns an error. To enable the connector to fallback to the REST API, set UnsupportedHTAPIFallback to 1.
- Features
- Installing and Using the Product
- Configuring the Product on page 1
- Large Results Driver Configuration Options