logo

4. Google Cloud Connectors

4.1 Bigtable Loader

Loads data from Google Bigtable.

Use Cases

  • Analyzing time series data
  • Managing large-scale databases

Inputs

  • Service Account Key Path: JSON file containing authentication information (required)
  • Project Name: Google Cloud project name (required)
  • Bigtable Instance ID: Bigtable instance identifier (required)
  • Table Name: Bigtable table name (required)

Outputs

Results in tabular or JSON format

Bigtable Loader Architecture

Bigtable Loader Architecture

4.2 GCP Big Query Loader

Allows you to load data from Google BigQuery.

Use Cases

  • Loading large amounts of data for analysis
  • Exploring big data

Inputs

  • Service Account Key Path: Path to the key JSON file (required)
  • Project Name: Google Cloud Project (required)
  • SQL Query: Query to extract the data (required)

Outputs

JSON data from the queries

Big Query Loader Architecture

Big Query Loader Architecture

4.3 GCP DataStore Loader

Loads data from Google Datastore.

Use Cases

  • Managing NoSQL databases in the cloud

Inputs

  • Service Account Key Path: Path to the key JSON file (required)
  • Project Name: Google Cloud project name (required)
  • Collection Name: Target collection for extracting data (required)

Outputs

JSON documents from the collection

DataStore Loader Architecture

DataStore Loader Architecture

4.4 GCP Firestore Loader

Loads documents from Google Firestore.

Use Cases

  • Retrieving NoSQL data
  • Managing document-oriented databases

Inputs

  • Service Account Key Path: JSON file containing authentication information (required)
  • Project Name: Google Cloud Project Name (required)
  • Collection Name: Target Collection (required)

Outputs

Documents in JSON format

Firestore Loader Architecture

Firestore Loader Architecture

4.5 GCP MS SQL Loader

Loads data from a MS SQL database hosted on Google Cloud.

Use Cases

  • Retrieving relational data for analysis
  • Integrating with other data management systems

Inputs

  • Service Account Key Path: JSON file containing authentication information (required)
  • Project Name: Google Cloud Project Name (required)
  • SQL Query: SQL query to extract the data (required)

Outputs

Tabular or JSON data

MS SQL Loader Architecture

MS SQL Loader Architecture

4.6 GCP MySQL Loader

Loads data from a MySQL database hosted on Google Cloud.

Use Cases

  • Analyzing relational databases
  • Exporting the data for further processing

Inputs

  • Service Account Key Path: JSON file containing authentication information (required)
  • Project Name: Google Cloud project name (required)
  • SQL Query: SQL query to execute (required)

Outputs

Table or JSON results

MySQL Loader Architecture

MySQL Loader Architecture

4.7 GCP Postgres Loader

Loads data from a PostgreSQL database hosted on Google Cloud.

Use Cases

  • Retrieving and processing relational data
  • Analysis and visualization

Inputs

  • Service Account Key Path: JSON file containing authentication information (required)
  • Project Name: Google Cloud project name (required)
  • SQL Query: SQL query to filter data (required)

Outputs

Tabular or JSON data

Postgres Loader Architecture

Postgres Loader Architecture

4.8 GCS Bucket Loader

Loads files from a Google Cloud Storage (GCS) bucket.

Use Cases

  • Extracting data stored in the cloud
  • Migrating or processing files

Inputs

  • Service Account Key Path: JSON file containing authentication information (required)
  • Bucket Name: GCS bucket name (required)
  • Prefix: Path in the bucket for files (optional)

Outputs

List of files and their contents

GCS Bucket Loader Architecture

GCS Bucket Loader Architecture

4.9 GCS File Loader

Loads a specific file from a Google Cloud Storage bucket.

Use Cases

  • Processing a single file
  • Parsing data from specific files

Inputs

  • Service Account Key Path: JSON file containing authentication information (required)
  • Bucket Name: GCS bucket name (required)
  • File Name: Path or name of the file in the bucket (required)

Outputs

Raw content of the file

GCS File Loader Architecture

GCS File Loader Architecture

4.10 Gmail Loader

Loads emails from Gmail using Google account credentials.

Use Cases

  • Inbox analysis for automated processes
  • Archiving or processing emails

Inputs

  • JSON Token: Gmail access token (required)
  • Label ID: Filter emails by label (required)
  • Max Results: Maximum number of emails to retrieve (optional)

Outputs

Emails in JSON format

Gmail Loader Architecture

Gmail Loader Architecture

4.11 Google Drive Loader

Loads files from Google Drive.

Use Cases

  • Integration and management of shared files
  • Automation of document workflows

Inputs

  • JSON Token: Access token to Google Drive (required)
  • Document ID: Unique identifier of the file (required)

Outputs

Contents of the files

Google Drive Loader Architecture

Google Drive Loader Architecture

4.12 Google Drive Search

Allows you to search for files in Google Drive.

Use Cases

  • Dynamic search of documents
  • Retrieval of files by keywords

Inputs

  • JSON Token: Access token to Google Drive (required)
  • Search Term: Keyword to find files (required)

Outputs

List of files that match the criteria

Google Drive Search Architecture

Google Drive Search Architecture

4.13 Google OAuth Token

Generates an OAuth 2.0 token using Google credentials.

Use Cases

  • Secure authentication for Google APIs

Inputs

  • Scope: Required permissions (required)
  • JSON key file: Service account key (required)

Outputs

Valid OAuth token

Google OAuth Token Architecture

Google OAuth Token Architecture

Note: Ensure you have appropriate service account permissions and API access enabled in your Google Cloud Console.