> ## Documentation Index
> Fetch the complete documentation index at: https://private-7c7dfe99-fix-nav-issues.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

> You can ingest data into ClickHouse using Google Dataflow

# Integrating Google Dataflow with ClickHouse

export const ClickHouseSupportedBadge = () => {
  return <div className="ClickHouseSupportedBadge">
            <div className="ClickHouseSupportedIcon">
                <svg width="16" height="16" viewBox="0 0 16 16" fill="none" xmlns="http://www.w3.org/2000/svg">
                    <path d="M1.30762 1.39073C1.30762 1.3103 1.37465 1.22986 1.46849 1.22986H2.64824C2.72868 1.22986 2.80912 1.29689 2.80912 1.39073V14.4886C2.80912 14.5691 2.74209 14.6495 2.64824 14.6495H1.46849C1.38805 14.6495 1.30762 14.5825 1.30762 14.4886V1.39073Z" fill="currentColor" />
                    <path d="M4.2832 1.39073C4.2832 1.3103 4.35023 1.22986 4.44408 1.22986H5.62383C5.70427 1.22986 5.7847 1.29689 5.7847 1.39073V14.4886C5.7847 14.5691 5.71767 14.6495 5.62383 14.6495H4.44408C4.36364 14.6495 4.2832 14.5825 4.2832 14.4886V1.39073Z" fill="currentColor" />
                    <path d="M7.25977 1.39073C7.25977 1.3103 7.3268 1.22986 7.42064 1.22986H8.60039C8.68083 1.22986 8.76127 1.29689 8.76127 1.39073V14.4886C8.76127 14.5691 8.69423 14.6495 8.60039 14.6495H7.42064C7.3402 14.6495 7.25977 14.5825 7.25977 14.4886V1.39073Z" fill="currentColor" />
                    <path d="M10.2354 1.39073C10.2354 1.3103 10.3024 1.22986 10.3962 1.22986H11.576C11.6564 1.22986 11.7369 1.29689 11.7369 1.39073V14.4886C11.7369 14.5691 11.6698 14.6495 11.576 14.6495H10.3962C10.3158 14.6495 10.2354 14.5825 10.2354 14.4886V1.39073Z" fill="currentColor" />
                    <path d="M13.2256 6.6057C13.2256 6.52526 13.2926 6.44482 13.3865 6.44482H14.5662C14.6466 6.44482 14.7271 6.51186 14.7271 6.6057V9.27354C14.7271 9.35398 14.6601 9.43442 14.5662 9.43442H13.3865C13.306 9.43442 13.2256 9.36739 13.2256 9.27354V6.6057Z" fill="currentColor" />
                </svg>
            </div>
            ClickHouse Supported
        </div>;
};

[Google Dataflow](https://cloud.google.com/dataflow) is a fully managed stream and batch data processing service. It supports pipelines written in Java or Python and is built on the Apache Beam SDK.

There are two main ways to use Google Dataflow with ClickHouse, both of which leverage [`ClickHouseIO Apache Beam connector`](/integrations/connectors/data-ingestion/etl-tools/apache-beam).
These are:

* [Java runner](#1-java-runner)
* [Predefined templates](#2-predefined-templates)

<h2 id="1-java-runner">
  Java runner
</h2>

The [Java runner](/integrations/connectors/data-ingestion/GCP/google-dataflow/java-runner) allows you to implement custom Dataflow pipelines using the Apache Beam SDK `ClickHouseIO` integration. This approach provides full flexibility and control over the pipeline logic, enabling you to tailor the ETL process to specific requirements.
However, this option requires knowledge of Java programming and familiarity with the Apache Beam framework.

<h3 id="key-features">
  Key features
</h3>

* High degree of customization.
* Ideal for complex or advanced use cases.
* Requires coding and understanding of the Beam API.

<h2 id="2-predefined-templates">
  Predefined templates
</h2>

ClickHouse offers [predefined templates](/integrations/connectors/data-ingestion/GCP/google-dataflow/templates) designed for specific use cases, such as batch imports from BigQuery or streaming ingestion from Pub/Sub into ClickHouse. These templates are ready-to-use and simplify the integration process, making them an excellent choice if you prefer a no-code solution.

<h3 id="key-features-1">
  Key features
</h3>

* No Beam coding required.
* Quick and easy setup for simple use cases.
* Also suitable if you have minimal programming expertise.

Both approaches are fully compatible with Google Cloud and the ClickHouse ecosystem, offering flexibility depending on your technical expertise and project requirements.