> ## Documentation Index
> Fetch the complete documentation index at: https://private-7c7dfe99-fix-nav-issues.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

> Seamlessly ingest data from your MySQL or MariaDB database to ClickHouse Cloud.

# Ingesting data from MySQL to ClickHouse (using CDC)

export const Image = ({img, alt, size}) => {
  return <Frame>
      <img src={img} alt={alt} />
    </Frame>;
};

export const galaxyOnClick = eventName => () => {
  try {
    if (typeof window !== "undefined" && window.galaxy && eventName) {
      window.galaxy.track(eventName, {
        interaction: "click"
      });
    }
  } catch (e) {}
};

export const BetaBadge = ({link, galaxyTrack, galaxyEvent}) => {
  if (link) {
    return <a href={link} target="_blank" rel="noopener noreferrer" className="betaBadge" onClick={galaxyTrack && galaxyEvent ? galaxyOnClick(galaxyEvent) : undefined}>
                <Icon />
                <span>Beta</span>
            </a>;
  }
  return <div className="betaBadge">
            <Icon />
            <span>
                Beta feature. 
                <u>
                    <a href="/docs/beta-and-experimental-features#beta-features">
                        Learn more.
                    </a>
                </u>
            </span>
        </div>;
};

<Info>
  Ingesting data from MySQL to ClickHouse Cloud via ClickPipes is in public beta.
</Info>

The MySQL ClickPipe provides a fully-managed and resilient way to ingest data from MySQL and MariaDB databases into ClickHouse Cloud. It supports both **bulk loads** for one-time ingestion and **Change Data Capture (CDC)** for continuous ingestion.

MySQL ClickPipes can be deployed and managed manually using the ClickPipes UI, as well as programmatically using [OpenAPI](/integrations/clickpipes/programmatic-access/openapi) and [Terraform](/integrations/clickpipes/programmatic-access/terraform).

<h2 id="prerequisites">
  Prerequisites
</h2>

[//]: # "TODO Binlog replication configuration isn't needed for one-time ingestion pipes. This has been a source of confusion in the past, so we should also provide the bare minimum requirements for bulk loads to avoid scaring users off."

To get started, you first need to ensure that your MySQL database is correctly configured for binlog replication. The configuration steps depend on how you're deploying MySQL, so please follow the relevant guide below:

<h3 id="supported-data-sources">
  Supported data sources
</h3>

| Name                                                       | Logo | Details                                                                                                                              |
| ---------------------------------------------------------- | ---- | ------------------------------------------------------------------------------------------------------------------------------------ |
| **Amazon RDS MySQL** <br /> *One-time load, CDC*           |      | Follow the [Amazon RDS MySQL](/integrations/clickpipes/mysql/source/rds) configuration guide.                                        |
| **Amazon Aurora MySQL** <br /> *One-time load, CDC*        |      | Follow the [Amazon Aurora MySQL](/integrations/clickpipes/mysql/source/aurora) configuration guide.                                  |
| **Cloud SQL for MySQL** <br /> *One-time load, CDC*        |      | Follow the [Cloud SQL for MySQL](/integrations/clickpipes/mysql/source/gcp) configuration guide.                                     |
| **Azure Flexible Server for MySQL** <br /> *One-time load* |      | Follow the [Azure Flexible Server for MySQL](/integrations/clickpipes/mysql/source/azure-flexible-server-mysql) configuration guide. |
| **Self-hosted MySQL** <br /> *One-time load, CDC*          |      | Follow the [Generic MySQL](/integrations/clickpipes/mysql/source/generic) configuration guide.                                       |
| **Amazon RDS MariaDB** <br /> *One-time load, CDC*         |      | Follow the [Amazon RDS MariaDB](/integrations/clickpipes/mysql/source/rds-maria) configuration guide.                                |
| **Self-hosted MariaDB** <br /> *One-time load, CDC*        |      | Follow the [Generic MariaDB](/integrations/clickpipes/mysql/source/generic-maria) configuration guide.                               |

Once your source MySQL database is set up, you can continue creating your ClickPipe.

<h2 id="create-your-clickpipe">
  Create your ClickPipe
</h2>

Make sure you're logged in to your ClickHouse Cloud account. If you don't have an account yet, you can sign up [here](https://cloud.clickhouse.com/).

[//]: # "   TODO update image here"

1. In the ClickHouse Cloud console, navigate to your ClickHouse Cloud Service.

<Image img="https://mintcdn.com/private-7c7dfe99-fix-nav-issues/lGskH5qUgz9Vtlav/images/integrations/data-ingestion/clickpipes/cp_service.png?fit=max&auto=format&n=lGskH5qUgz9Vtlav&q=85&s=1fa8d992c5f2b6c481249d120e8715a2" alt="ClickPipes service" size="lg" border width="1184" height="482" data-path="images/integrations/data-ingestion/clickpipes/cp_service.png" />

2. Select the `Data Sources` button on the left-side menu and click on "Set up a ClickPipe"

<Image img="https://mintcdn.com/private-7c7dfe99-fix-nav-issues/lGskH5qUgz9Vtlav/images/integrations/data-ingestion/clickpipes/cp_step0.png?fit=max&auto=format&n=lGskH5qUgz9Vtlav&q=85&s=8604d9b9df8655f1edcdc73536e5def1" alt="Select imports" size="lg" border width="2606" height="790" data-path="images/integrations/data-ingestion/clickpipes/cp_step0.png" />

3. Select the `MySQL CDC` tile

<Image img="https://mintcdn.com/private-7c7dfe99-fix-nav-issues/lGskH5qUgz9Vtlav/images/integrations/data-ingestion/clickpipes/mysql/mysql-tile.png?fit=max&auto=format&n=lGskH5qUgz9Vtlav&q=85&s=440aa5315af39662e9dc50c6bc19a0d2" alt="Select MySQL" size="lg" border width="2612" height="892" data-path="images/integrations/data-ingestion/clickpipes/mysql/mysql-tile.png" />

<h3 id="add-your-source-mysql-database-connection">
  Add your source MySQL database connection
</h3>

4. Fill in the connection details for your source MySQL database which you configured in the prerequisites step.

<Info>
  Before you start adding your connection details make sure that you have whitelisted ClickPipes IP addresses in your firewall rules. On the following page you can find a [list of ClickPipes IP addresses](/integrations/clickpipes/home#list-of-static-ips).
  For more information refer to the source MySQL setup guides linked at [the top of this page](#prerequisites).
</Info>

<Image img="https://mintcdn.com/private-7c7dfe99-fix-nav-issues/lGskH5qUgz9Vtlav/images/integrations/data-ingestion/clickpipes/mysql/mysql-connection-details.png?fit=max&auto=format&n=lGskH5qUgz9Vtlav&q=85&s=26fbf77f95a6bc2a8f918f58c67e9530" alt="Fill in connection details" size="lg" border width="1842" height="1556" data-path="images/integrations/data-ingestion/clickpipes/mysql/mysql-connection-details.png" />

<h4 id="optional-set-up-ssh-tunneling">
  (Optional) Set up SSH Tunneling
</h4>

You can specify SSH tunneling details if your source MySQL database isn't publicly accessible.

1. Enable the "Use SSH Tunnelling" toggle.

2. Fill in the SSH connection details.

   <Image img="https://mintcdn.com/private-7c7dfe99-fix-nav-issues/8xU-7NRzcVe16bmG/images/integrations/data-ingestion/clickpipes/postgres/ssh-tunnel.jpg?fit=max&auto=format&n=8xU-7NRzcVe16bmG&q=85&s=c257ddbc479fb91047b029a3ddef29aa" alt="SSH tunneling" size="lg" border width="1780" height="1342" data-path="images/integrations/data-ingestion/clickpipes/postgres/ssh-tunnel.jpg" />

3. To use Key-based authentication, click on "Revoke and generate key pair" to generate a new key pair and copy the generated public key to your SSH server under `~/.ssh/authorized_keys`.

4. Click on "Verify Connection" to verify the connection.

<Note>
  Make sure to whitelist [ClickPipes IP addresses](/integrations/clickpipes/home#list-of-static-ips) in your firewall rules for the SSH bastion host so that ClickPipes can establish the SSH tunnel.
</Note>

Once the connection details are filled in, click `Next`.

<h4 id="advanced-settings">
  Configure advanced settings
</h4>

You can configure the advanced settings if needed. A brief description of each setting is provided below:

* **Sync interval**: This is the interval at which ClickPipes will poll the source database for changes. This has an implication on the destination ClickHouse service, for cost-sensitive users we recommend to keep this at a higher value (over `3600`).
* **Parallel threads for initial load**: This is the number of parallel workers that will be used to fetch the initial snapshot. This is useful when you have a large number of tables and you want to control the number of parallel workers used to fetch the initial snapshot. This setting is per-table.
* **Pull batch size**: The number of rows to fetch in a single batch. This is a best effort setting and may not be respected in all cases.
* **Snapshot number of rows per partition**: This is the number of rows that will be fetched in each partition during the initial snapshot. This is useful when you have a large number of rows in your tables and you want to control the number of rows fetched in each partition.
* **Snapshot number of tables in parallel**: This is the number of tables that will be fetched in parallel during the initial snapshot. This is useful when you have a large number of tables and you want to control the number of tables fetched in parallel.

<h3 id="configure-the-tables">
  Configure the tables
</h3>

5. Here you can select the destination database for your ClickPipe. You can either select an existing database or create a new one.

   <Image img="https://mintcdn.com/private-7c7dfe99-fix-nav-issues/8xU-7NRzcVe16bmG/images/integrations/data-ingestion/clickpipes/mysql/select-destination-db.png?fit=max&auto=format&n=8xU-7NRzcVe16bmG&q=85&s=25847aef93aa36d6d3f94cdc2649579a" alt="Select destination database" size="lg" border width="2368" height="614" data-path="images/integrations/data-ingestion/clickpipes/mysql/select-destination-db.png" />

6. You can select the tables you want to replicate from the source MySQL database. While selecting the tables, you can also choose to rename the tables in the destination ClickHouse database as well as exclude specific columns.

<h3 id="review-permissions-and-start-the-clickpipe">
  Review permissions and start the ClickPipe
</h3>

7. Select the "Full access" role from the permissions dropdown and click "Complete Setup".

   <Image img="https://mintcdn.com/private-7c7dfe99-fix-nav-issues/8xU-7NRzcVe16bmG/images/integrations/data-ingestion/clickpipes/postgres/ch-permissions.jpg?fit=max&auto=format&n=8xU-7NRzcVe16bmG&q=85&s=220285c9d16d656b26c97fd0587b9086" alt="Review permissions" size="lg" border width="1844" height="716" data-path="images/integrations/data-ingestion/clickpipes/postgres/ch-permissions.jpg" />

Finally, please refer to the ["ClickPipes for MySQL FAQ"](/integrations/clickpipes/mysql/faq) page for more information about common issues and how to resolve them.

<h2 id="whats-next">
  What's next?
</h2>

[//]: # "TODO Write a MySQL-specific migration guide and best practices similar to the existing one for PostgreSQL. The current migration guide points to the MySQL table engine, which isn't ideal."

Once you've set up your ClickPipe to replicate data from MySQL to ClickHouse Cloud, you can focus on how to query and model your data for optimal performance. For common questions around MySQL CDC and troubleshooting, see the [MySQL FAQs page](/integrations/clickpipes/mysql/faq).