> ## Documentation Index
> Fetch the complete documentation index at: https://private-7c7dfe99-fix-nav-issues.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

> Set up Amazon Aurora Postgres as a source for ClickPipes

# Aurora Postgres source setup guide

export const Image = ({img, alt, size}) => {
  return <Frame>
      <img src={img} alt={alt} />
    </Frame>;
};

<h2 id="supported-postgres-versions">
  Supported Postgres versions
</h2>

ClickPipes supports Aurora PostgreSQL-Compatible Edition version 12 and later.

<h2 id="enable-logical-replication">
  Enable logical replication
</h2>

You can skip this section if your Aurora instance already has the following settings configured:

* `rds.logical_replication = 1`

This setting is typically pre-configured if you previously used another data replication tool.

```text theme={null}
postgres=> SHOW rds.logical_replication ;
 rds.logical_replication
-------------------------
 on
(1 row)
```

If not already configured, follow these steps:

1. Create a new parameter group for your Aurora PostgreSQL version with the required setting:
   * Set `rds.logical_replication` to 1

<Image img="https://mintcdn.com/private-7c7dfe99-fix-nav-issues/8xU-7NRzcVe16bmG/images/integrations/data-ingestion/clickpipes/postgres/source/rds/parameter_group_in_blade.png?fit=max&auto=format&n=8xU-7NRzcVe16bmG&q=85&s=bf3dec7b460a2854bea825b58d4ca700" alt="Where to find Parameter groups in Aurora" size="lg" border width="1800" height="819" data-path="images/integrations/data-ingestion/clickpipes/postgres/source/rds/parameter_group_in_blade.png" />

<Image img="https://mintcdn.com/private-7c7dfe99-fix-nav-issues/8xU-7NRzcVe16bmG/images/integrations/data-ingestion/clickpipes/postgres/source/rds/change_rds_logical_replication.png?fit=max&auto=format&n=8xU-7NRzcVe16bmG&q=85&s=ec9a8af1520d7926d965fcd057465b7c" alt="Changing rds.logical_replication" size="lg" border width="1800" height="795" data-path="images/integrations/data-ingestion/clickpipes/postgres/source/rds/change_rds_logical_replication.png" />

2. Apply the new parameter group to your Aurora PostgreSQL cluster

<Image img="https://mintcdn.com/private-7c7dfe99-fix-nav-issues/8xU-7NRzcVe16bmG/images/integrations/data-ingestion/clickpipes/postgres/source/rds/modify_parameter_group.png?fit=max&auto=format&n=8xU-7NRzcVe16bmG&q=85&s=3f55bb68a8609d57fbdd838ea478675e" alt="Modifying Aurora PostgreSQL with new parameter group" size="lg" border width="1800" height="1352" data-path="images/integrations/data-ingestion/clickpipes/postgres/source/rds/modify_parameter_group.png" />

3. Reboot your Aurora cluster to apply the changes

<Image img="https://mintcdn.com/private-7c7dfe99-fix-nav-issues/8xU-7NRzcVe16bmG/images/integrations/data-ingestion/clickpipes/postgres/source/rds/reboot_rds.png?fit=max&auto=format&n=8xU-7NRzcVe16bmG&q=85&s=e7be43059259551c1f747a2abe9fe166" alt="Reboot Aurora PostgreSQL" size="lg" border width="1800" height="757" data-path="images/integrations/data-ingestion/clickpipes/postgres/source/rds/reboot_rds.png" />

<h2 id="configure-database-user">
  Configure database user
</h2>

Connect to your Aurora PostgreSQL writer instance as an admin user and execute the following commands:

1. Create a dedicated user for ClickPipes:

   ```sql theme={null}
   CREATE USER clickpipes_user PASSWORD 'some-password';
   ```

2. Grant schema-level, read-only access to the user you created in the previous step. The following example shows permissions for the `public` schema. Repeat these commands for each schema containing tables you want to replicate:

   ```sql theme={null}
   GRANT USAGE ON SCHEMA "public" TO clickpipes_user;
   GRANT SELECT ON ALL TABLES IN SCHEMA "public" TO clickpipes_user;
   ALTER DEFAULT PRIVILEGES IN SCHEMA "public" GRANT SELECT ON TABLES TO clickpipes_user;
   ```

3. Grant replication privileges to the user:

   ```sql theme={null}
   GRANT rds_replication TO clickpipes_user;
   ```

4. Create a [publication](https://www.postgresql.org/docs/current/logical-replication-publication.html) with the tables you want to replicate. We strongly recommend only including the tables you need in the publication to avoid performance overhead.

<Warning>
  Any table included in the publication must either have a **primary key** defined *or* have its **replica identity** configured to `FULL`. See the [Postgres FAQs](/integrations/clickpipes/postgres/faq#how-should-i-scope-my-publications-when-setting-up-replication) for guidance on scoping.
</Warning>

* To create a publication for specific tables:

  ```sql theme={null}
  CREATE PUBLICATION clickpipes FOR TABLE table_to_replicate, table_to_replicate2;
  ```

* To create a publication for all tables in a specific schema:

  ```sql theme={null}
  CREATE PUBLICATION clickpipes FOR TABLES IN SCHEMA "public";
  ```

The `clickpipes` publication will contain the set of change events generated from the specified tables, and will later be used to ingest the replication stream.

<h2 id="configure-network-access">
  Configure network access
</h2>

<h3 id="ip-based-access-control">
  IP-based access control
</h3>

If you want to restrict traffic to your Aurora cluster, please add the [documented static NAT IPs](/integrations/clickpipes/home#list-of-static-ips) to the `Inbound rules` of your Aurora security group.

<Image img="https://mintcdn.com/private-7c7dfe99-fix-nav-issues/8xU-7NRzcVe16bmG/images/integrations/data-ingestion/clickpipes/postgres/source/rds/security_group_in_rds_postgres.png?fit=max&auto=format&n=8xU-7NRzcVe16bmG&q=85&s=d809713b15c3903640dfecd1b53a473c" alt="Where to find security group in Aurora PostgreSQL?" size="lg" border width="1800" height="707" data-path="images/integrations/data-ingestion/clickpipes/postgres/source/rds/security_group_in_rds_postgres.png" />

<Image img="https://mintcdn.com/private-7c7dfe99-fix-nav-issues/8xU-7NRzcVe16bmG/images/integrations/data-ingestion/clickpipes/postgres/source/rds/edit_inbound_rules.png?fit=max&auto=format&n=8xU-7NRzcVe16bmG&q=85&s=d8d14ebac6077992b5cd0e060b81c315" alt="Edit inbound rules for the above security group" size="lg" border width="1800" height="935" data-path="images/integrations/data-ingestion/clickpipes/postgres/source/rds/edit_inbound_rules.png" />

<h3 id="private-access-via-aws-privatelink">
  Private access via AWS PrivateLink
</h3>

To connect to your Aurora cluster through a private network, you can use AWS PrivateLink. Follow our [AWS PrivateLink setup guide for ClickPipes](/resources/support-center/knowledge-base/cloud-services/aws-privatelink-setup-for-clickpipes) to set up the connection.

<h3 id="aurora-specific-considerations">
  Aurora-specific considerations
</h3>

When setting up ClickPipes with Aurora PostgreSQL, keep these considerations in mind:

1. **Connection Endpoint**: Always connect to the writer endpoint of your Aurora cluster, as logical replication requires write access to create replication slots and must connect to the primary instance.

2. **Failover Handling**: In the event of a failover, Aurora will automatically promote a reader to be the new writer. ClickPipes will detect the disconnection and attempt to reconnect to the writer endpoint, which will now point to the new primary instance.

3. **Global Database**: If you're using Aurora Global Database, you should connect to the primary region's writer endpoint, as cross-region replication already handles data movement between regions.

4. **Storage Considerations**: Aurora's storage layer is shared across all instances in a cluster, which can provide better performance for logical replication compared to standard RDS.

<h3 id="dealing-with-dynamic-cluster-endpoints">
  Dealing with dynamic cluster endpoints
</h3>

While Aurora provides stable endpoints that automatically route to the appropriate instance, here are some additional approaches for ensuring consistent connectivity:

1. For high-availability setups, configure your application to use the Aurora writer endpoint, which automatically points to the current primary instance.

2. If using cross-region replication, consider setting up separate ClickPipes for each region to reduce latency and improve fault tolerance.

<h2 id="whats-next">
  What's next?
</h2>

You can now [create your ClickPipe](/integrations/clickpipes/postgres/index) and start ingesting data from your Aurora PostgreSQL cluster into ClickHouse Cloud.
Make sure to note down the connection details you used while setting up your Aurora PostgreSQL cluster as you will need them during the ClickPipe creation process.
