[CDCSDK] Data loss observed on master 2.25.1.0-b30

[ShikharSahay]

Jira Link: DB-14768

Description

test_cdc_lru_nemesis_splitting_postgres_debezium failed on 2.25.1.0-b30 due to data loss. Sink count seems to be stuck after ~ 45 mins.

2024-12-17 01:31:01,036: INFO: postgresql output: [{'count': '396854'}]
2024-12-17 01:31:01,036: INFO: waiting replication test_cdc_c6cdf7 yb=396857 sink=396854 kafka_topic_size=7271865

----------->

2024-12-17 02:11:58,318: INFO: waiting replication test_cdc_c6cdf7 yb=396857 sink=396854 kafka_topic_size=7271865
2024-12-17 02:11:58,318: INFO: found stuck replication table=test_cdc_c6cdf7 yb=396857 sink=396854 kafka_topic_size=7271865

Source connector version

quay.io/yugabyte/debezium-connector:dz.1.9.5.yb.grpc.2024.1

Connector configuration

add connector connector_name='ybconnector_cdc_53b88a_test_cdc_c6cdf7_test_cdc_8dae80' stream_id='71dc8e01325864ad9e466db678d0a9ca' db_name='cdc_53b88a' connector_host='172.151.27.25' table_list=['test_cdc_c6cdf7', 'test_cdc_8dae80'] {'name': 'ybconnector_cdc_53b88a_test_cdc_c6cdf7_test_cdc_8dae80', 'config': {'database.master.addresses': '172.151.20.134:7100,172.151.28.132:7100,172.151.28.0:7100', 'database.hostname': '172.151.20.134:5433,172.151.28.132:5433,172.151.28.0:5433', 'database.port': 5433, 'database.masterhost': '172.151.20.134', 'database.masterport': '7100', 'database.user': 'yugabyte', 'database.password': 'yugabyte', 'database.dbname': 'cdc_53b88a', 'snapshot.mode': 'never', 'admin.operation.timeout.ms': 600000, 'socket.read.timeout.ms': 300000, 'max.connector.retries': '10', 'operation.timeout.ms': 600000, 'topic.creation.default.compression.type': 'lz4', 'topic.creation.default.cleanup.policy': 'delete', 'topic.creation.default.partitions': 2, 'topic.creation.default.replication.factor': '1', 'tasks.max': '5', 'connector.class': 'io.debezium.connector.yugabytedb.YugabyteDBgRPCConnector', 'database.server.name': 'db_cdc', 'database.streamid': '71dc8e01325864ad9e466db678d0a9ca', 'table.include.list': 'public.test_cdc_c6cdf7,public.test_cdc_8dae80', 'database.sslrootcert': '/kafka/ca.crt'}}

YugabyteDB version

2.25.1.0-b30

Issue Type

kind/bug

Warning: Please confirm that this issue does not contain any sensitive information

• I confirm this issue does not contain any sensitive information.