You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
pgloader --version
pgloader version "3.6.a94a0a3"
compiled with SBCL 1.4.0-1.el7
did you test a fresh compile from the source tree? ❌
Compiling pgloader from sources is documented in the README, it's easy to do, and if patches are to be made to fix your bug, you're going to have to build from sources to get the fix anyway…
did you search for other similar issues? => Couldn't find one ❌
how can I reproduce the bug?
I don't know, I only see that on my customers production plateformes from which i'm not allowed to export data
This bug appears from 1/3 to 2/3 of runs, otherwise it just works as expected
Incude a self-contained pgloader command file.
LOAD DATABASE
FROM mysql://username:passwd@ip:3306/pmsipilot2
INTO postgresql://username:passwd@localhost:5432/ods_instance0
with include drop, truncate, create tables, create indexes, batch rows = 10000, batch concurrency = 3,
reset sequences,
downcase identifiers
set
work_mem to '64MB',
maintenance_work_mem to '256MB',
search_path to 'pmsipilot, public'
SET MySQL PARAMETERS
net_read_timeout = '240',
net_write_timeout = '240'
cast type datetime to timestamptz drop default drop not null using zero-dates-to-null,
type date drop not null drop not null drop default using zero-dates-to-null
INCLUDING ONLY TABLE NAMES MATCHING ~/^bda_|^compta_|^cdg_|^sa_|^uo_|^thesaurus/
EXCLUDING TABLE NAMES MATCHING ~/_[0-9]+$/
ALTER SCHEMA 'pmsipilot2' RENAME TO 'pmsipilot'
BEFORE LOAD DO
$$ SELECT pmsipilot.pgloader_drop_tables('qualifact_', true); $$,
$$ SELECT pmsipilot.pgloader_drop_tables('bda_', true); $$,
$$ SELECT pmsipilot.pgloader_drop_tables('compta_', true); $$,
$$ SELECT pmsipilot.pgloader_drop_tables('cdg_', true); $$,
$$ SELECT pmsipilot.pgloader_drop_tables('sa_', true); $$,
$$ SELECT pmsipilot.pgloader_drop_tables('uo_', true); $$,
$$ SELECT pmsipilot.pgloader_drop_tables('thesaurus', true); $$
AFTER LOAD DO
$$ SELECT pmsipilot.grant_select_tables(); $$,
$$ SELECT public.analyze_all_tables_regex('pmsipilot','^qualifact_|^bda_|^compta_|^cdg_|^sa_|^uo_|^thesaurus','_[0-9]+$'); $$,
2025-03-02 18:10:34 175449 [Warning] Aborted connection 175449 to db: 'pmsipilot2' user: 'username' host: '10.29.101.68' (Got an error writing communication packets)
2025-03-02 18:10:34 175448 [Warning] Aborted connection 175448 to db: 'pmsipilot2' user: 'username host: '10.29.101.68' (Got an error writing communication packets)
2025-03-02 18:10:34 CET [60491]: [1-1] app=pgloader,user=usernamedb=ods_instance0LOG: durée : 533325.562 ms, instruction :
copy pmsipilot.rum (id,id_tmp,rss_id,etablissement_id, [...]finess_geo_id,conversion_hc,pec_raac,surveillance_part,admin_prod_rh,rescrit_tarifaire,nb_intervention,codesupinnovation,sup_innovation,non_programme,val_cartcell,val_prot_ped,passage_urgence) FROM STDIN
2025-03-02 18:10:34 CET [65621]: [1-1] app=pgloader,user=usernamedb=ods_instance0LOG: durée : 133051.425 ms, instruction :
copy pmsipilot.rss (id,id_tmp,n[...]_urgence,finess) FROM STDIN
MariaDB version : 10.11.7
Postgresql version : 12.14.1
OS : CentOS7
At this point, the data copying stopped, and pgloader returned an exit code of 1, causing my pipeline to fail.
The text was updated successfully, but these errors were encountered:
for information :
This is happening to multiple customers.
Even when the vm of pgloader and the vm of mariadb are on the same hypervisor, the network is fine.
Please provide the following information:
pgloader --version pgloader version "3.6.a94a0a3" compiled with SBCL 1.4.0-1.el7
did you test a fresh compile from the source tree? ❌
Compiling pgloader from sources is documented in the README, it's easy to do, and if patches are to be made to fix your bug, you're going to have to build from sources to get the fix anyway…
did you search for other similar issues? => Couldn't find one ❌
how can I reproduce the bug?
Incude a self-contained pgloader command file.
PASTE HERE THE OUTPUT OF THE PGLOADER COMMAND
This is mariadb's output that
The text was updated successfully, but these errors were encountered: