Is there a way to run quickmerge wrapper in multi-threads?

[ShuChen1986]

Hi,

I am dealing with a large genome and therefore running quickmerge took a very long time, is there a way to run the quickmerge wrapper in multi-thread to accelerate the process? I check the wrapper and nucmer help manuals, there seems no option for multi-thread.

Shu

[mahulchak]

You can use mummer 4 which uses multi threading. You can run quickmerge manually after that.

…

On Thu, Nov 21, 2019, 18:15 ShuChen1986 ***@***.***> wrote: Hi, I am dealing with a large genome and therefore running quickmerge took a very long time, is there a way to run the quickmerge wrapper in multi-thread to accelerate the process? I check the wrapper and nucmer help manuals, there seems no option for multi-thread. Shu — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#49?email_source=notifications&email_token=ABZQH2DOUHMVKI3DONIJFATQU46ANA5CNFSM4JQKWSEKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4H3IRY7Q>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABZQH2DIOU7YWU4G4LFNMITQU46ANANCNFSM4JQKWSEA> .

[ShuChen1986]

You can use mummer 4 which uses multi threading. You can run quickmerge manually after that.
…
On Thu, Nov 21, 2019, 18:15 ShuChen1986 @.***> wrote: Hi, I am dealing with a large genome and therefore running quickmerge took a very long time, is there a way to run the quickmerge wrapper in multi-thread to accelerate the process? I check the wrapper and nucmer help manuals, there seems no option for multi-thread. Shu — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#49?email_source=notifications&email_token=ABZQH2DOUHMVKI3DONIJFATQU46ANA5CNFSM4JQKWSEKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4H3IRY7Q>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABZQH2DIOU7YWU4G4LFNMITQU46ANANCNFSM4JQKWSEA .

I used to mummer 4, but multi thread can only used when output format is sam file. Then I used a script sam2delta.py ( https://gist.github.com/malonge/a05da7f99e651ea55e8a4e57a5c4aab1) to convert sam file to delta file, but it gave me the following error:

Traceback (most recent call last):
  File "./sam2delta.py", line 239, in <module>
    write_delta(alns, sam_file + '.delta')
  File "./sam2delta.py", line 140, in write_delta
    f.write('>%s %s %r %r\n' % (aln[0], aln[1], ref_chr_lens[aln[0]], query_len))
KeyError: '001005F'

Could you give me some suggestion on this?

[mahulchak]

sorry for the late response. MUMmer v4 supports multi-threading for both sam and delta. run nucmer like this -
nucmer --threads 4 --prefix foo ref.fasta query.fasta
[assuming that you are using 4 threads]

[Artifice120]

I am having the same issue except mummer is actually running faster at 176 secconds while 4th step in the wrapper takes ~2 and a half hours.

[mahulchak]

Have you tried the running the standalone program?

…

On Mon, Nov 18, 2024, 3:57 PM Artifice120 ***@***.***> wrote: I am having the same issue except mummer is actually running faster at 176 secconds while 4th step in the wrapper takes ~2 and a half hours. image.png (view on web) <https://urldefense.com/v3/__https://github.com/user-attachments/assets/bf8fdd3d-71a0-48cf-aa45-02063e809f2b__;!!CzAuKJ42GuquVTTmVmPViYEvSg!L0UoDdeAwjjnNemg3EtYd414wpwU4Hfvfq2BDmAgy5H43QC0t_oB1P4tY18xImdnhPp54PkSs0G5tHq_eoU1q3wD$> image.png (view on web) <https://urldefense.com/v3/__https://github.com/user-attachments/assets/e9d1b138-5afc-4471-a54a-00ee45fab2a0__;!!CzAuKJ42GuquVTTmVmPViYEvSg!L0UoDdeAwjjnNemg3EtYd414wpwU4Hfvfq2BDmAgy5H43QC0t_oB1P4tY18xImdnhPp54PkSs0G5tHq_egfvWpQg$> — Reply to this email directly, view it on GitHub <https://urldefense.com/v3/__https://github.com/mahulchak/quickmerge/issues/49*issuecomment-2484215459__;Iw!!CzAuKJ42GuquVTTmVmPViYEvSg!L0UoDdeAwjjnNemg3EtYd414wpwU4Hfvfq2BDmAgy5H43QC0t_oB1P4tY18xImdnhPp54PkSs0G5tHq_ehplJz6z$>, or unsubscribe <https://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/ABZQH2BLAX3H6AVEO5JUMRD2BJPFDAVCNFSM6AAAAABSAVCHY6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIOBUGIYTKNBVHE__;!!CzAuKJ42GuquVTTmVmPViYEvSg!L0UoDdeAwjjnNemg3EtYd414wpwU4Hfvfq2BDmAgy5H43QC0t_oB1P4tY18xImdnhPp54PkSs0G5tHq_evZflM0X$> . You are receiving this because you commented.Message ID: ***@***.***>

[jgbaldwinbrown]

Hi all,

Just so you know, merge_wrapper.py is compatible with MUMmer v4. If you have MUMmer v4 installed, you should be able to run with multithreading by including the options -v and -t num_threads when running merge_wrapper.py. Thus, it should look like this:

merge_wrapper.py -v -t num_of_threads assembly1.fa assembly2.fa

[Artifice120]

Many thanks,

It was just a quick

micromamba install -c bioconda mummer4

and the mummer was updated to 4.0

[Artifice120]

Seems to not recognize the extra options even with mummer4

[jgbaldwinbrown]

Hi Artifice,

Are you sure you're using the latest version of merge_wrapper.py? The latest version should have a usage statement that looks as follows:

usage: merge_wrapper.py [-h] [-pre PREFIX] [-hco HCO] [-c C] -l LENGTH_CUTOFF [--no_nucmer] [--no_delta] [--stop_after_nucmer] [--stop_after_df]
                        [-ml MERGING_LENGTH_CUTOFF] [-t THREADS] [-v] [--clean_only]
                        hybrid_assembly_fasta self_assembly_fasta
merge_wrapper.py: error: the following arguments are required: hybrid_assembly_fasta, self_assembly_fasta, -l/--length_cutoff

[Artifice120]

According to conda yes.

maybe its a dev version?

List of packages in environment: "/lustre/isaac/scratch/jtorre28/enviro/mamba/micro/envs/qmerge"

  Name              Version    Build               Channel
────────────────────────────────────────────────────────────────
  _libgcc_mutex     0.1        conda_forge         conda-forge
  _openmp_mutex     4.5        2_gnu               conda-forge
  bzip2             1.0.8      h4bc722e_7          conda-forge
  ca-certificates   2024.8.30  hbcca054_0          conda-forge
  ld_impl_linux-64  2.43       h712a8e2_2          conda-forge
  libexpat          2.6.4      h5888daf_0          conda-forge
  libffi            3.4.2      h7f98852_5          conda-forge
  libgcc            14.2.0     h77fa898_1          conda-forge
  libgcc-ng         14.2.0     h69a702a_1          conda-forge
  libgomp           14.2.0     h77fa898_1          conda-forge
  libmpdec          4.0.0      h4bc722e_0          conda-forge
  libsqlite         3.47.0     hadc24fc_1          conda-forge
  libstdcxx         14.2.0     hc0a3c3a_1          conda-forge
  libstdcxx-ng      14.2.0     h4852527_1          conda-forge
  libuuid           2.38.1     h0b41bf4_0          conda-forge
  libxcrypt         4.4.36     hd590300_1          conda-forge
  libzlib           1.3.1      hb9d3cd8_2          conda-forge
  mummer            3.23       pl5321hdbdd923_20   bioconda
  mummer4           4.0.0rc1   pl5321hdbdd923_7    bioconda
  ncurses           6.5        he02047a_1          conda-forge
  openssl           3.4.0      hb9d3cd8_0          conda-forge
  perl              5.32.1     7_hd590300_perl5    conda-forge
  pip               24.3.1     pyh145f28c_0        conda-forge
  python            3.13.0     h9ebbce0_100_cp313  conda-forge
  python_abi        3.13       5_cp313             conda-forge
  quickmerge        0.3        pl5321hdbdd923_5    bioconda
  readline          8.2        h8228510_1          conda-forge
  tk                8.6.13     noxft_h4845f30_101  conda-forge
  tzdata            2024b      hc8b5060_0          conda-forge
  xz                5.2.6      h166bdaf_0          conda-forge
  zlib              1.3.1      hb9d3cd8_2          conda-forge