database in mapping report

[IanSudbery]

The location of the database for pipelines is set by database_name in the ini. However, by default CGATReport looks for the database name in report_sql_backend. Some pipelines bypass this by passing backend=PARAMS["database_name"] to the TrackerSQL constructor in their pipeline specific subclassing of TrackerSQL.

I don't know if CGATReport should look for database_name for report trackers should pass database_name, but one of these should happen else the report can't find the database unless it is called csvdb and is in the current directory.

[sebastian-luna-valero]

Thanks @IanSudbery

Should this be reported in the CGATReport repository instead?
https://github.com/AndreasHeger/CGATReport/issues

Best regards,
Sebastian

[sebastian-luna-valero]

Is this issue related with this one?
#54

If so, have you tried the proposed solution?

[AndreasHeger]

CGATReport and CGATPipelines are best considered separate tools, where the former is used by the latter. report_sql_backend makes sense for CGATReport, as it describes better than "database" what is needed (i.e. not a NoSQL database). While in CGATPipelines we have made the decision to use SQL database, so "database" is sufficient. Renaming it to "report_sql_backend" would also not be quite right, as the database is the primary data repository for the database. It is not just used for report building for CGAT, but also, for example, in notebooks or within the Pipeline itself.

[IanSudbery]

I agree that I don’t think CGATReport should change. However, having to add the path to the database in two different places in production pipelines is tiresome, particularly when it isn’t a setting that is present to be set by default in the pipeline.ini In other pipelines, the base sub-class of TrackerSQL passes database_name to the constructor. I think this is a good solution.

[Acribbs]

@IanSudbery should this be closed?

[IanSudbery]

Many pipelines are still not setting the SQL backend correctly. A brief check shows that at least mapping and readqc are not:

Mapping uses the default CGATReport locations.
readqc actaully overrides this with its own special set of locations that are different from CGATReport, but still not the correct ones.

[Acribbs]

Ok, im not really best to solve this as I always found CGAT report an enigma. Just so you are aware we are hoping to transition away from CGAT report and rely on multiqc for generic QC visualisation and bespoke reporting using Rmarkdown or Jupiter notebook in the future.