Skip to content

Commit

Permalink
RC CRAN resubmission (#215)
Browse files Browse the repository at this point in the history
## Major changes
* **`sen2r`** package directory is no more used to store files, but a subfolder `.sen2r` of the user Home directory is created and used.    In this way, reinstalling sen2r will not require anymore to reinstall runtime dependencies / to reconfigure the package.    The permission to write on this folder is asked to the user when the package is loaded for the first time.

## Other changes
* Small base example files were added to the package for testing purposes.
* Sen2Cor version which the user wants to install can be chosen in the GUI.
* Documentation was updated and  improved, in particular adding function examples.

## New functions
* `build_example_param_file()` (not exported) to build an example parameter file.

## Fixes
* Fix generation of thumbnails in case of multiple product types.
  • Loading branch information
ranghetti authored Sep 27, 2019
1 parent 7fab92a commit 276860a
Show file tree
Hide file tree
Showing 240 changed files with 4,597 additions and 2,378 deletions.
1 change: 1 addition & 0 deletions .Rbuildignore
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
^.*\.Rproj$
^\.Rproj\.user$
^\.travis\.yml$
^\.covrignore\.yml$
^.*\.yaml$
^README\.R?md$
^WIP$
Expand Down
28 changes: 0 additions & 28 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -23,31 +23,3 @@ vignettes/*.pdf
*.knit.md
.Rproj.user
*.Rproj
# dependencies (sen2cor, xslt)
/inst/sen2cor_2-[58]-[50]/
/sen2cor_2-[58]-[50]/
/extdata/xslt/
/inst/extdata/xslt/
/query_results.xml
/inst/aria2.exe
/aria2.exe
*.pyc

# path file
/extdata/paths.json
/inst/extdata/paths.json
# other files
/extdata/vector/s2_tiles.rds
/inst/extdata/vector/s2_tiles.rds
/extdata/example_files/safe/
/inst/extdata/example_files/safe/
/extdata/testdata/example_files/safe/
/inst/extdata/testdata/example_files/safe/
/extdata/example_files/out/
/inst/extdata/example_files/out/
/extdata/testdata/example_files/out/
/inst/extdata/testdata/example_files/out/
/logs/
/inst/logs/
/extdata/apihub.txt
/inst/extdata/apihub.txt
10 changes: 7 additions & 3 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,10 @@
language: R
sudo: required
dist: bionic
dist: xenial
cache: packages
r:
- release
- devel

r_packages:
- RPostgreSQL
Expand All @@ -27,14 +30,15 @@ apt_packages:
before_install:
- sudo apt-get -qq update
- sudo apt-get install -y libgdal-dev libproj-dev libv8-dev libprotobuf-dev libudunits2-dev netcdf-bin libv8-dev libjq-dev
- sudo apt-get install -y gdal-bin python-gdal

script:
- R CMD build .
- travis_wait 120 R CMD check *tar.gz

after_success:
- Rscript -e 'covr::codecov(line_exclusions = c("R/s2_gui.R", "R/add_rgb_image.R", "R/geograbber_process.R", "R/editModPoly.R", "R/install_aria2.R", "R/install_sen2cor.R", "R/create_s2_dop.R"))'

- Rscript -e 'covr::codecov(line_exclusions = c("R/s2_gui.R", "R/add_rgb_image.R", "R/geograbber_process.R", "R/editModPoly.R", "R/install_aria2.R", "R/install_sen2cor.R", "R/ask_permission.R", "R/check_sen2r_deps.R", "R/create_indices_db.R", "R/helpers_extent.R", "R/create_indices_db.R", "R/gdal_formats_db.R", "R/path_check.R", "R/list_sen2r_paths.R", "R/check_param_list.R", "R/convert_datatype.R"))'
env:
global:
- R_LIBS="http://cran.rstudio.com"
Expand Down
2 changes: 2 additions & 0 deletions CRAN-RELEASE
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
This package was submitted to CRAN on 2019-09-13.
Once it is accepted, delete this file and tag the release (commit 48f043460c).
23 changes: 12 additions & 11 deletions DESCRIPTION
Original file line number Diff line number Diff line change
Expand Up @@ -2,25 +2,26 @@ Package: sen2r
Type: Package
Title: Find, Download and Process Sentinel-2 Data
Version: 1.1.0
Authors@R: c(person("Luigi", "Ranghetti",
email = "ranghetti.[email protected]",
role = c("aut", "cre"),
Authors@R: c(person("Luigi", "Ranghetti",
email = "luigi@ranghetti.info",
role = c("aut", "cre"),
comment = c(ORCID = "0000-0001-6207-5188")),
person("Lorenzo", "Busetto",
email = "[email protected]",
person("Lorenzo", "Busetto",
email = "[email protected]",
role = c("aut"),
comment = c(ORCID = "0000-0001-9634-6038")))
Description: Functions to download Sentinel-2 optical images
and perform preliminary processing operations.
'sen2r' provides the instruments required to easily perform
(and eventually automate) the steps necessary to build a complete
Description: Functions to download Sentinel-2 optical images
and perform preliminary processing operations.
'sen2r' provides the instruments required to easily perform
(and eventually automate) the steps necessary to build a complete
Sentinel-2 processing chain.
A Graphical User Interface to facilitate data processing is also provided.
License: GPL-3
Encoding: UTF-8
URL: http://sen2r.ranghetti.info, https://github.com/ranghetti/sen2r
BugReports: https://github.com/ranghetti/sen2r/issues
Imports:
Depends: R (>= 3.5.0)
Imports:
reticulate,
methods,
digest,
Expand All @@ -46,7 +47,7 @@ Imports:
units,
httr
Suggests:
spelling,
spelling,
geojsonlint,
httptest,
knitr,
Expand Down
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
FROM rocker/geospatial:3.5.1

LABEL maintainer="Luigi Ranghetti <ranghetti.[email protected]>"
LABEL maintainer="Luigi Ranghetti <luigi@ranghetti.info>"

# Install external dependencies
RUN apt-get update && apt-get install -y \
Expand Down
1 change: 1 addition & 0 deletions NAMESPACE
Original file line number Diff line number Diff line change
Expand Up @@ -245,6 +245,7 @@ importFrom(shinydashboard,tabItems)
importFrom(shinyjs,click)
importFrom(shinyjs,delay)
importFrom(shinyjs,disable)
importFrom(shinyjs,disabled)
importFrom(shinyjs,enable)
importFrom(shinyjs,extendShinyjs)
importFrom(shinyjs,hidden)
Expand Down
8 changes: 7 additions & 1 deletion NEWS.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,9 @@
* Sen2Cor v. 2.8.0 (faster and using less memory) is now installed by default.

## Major changes
* **`sen2r`** package directory is no more used to store files, but a subfolder `.sen2r` of the user Home directory is created and used.
In this way, reinstalling sen2r will not require anymore to reinstall runtime dependencies / to reconfigure the package.
The permission to write on this folder is asked to the user when the package is loaded for the first time.
* `s2_list()` and `s2_download()` now use internal methods instead of calling python function derived from s2download scripts by Hagolle (this substantially improve the speed of SAFE search), which were removed.
* **`sen2r`** is now supported also on MacOS.

Expand All @@ -22,6 +25,7 @@
* `raster_metadata()`, to get raster metadata, without rgdal dependency.
* `check_scihub_login()` to check if SciHub credentials are right.
* `check_scihub_connection()` to check for internet connectivity.
* `build_example_param_file()` (not exported) to build an example parameter file.

## Other changes
* Packages stability was improved by adding test functions, working with **`testthat`**.
Expand All @@ -35,8 +39,10 @@
* Several "soft" error/warning messages were introduced.
* ENVI headers now includes some information about band names (TOA/BOA) and SCL classes.
* Coupling SAFE: in offline mode, if some L2A products exists but not the corresponding L1C, and TOA-derived products are required, only the "coupled" SAFE archives are used.
* Small base example files were added to the package for testing purposes.
* Sen2Cor version which the user wants to install can be chosen in the GUI.
* Documentation was updated and improved, in particular adding function examples.
* Unused old code was removed.
* Documentation was updated.
* Several bugs were fixed.


Expand Down
4 changes: 2 additions & 2 deletions R/abs2rel.R
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
#' warning; if FALSE nothing is shown.
#' @return The relative path
#'
#' @author Luigi Ranghetti, phD (2017) \email{ranghetti.l@@irea.cnr.it}
#' @author Luigi Ranghetti, phD (2017) \email{luigi.ranghetti@@gmail.com}
#' @note License: GPL 3.0
#' @export
#' @importFrom magrittr "%>%"
Expand Down Expand Up @@ -71,7 +71,7 @@ abs2rel <- function(path, ref_path, mustWork=NA) {
print_message(
type="warning",
"Input (",path,") and reference (",ref_path,
") paths have not a common parent directory; ",
") paths do not have a common parent directory; ",
"an absolute path is returned.")
return(path)
}
Expand Down
2 changes: 1 addition & 1 deletion R/add_rgb_image.R
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
#' @description Modal dialog to define an RGB image.
#' @param s2_bands 2-length list (one for TOA, one for BOA),
#' each element being a list of S2 bands, as defined in [s2_gui].
#' @author Luigi Ranghetti, phD (2018) \email{ranghetti.l@@irea.cnr.it}
#' @author Luigi Ranghetti, phD (2019) \email{luigi@@ranghetti.info}
#' @note License: GPL 3.0
#' @importFrom shiny actionButton column fluidRow HTML icon modalButton
#' modalDialog radioButtons sliderInput strong tagList
Expand Down
45 changes: 45 additions & 0 deletions R/ask_permission.R
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
#' @title Ask permission to write settings on disk
#' @description Ask users for permission to create a .sen2r settings directory,
#' in which to store the files paths.json and apihub.txt.
#' @return 'logical' if TRUE, the user authorised saving in this directory.
#' @author Lorenzo Busetto, phD (2019) \email{lbusett@@gmail.com}
#' @author Luigi Ranghetti, phD (2019) \email{luigi@@ranghetti.info}
#' @note License: GPL 3.0

ask_permission <- function() {

settings_dir <- normalize_path("~/.sen2r", mustWork = FALSE)

if (!dir.exists(settings_dir)) {
message(paste(
"\nsen2r would like to create a hidden folder in your home directory, ",
paste0("named '",settings_dir,"', "),
"in which storing some files required by the package:",
"- 'paths.json', containing the paths of sen2r runtime dependencies ",
" (GDAL, aria2, Sen2Cor);",
"- 'apihub.txt', in which saving the user's SciHub credentials;",
"- 's2_tiles.rds', containing the Sentinel-2 tile boundaries;",
"- the 'log' subfolder, used to keep trace of temporary files created ",
" during processing operations.\n",
"Do you authorise this?\n",
"Yes: folder '.sen2r' will be created permanently, and the files ",
" will be created and updated when needed. ",
" You will not see this message anymore.\n",
"No: a temporary folder will be used and will be lost when exiting ",
" R. You will see this message every time you will restart R.\n",
sep = "\n"
))
choice <- ""
while (!tolower(choice) %in% c("y", "yes", "n", "no")) {
choice <- readline(prompt = "Choice (y/n): ")
}
if (tolower(choice) %in% c("y", "yes")) {
dir.create(settings_dir)
}
} else {
choice <- "y"
}

return(tolower(choice) %in% c("y", "yes"))

}
110 changes: 110 additions & 0 deletions R/build_example_param_file.R
Original file line number Diff line number Diff line change
@@ -0,0 +1,110 @@
#' @title Build an example JSON parameter file
#' @description Function used to write JSON parameter file.
#' A function is provided instead than a json file to ensure
#' directories to match the user folder tree.
#' @param json_path Path of the output file. Default is to save it on a
#' temporary file, whose path is returned.
#' @param overwrite Logical value: should existing output file be
#' overwritten? (default: TRUE)
#' @return The path of the created file.
#'
#' @author Luigi Ranghetti, phD (2019) \email{luigi@@ranghetti.info}
#' @note License: GPL 3.0
#' @importFrom jsonlite toJSON
#' @examples
#' sen2r:::build_example_param_file()

build_example_param_file <- function(
json_path = tempfile(fileext = "_sen2r_params.json"),
overwrite = TRUE
) {

# Check json_path
if (json_path == "") {json_path <- NULL}
if (!attr(path_check(dirname(json_path)), "isvalid")) {
print_message(
type = "error",
"Path '",dirname(json_path),"' is missing or not writable."
)
}

# Delete existing file
if (file.exists(json_path)) {
if (overwrite == TRUE) {
file.remove(json_path)
} else {
print_message(
type = "error",
"File '",json_path,"' already exists."
)
}
}

writeLines(
jsonlite::toJSON(
list(
"preprocess" = TRUE,
"s2_levels" = c("l1c", "l2a"),
"sel_sensor" = c("s2a", "s2b"),
"online" = TRUE,
"downloader" = "builtin",
"overwrite_safe" = FALSE,
"rm_safe" = "no",
"max_cloud_safe" =100,
"step_atmcorr" = "l2a",
"timewindow" = c("2017-07-03", "2017-07-03"),
"timeperiod" = "full",
"extent" = system.file("extdata/vector/barbellino.geojson", package = "sen2r"),
"s2tiles_selected" = NA,
"s2orbits_selected" = NA,
"list_prods" = c("TOA", "BOA", "SCL"),
"list_indices" = c("MSAVI2", "NDVI"),
"list_rgb" = c("RGB432B", "RGB432T", "RGB843B"),
"rgb_ranges" = list(
c(0, 2500),
c(0, 2500),
matrix(c(0, 0, 0, 7500, 2500, 2500), ncol = 2)
),
"index_source" = "BOA",
"mask_type" = NA,
"max_mask" = 80,
"mask_smooth" = 0,
"mask_buffer" = 0,
"clip_on_extent" = TRUE,
"extent_as_mask" = FALSE,
"extent_name" = "sen2r",
"reference_path" = NA,
"res" = NA,
"res_s2" ="10m",
"unit" = "Meter",
"proj"= NA,
"resampling" = "near",
"resampling_scl" = "near",
"outformat" = "GTiff",
"rgb_outformat" = "GTiff",
"index_datatype" = "Int16",
"compression" = "DEFLATE",
"rgb_compression" = "DEFLATE",
"overwrite" = FALSE,
"path_l1c" = file.path(dirname(attr(load_binpaths(), "path")), "safe"),
"path_l2a" = file.path(dirname(attr(load_binpaths(), "path")), "safe"),
"path_tiles"= NA,
"path_merged"= NA,
"path_out" = tempfile(pattern = "sen2r_out_"),
"path_rgb" = "",
"path_indices" = "",
"path_subdirs"= TRUE,
"thumbnails" = TRUE,
"log"= NA,
"parallel" = FALSE,
"processing_order" = "by_groups",
"pkg_version" = as.character(packageVersion("sen2r"))
),
pretty = TRUE
),
json_path
)

return(json_path)

}
Loading

0 comments on commit 276860a

Please sign in to comment.