Maintenance for Poseidon v3.0.0

.github/dependabot.yml

@@ -0,0 +1,10 @@
+# Set update schedule for GitHub Actions
+
+version: 2
+updates:
+
+  - package-ecosystem: "github-actions"
+    directory: "/"
+    schedule:
+      # Check for updates to GitHub Actions every month
+      interval: "monthly"

.github/workflows/check-release.yaml

@@ -12,7 +12,7 @@ jobs:
 
     steps:
 
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v6
 
       - uses: r-lib/actions/setup-r@v2
         with:

DESCRIPTION

@@ -1,6 +1,6 @@
 Package: janno
 Title: Poseidon .janno file handling in R
-Version: 1.0.0
+Version: 1.1.0
 Authors@R: c(person(given = "Clemens", family = "Schmid", role =
         c("aut", "cre", "cph"), email = "clemens@nevrome.de", comment =
         c(ORCID = "0000-0003-3448-5715")), person(given = "Ayshin",
@@ -10,9 +10,10 @@ License: MIT + file LICENSE
 Encoding: UTF-8
 LazyData: true
 Roxygen: list(markdown = TRUE)
-RoxygenNote: 7.3.1
+RoxygenNote: 7.3.3
 Imports: Bchron, checkmate, dplyr, tidyr, hash, magrittr,
         progress, rlang, readr, tibble, utils, tidyselect, purrr
-Depends: R (>= 2.10)
+Depends: 
+    R (>= 3.5)
 Suggests: testthat (>= 3.0.0)
 Config/testthat/edition: 3

LICENSE

@@ -1,21 +1,2 @@
-# MIT License
-
-Copyright (c) 2020 Clemens Schmid
-
-Permission is hereby granted, free of charge, to any person obtaining a copy
-of this software and associated documentation files (the "Software"), to deal
-in the Software without restriction, including without limitation the rights
-to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-copies of the Software, and to permit persons to whom the Software is
-furnished to do so, subject to the following conditions:
-
-The above copyright notice and this permission notice shall be included in all
-copies or substantial portions of the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-SOFTWARE.
+YEAR: 2025
+COPYRIGHT HOLDER: Clemens Schmid

NEWS.md

@@ -1,22 +1,81 @@
-- V 1.0.0: Multiple changes that justify a new major version number
-  - switched to a new schema version Poseidon v2.7.0
-  - changed the package name from poseidonR to just janno
-  - removed the `upgrade_janno` function
-  - simplified the validation issue reporting in `read_janno`
-  - added a start-up message that highlights that the janno package only supports one Poseidon version
-  - other minor changes, e.g. to the phrasing of error messages
-- V 0.11.1: Unspecified .janno columns are now reported as an issue in the `validate_janno` output table, and not just with a message on the command line.
-- V 0.11.0: Turned off reading .janno columns with an empty header.
-- V 0.10.1: The change in v0.10.0 broke `write_janno`, which could not handle the NULL values properly. This should be fixed now.
-- V 0.10.0: Changed the representation of empty list column values (in `as_janno`). So far when all values in a string list column in a package were empty, then the resulting janno object (after reading) had only entries of type NULL. If some values were non-empty, though, then the empty values were vectors with one element of type NA. Now empty string list entries are always represented by NULL.
-- V 0.9.1: Small adjustments in `upgrade_janno` and update of the source data for Poseidon v2.6.0
-- V 0.9.0: Fixed (changed to a better algorithm) the HDR determination for the radiocarbon calibration, which effects the "Start" and "Stop" output of both `process_age` and `quickcalibrate`
-- V 0.8.0: `process_age` now supports a simple start and stop date output, which can be useful for plotting
-- V 0.7.2: Turned off double escaping of quotes in `write_janno`
-- V 0.7.1: Removed post-R-4.1.0 syntax to make the package compatible again with older R versions
-- V 0.7.0: Added a function `upgrade_janno` to update janno files from Poseidon v2.4.0 to v2.5.0
-- V 0.6.0: Adjusted the package to Poseidon v2.5.0
-- V 0.5.0: Added support for additional arguments to be passed to Bchron in `quickcalibrate`
-- V 0.4.2: Fixed link in documentation
-- V 0.4.1: Improved the message for additional columns in `validate_janno` with a suggestion mechanism
-- V 0.4.0: Introduced a function to easily write janno objects back to .janno files: `write_janno`
+# janno NEWS
+
+## janno 1.1.0
+
+- Switched to a new schema version: Poseidon v3.0.0.
+- Implemented special handling of `_Note` columns, as they are no longer individually defined in the schema, but should still be treated as "defined" columns.
+- Clarified and simplified various info-, warning-, and validation messages.
+- Avoided sorting for the "defined" columns in the reading process.
+
+## janno 1.0.0
+
+- Switched to a new schema version: Poseidon v2.7.0.
+- Changed the package name from poseidonR to janno.
+- Removed the `upgrade_janno()` function.
+- Simplified validation issue reporting in `read_janno()`.
+- Added a start-up message highlighting that janno supports only a single Poseidon version.
+- Various minor improvements, including clearer error message phrasing.
+
+## janno 0.11.1
+
+- Unspecified `.janno` columns are now reported as issues in the `validate_janno()` output table, rather than only via command-line messages.
+
+## janno 0.11.0
+
+- Disabled reading of `.janno` columns with empty headers.
+
+## janno 0.10.1
+
+- Fixed a regression introduced in v0.10.0 where `write_janno()` could not properly handle `NULL` values.
+
+## janno 0.10.0
+
+- Changed the representation of empty string list column values in `as_janno()`.
+  - Previously, columns with mixed empty and non-empty values produced `NA` entries.
+  - Empty string list entries are now consistently represented as `NULL`.
+
+## janno 0.9.1
+
+- Minor adjustments to `upgrade_janno()`.
+- Updated source data for Poseidon v2.6.0.
+
+## janno 0.9.0
+
+- Improved the algorithm for HDR determination in radiocarbon calibration.
+- This affects the `"Start"` and `"Stop"` outputs of both `process_age()` and `quickcalibrate()`.
+
+## janno 0.8.0
+
+- `process_age()` now supports simple start and stop date outputs, useful for plotting.
+
+## janno 0.7.2
+
+- Disabled double escaping of quotes in `write_janno()`.
+
+## janno 0.7.1
+
+- Removed post–R 4.1.0 syntax to restore compatibility with older R versions.
+
+## janno 0.7.0
+
+- Added `upgrade_janno()` to update janno files from Poseidon v2.4.0 to v2.5.0.
+
+## janno 0.6.0
+
+- Updated the package to support Poseidon v2.5.0.
+
+## janno 0.5.0
+
+- Added support for passing additional arguments to **Bchron** in `quickcalibrate()`.
+
+## janno 0.4.2
+
+- Fixed a broken documentation link.
+
+## janno 0.4.1
+
+- Improved messaging for additional columns in `validate_janno()` with a suggestion mechanism.
+
+## janno 0.4.0
+
+- Introduced `write_janno()` to write janno objects back to `.janno` files.

R/janno_enforce_types.R

@@ -1,7 +1,9 @@
 enforce_types <- function(x, suppress_na_introduced_warnings = TRUE) {
 
-  defined_janno_columns <- x %>% dplyr::select(tidyselect::any_of(janno_column_names))
-  undefined_janno_columns <- x %>% dplyr::select(-tidyselect::any_of(janno_column_names))
+  keep_cols <- names(x) %in% janno_column_names | endsWith(names(x), "_Note")
+  defined_janno_columns <- x %>% dplyr::select(which(keep_cols))
+  undefined_janno_columns <- x %>%
+    dplyr::select(-tidyselect::any_of(colnames(defined_janno_columns)))
 
   defined_janno_columns_typed <- purrr::map2(
     as.list(defined_janno_columns), 
@@ -21,8 +23,14 @@ enforce_types <- function(x, suppress_na_introduced_warnings = TRUE) {
 
 apply_col_types <- function(col_data, col_name, suppress_na_introduced_warnings) {
   res <- col_data
-  # lookup context for variable in hashes
-  expected_type <- hash::values(janno_column_name_data_type, col_name)
+  # determine context from column name
+  expected_type <- if (endsWith(col_name, "_Note")) {
+    # special treatment for _Note columns
+    "String"
+  } else {
+    # lookup context for column name in hash map
+    hash::values(janno_column_name_data_type, col_name)
+  }
   # get trans function
   col_trans_function <- string_to_as(expected_type)
   # split to multi if necessary

R/janno_validate.R

@@ -1,10 +1,9 @@
 informative_validation <- function(paths) {
   validation_result <- validate_janno(paths)
-  if (nrow(validation_result) > 0) {
-    message("The following types of issues were detected:")
-    unique_issues <- unique(validation_result$issue)
-    purrr::iwalk(unique_issues, function(x, i) { message(paste0(i, ": ", x)) })
-    message("Run validate_janno() to get the full table of issues.")
+  nr_issues <- nrow(validation_result)
+  if (nr_issues > 0) {
+    message("Issues detected: ", nr_issues)
+    message("Run validate_janno(...) to get the list.")
   } else {
     message("No issues with these .janno files")
   }
@@ -13,7 +12,7 @@ informative_validation <- function(paths) {
 #' @rdname janno
 #' @export
 validate_janno <- function(path) {
-  message("Validating .janno files...")
+  message("Validating .janno files against Poseidon v", poseidon_version, "...")
   message("This validation only checks individual column types, no cross-column integrity")
   # input checks and search for janno files
   janno_file_paths <- get_janno_file_paths(path)
@@ -49,18 +48,23 @@ validate_one_janno <- function(path) {
   check_if_all_mandatory_columns_present(raw_janno)
   # report undefined columns
   undefined_janno_columns <- raw_janno %>% 
+    # remove known columns
     dplyr::select(-tidyselect::any_of(janno_column_names)) %>%
+    # remove _Note columns
+    dplyr::select(-tidyselect::ends_with("_Note")) %>%
+    # show remaining columns
     colnames()
   for (cur_col in undefined_janno_columns) {
     # search for possible column name suggestions
-    string_comparison_index <- utils::adist(cur_col, janno_column_names) %>% which.min
-    closest_colname <- janno_column_names[string_comparison_index]
+    #string_comparison_index <- utils::adist(cur_col, janno_column_names) %>% which.min
+    #closest_colname <- janno_column_names[string_comparison_index]
     issues <- issues %>% append_issue(
       column = cur_col,
       issue = paste(
         "Column not defined in the Poseidon schema.",
         "It will be read as a character column.",
-        "Maybe you mistyped", paste0(closest_colname, "?")
+        "Maybe you mistyped?"
+        #"Maybe you mistyped", paste0(closest_colname, "?")
       )
     )
   }

R/zzz.R

@@ -9,7 +9,11 @@ globalVariables(".")
 NULL
 
 startupmsg <- function() {
-  paste0("janno was build for Poseidon v", poseidon_version)
+  paste(
+    paste0("This janno version was build for Poseidon v", poseidon_version),
+    paste0("Reading and validation will strictly follow this schema version."),
+    sep = "\n"
+  )
 }
 
 # package startup message

data-raw/data_prep.R

@@ -1,8 +1,10 @@
-poseidon_version <- "2.7.0"
 janno_columns <- readr::read_tsv(
   #"https://raw.githubusercontent.com/poseidon-framework/poseidon2-schema/v2.7.0/janno_columns.tsv"
-  "https://raw.githubusercontent.com/poseidon-framework/poseidon2-schema/6133bcd6976ad778683afb6d19af9b2443444899/janno_columns.tsv"
+  #"https://raw.githubusercontent.com/poseidon-framework/poseidon2-schema/6133bcd6976ad778683afb6d19af9b2443444899/janno_columns.tsv"
+  # dev version of v3.0.0
+  "https://raw.githubusercontent.com/poseidon-framework/poseidon-schema/991cfa17aae5476d390b6bd53a33fd6a8655a6d1/janno_columns.tsv"
 )
+poseidon_version <- "3.0.0"
 
 # column names
 janno_column_names <- janno_columns$janno_column_name

tests/testthat/_snaps/process_age.md

@@ -6,7 +6,7 @@
         "names": {
           "type": "character",
           "attributes": {},
-          "value": ["source_file", "Poseidon_ID", "Genetic_Sex", "Group_Name", "Alternative_IDs", "Relation_To", "Relation_Degree", "Relation_Type", "Relation_Note", "Collection_ID", "Country", "Country_ISO", "Location", "Site", "Latitude", "Longitude", "Date_Type", "Date_C14_Labnr", "Date_C14_Uncal_BP", "Date_C14_Uncal_BP_Err", "Date_BC_AD_Start", "Date_BC_AD_Median", "Date_BC_AD_Stop", "Date_Note", "MT_Haplogroup", "Y_Haplogroup", "Source_Tissue", "Nr_Libraries", "Library_Names", "Capture_Type", "UDG", "Library_Built", "Genotype_Ploidy", "Data_Preparation_Pipeline_URL", "Endogenous", "Nr_SNPs", "Coverage_on_Target_SNPs", "Damage", "Contamination", "Contamination_Err", "Contamination_Meas", "Contamination_Note", "Genetic_Source_Accession_IDs", "Primary_Contact", "Publication", "Note", "Keywords", "AdditionalColumn1", "AdditionalColumn2", "Date_BC_AD_Prob", "Date_BC_AD_Start_Derived", "Date_BC_AD_Median_Derived", "Date_BC_AD_Stop_Derived", "Date_BC_AD_Sample"]
+          "value": ["source_file", "Poseidon_ID", "Genetic_Sex", "Group_Name", "Alternative_IDs", "Relation_To", "Relation_Degree", "Relation_Type", "Relation_Note", "Collection_ID", "Country", "Country_ISO", "Location", "Site", "Latitude", "Longitude", "Date_Type", "Date_C14_Labnr", "Date_C14_Uncal_BP", "Date_C14_Uncal_BP_Err", "Date_BC_AD_Start", "Date_BC_AD_Median", "Date_BC_AD_Stop", "Date_Note", "MT_Haplogroup", "Y_Haplogroup", "Source_Material", "Nr_Libraries", "Library_Names", "Capture_Type", "UDG", "Library_Built", "Genotype_Ploidy", "Data_Preparation_Pipeline_URL", "Endogenous", "Nr_SNPs", "Coverage_on_Target_SNPs", "Damage", "Contamination", "Contamination_Err", "Contamination_Meas", "Contamination_Note", "Genetic_Source_Accession_IDs", "Primary_Contact", "Publication", "Note", "Keywords", "AdditionalColumn1", "AdditionalColumn2", "Date_BC_AD_Prob", "Date_BC_AD_Start_Derived", "Date_BC_AD_Median_Derived", "Date_BC_AD_Stop_Derived", "Date_BC_AD_Sample"]
         },
         "row.names": {
           "type": "integer",
@@ -142,9 +142,19 @@
           "value": ["yyy", null, "xxx"]
         },
         {
-          "type": "character",
+          "type": "list",
           "attributes": {},
-          "value": [null, null, null]
+          "value": [
+            {
+              "type": "NULL"
+            },
+            {
+              "type": "NULL"
+            },
+            {
+              "type": "NULL"
+            }
+          ]
         },
         {
           "type": "character",
@@ -269,17 +279,17 @@
             {
               "type": "character",
               "attributes": {},
-              "value": ["xxx", "yyy"]
+              "value": ["petrous", "bone"]
             },
             {
               "type": "character",
               "attributes": {},
-              "value": ["xxx"]
+              "value": ["soft"]
             },
             {
               "type": "character",
               "attributes": {},
-              "value": ["xxx"]
+              "value": ["other"]
             }
           ]
         },
@@ -364,9 +374,25 @@
           "value": [0, 0, 0]
         },
         {
-          "type": "double",
+          "type": "list",
           "attributes": {},
-          "value": [0, 100, 50]
+          "value": [
+            {
+              "type": "double",
+              "attributes": {},
+              "value": [0]
+            },
+            {
+              "type": "double",
+              "attributes": {},
+              "value": [100]
+            },
+            {
+              "type": "double",
+              "attributes": {},
+              "value": [50]
+            }
+          ]
         },
         {
           "type": "list",