windows-latest and macos-latest pipelines not failing with `rcmdcheck`

## Describe the bug
We're currently experiencing issues in pipelines where `rcmdcheck::rcmdcheck` is not raising an error when failing.

## To Reproduce
Since this is a pipeline we're running in internal system, not sure how to share the link with you. I will post logs, if it works it works:


### Pipeline 1 - windows-latest and macos-latest

In this one we have error when trying to catch dependencies it fails and tests are not even run:

<details>
    <summary>Logs </summary>

```
## --------------------------------------------------------------------
  options(crayon.enabled = TRUE)
  cat("LOGNAME=", Sys.info()[["user"]], "\n", sep = "", file = Sys.getenv("GITHUB_ENV"), append = TRUE)
  if (Sys.getenv("_R_CHECK_FORCE_SUGGESTS_", "") == "") Sys.setenv("_R_CHECK_FORCE_SUGGESTS_" = "false")
  if (Sys.getenv("_R_CHECK_CRAN_INCOMING_", "") == "") Sys.setenv("_R_CHECK_CRAN_INCOMING_" = "false")
  cat("check-dir-path=", file.path(getwd(), ("check")), "\n", file = Sys.getenv("GITHUB_OUTPUT"), sep = "", append = TRUE)
  check_results <- rcmdcheck::rcmdcheck(args = (c("--no-manual", "--as-cran")), build_args = ("--no-manual"), error_on = ("warning"), check_dir = ("check"))
  shell: C:\R\bin\Rscript.EXE {0}
  env:
    R_KEEP_PKG_SOURCE: yes
    ERROR_ON_DEFAULT: "warning"
    _R_INSTALL_TIME_PATCHES_: no
    R_LIBS_USER: D:\a\_temp\Library
    TZ: UTC
    _R_CHECK_SYSTEM_CLOCK_: FALSE
    NOT_CRAN: true
    R_LIB_FOR_PAK: C:/R/site-library
── R CMD build ─────────────────────────────────────────────────────────────────
* checking for file 'D:\a\mighty.metadata\mighty.metadata/DESCRIPTION' ... OK
* preparing 'mighty.metadata':
* checking DESCRIPTION meta-information ... OK
* installing the package to build vignettes
* creating vignettes ... OK
* checking for LF line-endings in source and make files and shell scripts
* checking for empty or unneeded directories
* building 'mighty.metadata_0.0.0.9002.tar.gz'

── R CMD check ─────────────────────────────────────────────────────────────────
* using log directory 'D:/a/mighty.metadata/mighty.metadata/check/mighty.metadata.Rcheck'
* using R version 4.3.1 (2023-06-16 ucrt)
* using platform: x86_64-w64-mingw32 (64-bit)
* R was compiled by
    gcc.exe (GCC) 12.2.0
    GNU Fortran (GCC) 12.2.0
* running under: Windows Server 2022 x64 (build 26100)
* using session charset: UTF-8
* using options '--no-manual --as-cran'
* checking for file 'mighty.metadata/DESCRIPTION' ... OK
* this is package 'mighty.metadata' version '0.0.0.9002'
* package encoding: UTF-8
* checking package namespace information ... OK
* checking package dependencies ...Warning in url(sprintf("%s/%s", cran, path), open = "rb") :
  cannot open URL 'https://packagemanager.posit.co/cran/2023-10-25/web/packages/packages.rds': HTTP status was '404 Not Found'
Error in url(sprintf("%s/%s", cran, path), open = "rb") : 
  cannot open the connection to 'https://packagemanager.posit.co/cran/2023-10-25/web/packages/packages.rds'
Execution halted
```

</details>

### Pipeline 2 - windows latest

This one is from other package, but again error is not being propagated. Also, tests are failing (I haven't posted that part, because log is too long, but there are 36 tests that have failed), but `rcmdcheck` doesn't catch any of it:

<details>
     <summary> Logs </summary>
 ## --------------------------------------------------------------------
  options(crayon.enabled = TRUE)
  cat("LOGNAME=", Sys.info()[["user"]], "\n", sep = "", file = Sys.getenv("GITHUB_ENV"), append = TRUE)
  if (Sys.getenv("_R_CHECK_FORCE_SUGGESTS_", "") == "") Sys.setenv("_R_CHECK_FORCE_SUGGESTS_" = "false")
  if (Sys.getenv("_R_CHECK_CRAN_INCOMING_", "") == "") Sys.setenv("_R_CHECK_CRAN_INCOMING_" = "false")
  cat("check-dir-path=", file.path(getwd(), ("check")), "\n", file = Sys.getenv("GITHUB_OUTPUT"), sep = "", append = TRUE)
  check_results <- rcmdcheck::rcmdcheck(args = (c("--no-manual", "--as-cran")), build_args = ("--no-manual"), error_on = ("warning"), check_dir = ("check"))
  shell: C:\R\bin\Rscript.EXE {0}
  env:
    R_KEEP_PKG_SOURCE: true
    ERROR_ON_DEFAULT: "warning"
    _R_INSTALL_TIME_PATCHES_: no
    R_LIBS_USER: D:\a\_temp\Library
    TZ: UTC
    _R_CHECK_SYSTEM_CLOCK_: FALSE
    NOT_CRAN: true
    RSPM: https://packagemanager.posit.co/cran/latest
    RENV_CONFIG_REPOS_OVERRIDE: https://packagemanager.posit.co/cran/latest
    R_LIB_FOR_PAK: C:/R/site-library
    BUNDLE_EXT: win.msi
── R CMD build ──────���──────────────────────────────────────────────────────────
* checking for file 'D:\a\mighty\mighty/DESCRIPTION' ... OK
* preparing 'mighty':
* checking DESCRIPTION meta-information ... OK
* installing the package to build vignettes
* creating vignettes ... OK
* checking for LF line-endings in source and make files and shell scripts
* checking for empty or unneeded directories
* building 'mighty_0.0.0.9001.tar.gz'

── R CMD check ─────────────────────────────────────────────────────────────────
* using log directory 'D:/a/mighty/mighty/check/mighty.Rcheck'
* using R version 4.5.2 (2025-10-31 ucrt)
* using platform: x86_64-w64-mingw32
* R was compiled by
    gcc.exe (GCC) 14.3.0
    GNU Fortran (GCC) 14.3.0
* running under: Windows Server 2022 x64 (build 26100)
* using session charset: UTF-8
* using options '--no-manual --as-cran'
* checking for file 'mighty/DESCRIPTION' ... OK
* this is package 'mighty' version '0.0.0.9001'
* package encoding: UTF-8
* checking package namespace information ... OK
* checking package dependencies ... OK
* checking if this is a source package ... OK
* checking if there is a namespace ... OK
* checking for executable files ... OK
* checking for hidden files and directories ... NOTE
Found the following hidden files and directories:
  .jscpd.json
These were most likely included in error. See section 'Package
structure' in the 'Writing R Extensions' manual.
* checking for portable file names ... OK
* checking whether package 'mighty' can be installed ... OK
* checking installed package size ... OK
* checking package directory ... OK
* checking for future file timestamps ... OK
* checking 'build' directory ... OK
* checking DESCRIPTION meta-information ... OK
* checking top-level files ... NOTE
Non-standard files/directories found at top level:
  'draft.md' 'tmp.md'
* checking for left-over files ... OK
* checking index information ... OK
* checking package subdirectories ... OK
* checking code files for non-ASCII characters ... OK
* checking R files for syntax errors ... OK
* checking whether the package can be loaded ... OK
* checking whether the package can be loaded with stated dependencies ... OK
* checking whether the package can be unloaded cleanly ... OK
* checking whether the namespace can be loaded with stated dependencies ... OK
* checking whether the namespace can be unloaded cleanly ... OK
* checking loading without being on the library search path ... OK
* checking dependencies in R code ... OK
* checking S3 generic/method consistency ... OK
* checking replacement functions ... OK
* checking foreign function calls ... OK
* checking R code for possible problems ... OK
* checking Rd files ... OK
* checking Rd metadata ... OK
* checking Rd line widths ... OK
* checking Rd cross-references ... OK
* checking for missing documentation entries ... OK
* checking for code/documentation mismatches ... OK
* checking Rd \usage sections ... OK
* checking Rd contents ... OK
* checking for unstated dependencies in examples ... OK
* checking installed files from 'inst/doc' ... OK
* checking files in 'vignettes' ... OK
* checking examples ... OK
* checking for unstated dependencies in 'tests' ... OK
* checking tests ...
  Running 'testthat.R' [44s]
 [44s] OK
* checking for unstated dependencies in vignettes ... OK
* checking package vignettes ... OK
* checking re-building of vignette outputs ... [24s] OK
* checking for non-standard things in the check directory ... OK
* checking for detritus in the temp directory ... OK
* DONE
Status: 2 NOTEs
See
  'D:/a/mighty/mighty/check/mighty.Rcheck/00check.log'
for details.
ERROR: Unknown command "TMPDIR=C:/Users/runneradmin/AppData/Local/Temp/RtmpOU6FLD/file216c2f2b532c". Did you mean command "uninstall"?
Warning message:
In system2("quarto", "-V", stdout = TRUE, env = paste0("TMPDIR=",  :
  running command '"quarto" TMPDIR=C:/Users/runneradmin/AppData/Local/Temp/RtmpOU6FLD/file216c2f2b532c -V' had status 1

</details>



## Expected behavior
I would expect both of these pipelines to fail.

## Additional context
We've also used different versions of R (4.3.1 and 4.4.1) with snapshot dates to check exact versions on that date. I'm aware that some of these error happen because of version mismatch, but none of those should affect failure of the pipeline.

Also not sure if this is a question for this repo or `rcmdcheck`


Provide feedback

Saved searches

Use saved searches to filter your results more quickly

windows-latest and macos-latest pipelines not failing with `rcmdcheck` #1038

Describe the bug

To Reproduce

Pipeline 1 - windows-latest and macos-latest

Pipeline 2 - windows latest

Expected behavior

Additional context

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

windows-latest and macos-latest pipelines not failing with rcmdcheck #1038

Description

Describe the bug

To Reproduce

Pipeline 1 - windows-latest and macos-latest

Pipeline 2 - windows latest

Expected behavior

Additional context

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

windows-latest and macos-latest pipelines not failing with `rcmdcheck` #1038