Skip to main content

Bioconductor Packages: Development, Maintenance, and Peer Review

23 Troubleshooting Build Report

23.1 How and when does the BBS pull? When will my changes propagate?

Please remember the daily builder pulls, installs, builds, and checks package only once per day. This process starts around 2:30 PM (14:30) EST everyday (i.e., UTC−05:00). Changes pushed to Bioconductor before 2:30 will be reflected in the following day’s build report that is posted around 11:30 AM EST. The build report has a time stamp at the top of the page when it was generated. Changes after 2:30 PM EST will not be reflect until the day after tomorrow, therefore possibly taking up to 36-48 hours. The build reports for devel and release show the package version and commit id that is being reflected for that build. The landing pages for packages (e.g., Biobase) will not be updated until the package installs/build/checks without ERROR; We do not propagate broken packages. This could account for a different version on the landing page than was pushed to Bioconductor. Please also remember a package ALWAYS needs a valid version bump to propagate to users.

The schedule for software, data experiment, annotation, books, workflows can be found on the check results page. In order for changes to be included in the report, changes should be committed to the Bioconductor git.bioconductor.org location the morning before the listed day with a valid version bump.

23.2 How do I reproduce the build system ERROR?

In order to reproduce the ERROR’s accurately locally, remember to be using the correct version of R and Bioconductor. The version of R used for the build report can be found at the top of the release and devel build reports. Once you are using the correct version of R make sure all your packages are up-to-date with BiocManager::valid() and BiocManager::install(). There are some additional environment variables the daily builder uses during R CMD check.

The Single Package build has some extra documentation about how to set up your local system to use optional environment variables. Please note that if you look at the file listed on this page, it has many additional variables; Bioconductor does a much more stringent check on incoming packages than on the daily builder (for now). You are welcome to use this file if you wish as it is a more comprehensive check but the above listed environment variables should be included minimally.

Another option to debug and test is to utilize the Bioconductor docker image. The documentation for using docker images can be found here. The docker image does contain the environment variable setting found on the daily builder.

23.3 Common Build Report Errors

Often common Error’s will arise as R develops and matures or as Bioconductor packages are modified and advance. This document provides some guidance on Error’s and potential solutions.

23.4 Bioconductor 3.17 with R 4.3

23.4.1 Length = coercion to logical(1)

In R 4.3, a warning for a vector used in an if statement has increased to an ERROR. So this:

 x = c(TRUE,TRUE, FALSE)
 > if (x) 1
 [1] 1
 Warning message:
 In if (x) 1 :
   the condition has length > 1 and only the first element will be used

Will now be:

 > x = c(TRUE, TRUE, FALSE)
 > if (x) 1
 Error in if (x) 1 : the condition has length > 1

Solution: In most cases this is a misjudgment in the length of the argument rather than intentional use of checking if the value was assigned. The code should be reviewed to see if argument is being assigned correctly. In most cases the simple solution might be to use an any( ) or all( ) surrounding the vector.

23.5 Bioconductor 3.11 with R 4.0

R switched from 3.x to 4.0 which generally means some significant changes.

23.5.1 S3 method registration

Many packages are currently failing because of undeclared S3 methods in the NAMESPACE. There is some background information found on the R developers blog post: S3 Method Lookup

This ERROR takes many different forms on the build report. Some of the more common forms include

  • Cannot coerce class <structure> to a data.frame
  • Cannot coerce type ‘S4’ to vector of type ‘double’
  • No applicable method for <foo> applied to an object of class <bar>
  • ‘X’ is a list, but does not have components ‘x’ and ‘y’
  • Error in colMeans(x, na.rm = TRUE) : ‘x’ must be numeric
  • Error in RG\[1:2, \] : incorrect number of dimensions
  • formal argument <foo> matched by multiple actual arguments
  • object <foo> of mode ‘function’ was not found
  • ‘x’ and ‘y’ lengths differ

Solution: Register the S3 method in the NAMESPACE

S3method(<function>, <dispatch>)

A simple example which effects many packages is a S3 plotting method. The following line would be added to the package NAMESPACE.

S3method(plot, TCC)  # example from TCC package

23.5.2 Removed Setting in R CMD config

The source of the ERROR is utilizing settings in package configure script that have been removed or replaced. There is a section of R NEWS “R CMD config no longer knows about the unused settings F77 and FCPIFCPLAGS, nor CXX98 and similar.” Executing the configuration script when installing the package fails, and the output contains lot of messages along the lines of the following:

  • configure: WARNING: The flags FFLAGS=“” do not work
  • checking whether the ERROR: no information for variable ‘F77’
  • configure: WARNING: This value for FFLAGS does not work.

Solution: Replace instances of “${R} CMD config F77” with “${R} CMD config FC”

23.5.3 Conditional Length > 1

In R 4.0 a conditional with a length greater than 1 will produce a WARNING. On the Bioconductor Daily Builder and Single Package Builder this is increased to an ERROR.

Traditionally if / while statements could accept vectors using the first element as the conditional value and ignoring the remaining values. This now produces a WARNING as seem in this dummy example and documented at Conditions of Length Greater Than One

> if (c(TRUE, FALSE)) {}
NULL
Warning message:
In if (c(TRUE, FALSE)) { :
    the condition has length > 1 and only the first element will be used

Solution: Bioconductor increased the severity as in most cases this is a misjudgment in the length of the argument rather than intentional. The code should be reviewed to see if argument is being assigned correctly. In most cases it might be appropriate to use an any( ) or all( ) surrounding the vector.

See also mailing list post

23.5.4 Scalar / Vector Logic

This is not a change in R yet but we have been notified that it is forth coming and have escalated to an ERROR on our daily builders in preparation. This type of ERROR occurs with the misuse of && and ||. The double && and || imply a scalar comparison rather than a vector comparison that the singular & and | expect. See the dummy example below:

> c(TRUE, TRUE) && TRUE
Error in c(TRUE, TRUE) && TRUE :
   'length(x) = 2 > 1' in coercion to 'logical(1)'

Solution: Most cases are misjudgment and misunderstanding of the use of a scalar comparison from a vector comparison. Changing the double && / || to a singular & / | will generally be sufficient if a vector comparison is intended or having the vector argument use an appropriate any( ) or all( ) surrounding the vector will result in the appropriate scalar comparison. Note: If this comparison is in a conditional please see the section above; any( ) or all( ) will most likely be a better alternative.

See also mailing list post

23.5.5 Class == vs is/inherits

While this isn’t a change in R / Bioconductor as of yet, there is strong discussion about the affects and consequences of this code structure. A better discussion and explanation can be found When you think class(.) == *, think again!

The sum up is class( x ) == "foo" should be avoided. It can be misleading if classes extend other classes. The better option is to use is( x , "foo") or inherits(x, "foo").

This is also advised in Bioconductor best practices

Starting in R 4.0, a matrix is considered an extension of array.

> m = matrix()
> class(m)
[1] "matrix" "array"

> class(m) == "matrix"
[1]  TRUE FALSE
> if ( class(m) == "matrix"){}
Error in if (class(m) == "matrix") { : the condition has length > 1

This change along with the previous section regarding conditional length results in many errors where users were doing something along the lines of if (class(m) == "matrix"); This is an excellent example where the following is the appropriate change if(is(m, "matrix")) or if(inherits(m, "matrix")) or if(is.matrix(m)).

Another common ERROR now occurring because of this change is something similar to the following:

Error in vapply(experiments(object), class, character(1)) :
  values must be length 1,
 but FUN(X[[4]]) result is length 2

23.5.6 data.frame stringsAsFactors

In R 4.0, the default for data.frame argument stringsAsFactors changed from TRUE to FALSE. This changes is causing the most breakage in tests where there are checks for particular factor levels or constructing factor levels. The ERROR’s take many different forms. The simple solution is to change or add the stringAsFactors=TRUE to the data.frame call, however maintainers may want to re-evaluate code for potential restructuring or ease of use.

23.5.7 stats::smoothEnds

A recent change to stats::smoothEnds(), now returns an integer vector with the input is an integer vector. Previously it could return a number vector.

Example R 3.6.3

> class(smoothEnds(c(401:403)))
[1] "integer"
> class(smoothEnds(c(401:403, 555L)))
[1] "numeric"

Example from 4.0.0

> class(smoothEnds(c(401:403)))
[1] "integer"
> class(smoothEnds(c(401:403, 555L)))
[1] "integer"

This has the potential to cause ERROR’s if a class type was checked.

23.5.8 Grid package changes

We do not have a lot of specifics on what has changed but were notified by email. Important sections of the email as follows:

    I am about to commit some internal changes to 'grid' units
        (for, in some cases, 100x speed-up of unit operations).
        A number of packages have already been fixed to work with these
        changes, but, according to my testing, the following CRAN
        packages will still fail R CMD check.
    Some of those are cascades ('armada', 'countToFPKM', and 'wilson'
        from 'ComplexHeatmap' - see below - and 'fingertipscharts' from 'lemon'
        and 'xpose' is actually a 'ggforce' problem), but all of the other
        package authors have been notified and several are already working on
        fixes.
        The most serious of those is 'ComplexHeatmap' because it causes multiple
        follow-on failures, the CRAN ones above and others on BioConductor:
    Again, the main package authors have been notified and the
    'ComplexHeatmap' author is working on an update.

23.5.9 plot generic moved

The plot generic has moved from graphics to base. The ERROR’s seen from this change are non specific and can take many forms. Some of the ERROR’s we see are

Error in getGeneric(f, TRUE, envir, package) :
 no generic function found for 'plot'

or

Error in as.double(y) :
      cannot coerce type 'S4' to vector of type 'double'

The explanation given:

  “The namespace controls the search strategy for variables used by
  functions in the package. If not found locally, R searches the package
  namespace first, then the imports, then the base namespace and then  the
  normal search path." as per
  https://cran.r-project.org/doc/manuals/r-devel/R-exts.html#Package-namespaces):

  CRAN and Bioconductor  had a few packages that "worked" because the right
  plot() was found in the normal search path, but now fail because it's
  calling the one in base instead.

23.5.10 Partial Argument Matching

There is now more strict checking of argument matching with regards to partial argument matching. Best described with the following example

setGeneric(“mycoolfunction”,   function(object,  breaks)
    standardGeneric(“mycoolfunction”)
setMethod(“mycoolfunction”,
    signature=c(object=”GRanges”, break=”GRanges”),
    <code>)

Notice the generic uses breaks while the setMethod uses break; This is an example of a partial argument match that will no longer be valid.

Partial argument matching when envoking functions should also be avoided as part of best practices. For example

mycoolfunction <- function( x, myargum, secondarg ) { code }

mycoolfunction(x=2, myar=1:2, second=3)          # BAD Coding!

mycoolfunction(x=2, myargum=1:2, secondarg=3)    # Good Practice!

23.5.11 Package inputenc Error: Invalid UTF-8

This ERROR started to appear on tokay2 (windows) in Spring 2020. We are not sure the exact source of the ERROR (change in MiKTek, Change in R, other?) but the solution is simple:

Please place \usepackage[utf8]{inputenc} in the beginning of your Sweave vignette right after the \documentclass line.

23.5.12 Dependency Issues

Dependency Issues can fall into a few sub-categories:

23.5.12.1 CRAN binaries not available for R 4.0

The fall cycle of Bioconductor uses R-devel in preparation for the new release of R in the spring. This is always a slightly more disruptive cycle with regards to package dependencies from CRAN. CRAN over the next 6 months leading up to the new release of R will make binaries for Windows and MacOS available. As they become available the Bioconductor builders will automatically add these binaries. If the binaries have not been created yet, they will be unavailable and result in a package not available error. Bioconductor will not go to extra efforts to find work around to install these packages; when they are available, they will be added. Solution: Please be patient!

23.5.12.2 Package have been removed from CRAN

CRAN packages are occasionally removed. Unfortunately, Bioconductor will only allow package dependencies to be actively maintained packages on CRAN or Bioconductor. A package will have to alter their package to not utilize code and not rely on this dependency. You may of course try to petition CRAN for reinstatement or reach out to the package maintainer to fix and submit to CRAN. Good Luck!

23.5.12.3 Package has been removed from Bioconductor

We try to be more aware of orphaned packages and packages that remain broken for extended periods of time. Package deprecation and removal occurs and packages will have to alter to not utilize code from these packages or could potential offer to take over maintenance of broken packages but that would require original maintainers permission. Bioconductor Package deprecation is announced throughout the release cycle on the mailing list and support site to try and allow dependent packages time to adjust code before removal. This release the most notable maintainer requested deprecation from 3.10 (therefore removed in 3.11) are SNPchip and GenomeGraphs. A full list of deprecated packages can be found List of Deprecated Packages 3.10. We also documented removed packages on our Removed Package Page

23.5.12.4 System Requirement missing in the Bioconductor Build System

Your package may fail to build or check in the Bioconductor Build System (BBS) because your package or some of its dependencies have a System Requirement not available in the BBS.

For instance, the first Bioconductor package that suggested the installation of the archive CRAN package, could not pass the checks at the BBS. The build report stated that “the archive package could not be found”, however the package had been present in CRAN for months. The archive package reports among its system requirements the libarchive library, and the BBS did not have that library installed.

If you happen to miss a system dependency in the BBS, please open an issue similar to BBS issue #220, where you mention which is the Bioconductor package that either directly or indirectly has a system requirement, and if possible provide information on the missing package that would need to be installed. If you follow the pull requests related to that issue, you will see that the solution was the installation of the libarchive-dev package, both in the BBS and in the Bioconductor docker images.

23.5.13 Deprecated Functions

Functions can be deprecated, defunct, and eventually removed. Bioconductor tries to enforce this progression to allow maintainers to adjust code. Most deprecated or defunct functions will (should) suggest the alternative. The following are noted in Bioconductor 3.11

23.5.13.1 RangedData 

Error : RangedData objects are defunct. Please migrate your code to use GRanges
  or GRangesList objects instead. See IMPORTANT NOTE in ?RangedData

23.5.13.2 Normalize 

Error:
'normalize' is defunct.
Use ''normalize,SingleCellExperiment-method' is defunct.
Use 'logNormCounts' instead' instead.

23.5.13.3 calculateQCMetrics 

Error:
'calculateQCMetrics' is defunct.
Use 'perCellQCMetrics' instead.