Compile separately pruned clustering trees and standardize thresholds

This function is designed for the use of CHOIR on atlas-scale data. It will take the output from buildParentTree, and the output from subsequent applications of CHOIR to each resulting subtree, and combine these results into a single compiled tree, which is then further pruned to standardize thresholds across each subtree.

Usage

combineTrees(
  object,
  subtree_list,
  key = "CHOIR",
  alpha = NULL,
  p_adjust = NULL,
  feature_set = NULL,
  exclude_features = NULL,
  n_iterations = NULL,
  n_trees = NULL,
  use_variance = NULL,
  min_accuracy = NULL,
  min_connections = NULL,
  max_repeat_errors = NULL,
  distance_approx = NULL,
  distance_awareness = NULL,
  collect_all_metrics = NULL,
  sample_max = NULL,
  downsampling_rate = NULL,
  min_reads = NULL,
  normalization_method = NULL,
  batch_correction_method = NULL,
  batch_labels = NULL,
  use_assay = NULL,
  use_slot = NULL,
  ArchR_matrix = NULL,
  countsplit = NULL,
  countsplit_suffix = NULL,
  input_matrix = NULL,
  nn_matrix = NULL,
  dist_matrix = NULL,
  reduction = NULL,
  n_cores = NULL,
  random_seed = NULL,
  verbose = TRUE
)

Arguments

object: An object of class Seurat, SingleCellExperiment, or ArchRProject that was output from function buildParentTree. For multi-omic data, we recommend using ArchRProject objects.
subtree_list: A list containing the CHOIR records from each subtree.
key: The name under which CHOIR-related data for this run is stored in the object. Defaults to “CHOIR”.
alpha: A numerical value indicating the significance level used for permutation test comparisons of cluster distinguishability. Defaults to 0.05. Decreasing the alpha value will yield more conservative clusters (fewer clusters) and will often decrease the computational time required, because fewer cluster comparisons may be needed.
p_adjust: A string indicating which multiple comparison adjustment method to use. Permitted values are “bonferroni”, “fdr”, and “none”. Defaults to “bonferroni”. Other correction methods may be less conservative, identifying more clusters, as CHOIR applies filters that reduce the total number of tests performed.
feature_set: A string indicating whether to train random forest classifiers on “all” features or only variable (“var”) features. Defaults to “var”. Computational time and memory required may increase if more features are used. Using all features instead of variable features may result in more conservative cluster calls.
exclude_features: A character vector indicating features that should be excluded from input to the random forest classifier. Defaults to NULL, which means that no features will be excluded. This parameter can be used, for example, to exclude features correlated with cell quality, such as mitochondrial genes. Failure to exclude problematic features could result in clusters driven by cell quality, while over-exclusion of features could reduce the ability of CHOIR to distinguish cell populations that differ by those features.
n_iterations: A numerical value indicating the number of iterations run for each permutation test comparison. Increasing the number of iterations will approximately linearly increase the computational time required but provide a more accurate estimation of the significance of the permutation test. Decreasing the number of iterations runs the risk of leading to underclustering due to lack of statistical power. The default value, 100 iterations, was selected because it avoids underclustering, while minimizing computational time and the diminishing returns from running CHOIR with additional iterations.
n_trees: A numerical value indicating the number of trees in each random forest. Defaults to 50. Increasing the number of trees is likely to increase the computational time required. Though not entirely predictable, increasing the number of trees up to a point may enable more nuanced distinctions, but is likely to provide diminishing returns.
use_variance: A Boolean value indicating whether to use the variance of the random forest accuracy scores as part of the permutation test threshold. Defaults to TRUE. Setting this parameter to FALSE will make CHOIR considerably less conservative, identifying more clusters, particularly on large datasets.
min_accuracy: A numerical value indicating the minimum accuracy required of the random forest classifier, below which clusters will be automatically merged. Defaults to 0.5, representing the random chance probability of assigning correct cluster labels; therefore, decreasing the minimum accuracy is not recommended. Increasing the minimum accuracy will lead to more conservative cluster assignments and will often decrease the computational time required, because fewer cluster comparisons may be needed.
min_connections: A numerical value indicating the minimum number of nearest neighbors between two clusters for those clusters to be considered adjacent. Non-adjacent clusters will not be merged. Defaults to 1. This threshold allows CHOIR to avoid running the full permutation test comparison for clusters that are highly likely to be distinct, saving computational time. Therefore, setting this parameter to 0 will increase the number of permutation test comparisons run and, thus, the computational time. The intent of this parameter is only to avoid running permutation test comparisons between clusters that are so different that they should not be merged. Therefore, we do not recommend increasing this parameter value beyond 10, as higher values may result in instances of overclustering.
max_repeat_errors: A numerical value indicating the maximum number of repeatedly mislabeled cells that will be taken into account during the permutation tests. This parameter is used to account for situations in which random forest classifier errors are concentrated among a few cells that are repeatedly misassigned. If set to 0, such repeat errors will not be evaluated. Defaults to 20. These situations are relatively infrequent, but setting this parameter to lower values (especially 0) may result in underclustering due to a small number of intermediate cells. Setting this parameter to higher values may lead to instances of overclustering and is not recommended.
distance_approx: A Boolean value indicating whether or not to use approximate distance calculations. Defaults to TRUE, which will use centroid-based distances. Setting distance approximation to FALSE will substantially increase the computational time and memory required, particularly for large datasets. Using approximated distances (TRUE) rather than absolute distances (FALSE) is unlikely to have a meaningful effect on the distance thresholds imposed by CHOIR.
distance_awareness: A numerical value representing the distance threshold above which a cluster will not merge with another cluster and significance testing will not be used. Specifically, this value is a multiplier applied to the distance between a cluster and its closest distinguishable neighbor based on random forest comparison. Defaults to 2, which sets this threshold at a two-fold increase in distance over the closest distinguishable neighbor. This threshold allows CHOIR to avoid running the full permutation test comparison for clusters that are highly likely to be distinct, saving computational time. To omit all distance calculations and perform permutation testing on all comparisons, set this parameter to FALSE. Setting this parameter to FALSE or increasing the input value will increase the number of permutation test comparisons run and, thus, the computational time. In rare cases, very small distant clusters may be erroneously merged when distance thresholds are not used. The intent of this parameter is only to avoid running permutation test comparisons between clusters that are so different that they should not be merged. We do not recommend decreasing this parameter value below 1.5, as lower values may result in instances of overclustering.
collect_all_metrics: A Boolean value indicating whether to collect and save additional metrics from the random forest classifiers, including feature importances for every comparison. Defaults to FALSE. Setting this parameter to TRUE will slightly increase the computational time required. This parameter has no effect on the final cluster calls.
sample_max: A numerical value indicating the maximum number of cells to be sampled per cluster to train/test each random forest classifier. Defaults to Inf (infinity), which does not cap the number of cells used, so all cells will be used in all comparisons. Decreasing this parameter may decrease the computational time required, but may result in instances of underclustering. If input is provided to both the downsampling_rate and sample_max parameters, the minimum resulting cell number is calculated and used for each comparison.
downsampling_rate: A numerical value indicating the proportion of cells to be sampled per cluster to train/test each random forest classifier. For efficiency, the default value, "auto", sets the downsampling rate according to the dataset size. Decreasing this parameter may decrease the computational time required, but may also make the final cluster calls more conservative. If input is provided to both downsampling_rate and sample_max parameters, the minimum resulting cell number is calculated and used for each comparison.
min_reads: A numeric value used to filter out features prior to input to the random forest classifier. The default value, NULL, will filter out features with 0 counts for the current clusters being compared. Higher values should be used with caution, but may increase the signal-to-noise ratio encountered by the random forest classifiers.
normalization_method: A character string or vector indicating which normalization method to use. In general, input data should be supplied to CHOIR after normalization, except when the user wishes to use Seurat SCTransform normalization. Permitted values are “none” or “SCTransform”. Defaults to “none”. Because CHOIR has not been tested thoroughly with SCTransform normalization, we do not recommend this approach at this time. For multi-omic datasets, provide a vector with a value corresponding to each provided value of use_assay or ArchR_matrix in the same order.
batch_correction_method: A character string indicating which batch correction method to use. Permitted values are “Harmony” and “none”. Defaults to “none”. Batch correction should only be used when the different batches are not expected to also have unique cell types or cell states. Using batch correction would ensure that clusters do not originate from a single batch, thereby making the final cluster calls more conservative.
batch_labels: A character string that, if applying batch correction, specifies the name of the column in the input object metadata containing the batch labels. Defaults to NULL.
use_assay: For Seurat or SingleCellExperiment objects, a character string or vector indicating the assay(s) to use in the provided object. The default value, NULL, will choose the current active assay for Seurat objects and the logcounts assay for SingleCellExperiment objects.
use_slot: For Seurat objects, a character string or vector indicating the layers(s)—previously known as slot(s)—to use in the provided object. The default value, NULL, will choose a layer/slot based on the selected assay. If an assay other than "RNA", "sketch”, "SCT”, or "integrated" is provided, you must specify a value for use_slot. For multi-omic datasets, provide a vector with a value corresponding to each provided value of use_assay in the same order.
ArchR_matrix: For ArchR objects, a character string or vector indicating which matrix or matrices to use in the provided object. The default value, NULL, will use the “GeneScoreMatrix” for ATAC-seq data or the “GeneExpressionMatrix” for RNA-seq data. For multi-omic datasets, provide a vector with a value corresponding to each modality.
countsplit: A Boolean value indicating whether or not to use count split input data (see countsplit package), such that one matrix of counts is used for clustering tree generation and a separate matrix is used for all random forest classifier permutation testing. Defaults to FALSE. Enabling count splitting is likely to result in more conservative final cluster calls and is likely to perform best in datasets with high read depths.
countsplit_suffix: A character vector indicating the suffixes that distinguish the two count split matrices to be used. Suffixes are appended onto the input string/vector for parameter use_slot for Seurat objects, use_assay for SingleCellExperiment objects, or ArchR_matrix for ArchR objects. When count splitting is enabled, the default value NULL uses suffixes "_1" and "_2".
input_matrix: An optional matrix containing the feature x cell data provided by the user, on which to train the random forest classifiers. By default, this parameter is set to NULL, and CHOIR will look for the feature x cell matri(ces) indicated by function buildParentTree.
nn_matrix: An optional matrix containing the nearest neighbor adjacency of the cells, provided by the user. By default, this parameter is set to NULL, and CHOIR will look for the adjacency matri(ces) generated by function buildParentTree.
dist_matrix: An optional distance matrix of cell to cell distances (based on dimensionality reduction cell embeddings), provided by the user. By default, this parameter is set to NULL, and CHOIR will look for the distance matri(ces) generated by function buildParentTree.
reduction: An optional matrix of dimensionality reduction cell embeddings provided by the user for subsequent clustering steps. By default, this parameter is set to NULL, and CHOIR will look for the dimensionality reductions generated by function buildParentTree().
n_cores: A numerical value indicating the number of cores to use for parallelization. By default, CHOIR will use the number of available cores minus 2. CHOIR is parallelized at the computation of permutation test iterations. Therefore, any number of cores up to the number of iterations will theoretically decrease the computational time required. In practice, 8–16 cores are recommended for datasets up to 500,000 cells.
random_seed: A numerical value indicating the random seed to be used. Defaults to 1. CHOIR uses randomization throughout the generation and pruning of the clustering tree. Therefore, changing the random seed may yield slight differences in the final cluster assignments.
verbose: A Boolean value indicating whether to use verbose output during the execution of CHOIR. Defaults to TRUE, but can be set to FALSE for a cleaner output.

Value

Returns the object with the following added data stored under the provided key:

clusters: Final clusters, full hierarchical cluster tree, and stepwise cluster results for each progressive pruning step
parameters: Record of parameter values used
records: Metadata for decision points during hierarchical tree construction, all recorded permutation test comparisons, and feature importance scores from all comparisons