PJC - Computational Biology

Honeybadger_karaayvaz.nb

2020-11-18T00:00:00+00:00

HoneyBadger on Karaayvaz et al 2018

Loading libraries


library(HoneyBADGER)
library(biomaRt)
library(pheatmap)
library(uwot)
library(ggfortify)

Loading required package: ggplot2

Loading data

Reading data from GEO.

exp_norm<- read.table("data/RSEM_TPM_240_NormalCells.txt")
exp_cancer<- read.table("data/GSE118389_tpm_rsem.txt")
meta_celltypes<- read.table("data/cell_types_tab_S9.txt", stringsAsFactors = FALSE, row.names=2)
meta_celltypes[,1]<- NULL
dim(meta_celltypes)

[1] 1112    1

names(meta_celltypes)<- "cell"
length(intersect(rownames(meta_celltypes), colnames(exp_cancer)))

[1] 1112

Taking a look at the data

dim(exp_norm)

[1] 23368   240

head(exp_norm[,1:10])

dim(exp_cancer)

[1] 21785  1534

head(exp_cancer[,1:10])

exp_cancer<- exp_cancer[, rownames(meta_celltypes)]

hist(colSums(exp_cancer))

hist(colSums(exp_norm))

Filtering for genes

In this case we are going to use a TPM filter of 10 for both the reference and test datasets.

mean_gene_norm<-rowMeans(exp_norm)
summary(mean_gene_norm)

    Min.  1st Qu.   Median     Mean  3rd Qu.     Max. 
    0.00     0.01     1.34    30.80    14.81 45114.25

length(which(mean_gene_norm> 10))

[1] 7221

exp_norm<- exp_norm[mean_gene_norm>10,]

mean_gene_cancer<- rowMeans(exp_cancer)
summary(mean_gene_cancer)

     Min.   1st Qu.    Median      Mean   3rd Qu.      Max. 
    0.000     0.375     5.749    44.176    25.944 21644.406

length(which(mean_gene_cancer> 10))

[1] 9041

exp_cancer<- exp_cancer[mean_gene_cancer>10,]

inter_genes<- intersect(rownames(exp_cancer), rownames(exp_norm))
length(inter_genes)

[1] 5560

exp_cancer<- exp_cancer[inter_genes,]
exp_norm <- exp_norm[inter_genes,]

Generating UMAP of the data

set.seed(43)
sce_cells_umap <- umap(t(exp_cancer),                       
                       n_components = 2,
                       #a=1,
                       #b=0.6,
                       min_dist=0.01,
                       metric="cosine",
                       verbose = TRUE,
                       n_threads = max(1, RcppParallel::defaultNumThreads()/2) )

11:54:53 UMAP embedding parameters a = 1.896 b = 0.8006
11:54:53 Read 1112 rows and found 5560 numeric columns
11:54:53 Using Annoy for neighbor search, n_neighbors = 15
11:54:53 Building Annoy index with metric = cosine, n_trees = 50
0%   10   20   30   40   50   60   70   80   90   100%
[----|----|----|----|----|----|----|----|----|----|
**************************************************|
11:54:54 Writing NN index file to temp file /tmp/Rtmp4veicN/filef10477e73ea
11:54:54 Searching Annoy index using 4 threads, search_k = 1500
11:54:56 Annoy recall = 100%
11:54:56 Commencing smooth kNN distance calibration using 4 threads
11:54:57 Initializing from normalized Laplacian + noise
11:54:57 Commencing optimization for 500 epochs, with 24190 positive edges
0%   10   20   30   40   50   60   70   80   90   100%
[----|----|----|----|----|----|----|----|----|----|
**************************************************|
11:54:59 Optimization finished

colnames(sce_cells_umap)<- c("UMAP1", "UMAP2")
rownames(sce_cells_umap)<- colnames(exp_cancer)
sce_cells_umap<- cbind(sce_cells_umap, meta)

head(sce_cells_umap)

NA

ggplot(sce_cells_umap, aes(UMAP1, UMAP2, color=cell) )+
  geom_point()

Creating meta data

meta<- data.frame( row.names=colnames(exp_cancer), 
                   patient= sapply(colnames(exp_cancer), function(X) unlist( strsplit(X, "_"))[1]),
                   plate= sapply(colnames(exp_cancer), function(X) unlist( strsplit(X, "_"))[2]),
                   well = sapply(colnames(exp_cancer), function(X) unlist( strsplit(X, "_"))[3]),
                   stringsAsFactors=FALSE)
length(intersect(rownames(meta), rownames(meta_celltypes)))

[1] 1112

meta<- cbind(meta, meta_celltypes)
head(meta)

NA

Running HoneyBadger analysis for PT039

mart.obj <- useMart(biomart = "ensembl", dataset = 'hsapiens_gene_ensembl', host = "feb2014.archive.ensembl.org") ## version used in manuscript

PT039_meta<- meta[meta$patient  == "PT039",]
## Subsetting cells
set.seed(43)
#cells_keep<- sample(rownames(PT039_meta), 100)
cells_keep<- rownames(PT039_meta)
PT039_meta<- PT039_meta[cells_keep,]
head(PT039_meta)

PT039_exp<- exp_cancer[,cells_keep]

Making a new HoneyBadger object

hb <- new('HoneyBADGER', name='PT039')

hb$setGexpMats(log2(as.matrix(PT039_exp)+1), log2(exp_norm+1), mart.obj, filter=FALSE, scale=TRUE, verbose=TRUE)

Initializing expression matrices ... 
Scaling coverage ... 
Normalizing gene expression for 5560 genes and 273 cells ...

`select_()` is deprecated as of dplyr 0.7.0.
Please use `select()` instead.
This warning is displayed once every 8 hours.
Call `lifecycle::last_warnings()` to see where this warning was generated.`filter_()` is deprecated as of dplyr 0.7.0.
Please use `filter()` instead.
See vignette('programming') for more help
This warning is displayed once every 8 hours.
Call `lifecycle::last_warnings()` to see where this warning was generated.
Batch submitting query [================>--------------------------------------------------------------------------------------]  17% eta:  4s
Batch submitting query [=================================>---------------------------------------------------------------------]  33% eta:  3s
Batch submitting query [===================================================>---------------------------------------------------]  50% eta:  2s
Batch submitting query [====================================================================>----------------------------------]  67% eta:  1s
Batch submitting query [=====================================================================================>-----------------]  83% eta:  1s

Done setting initial expression matrices!

hb$plotGexpProfile() ## initial visualization

hb$setMvFit(verbose=TRUE) ## model variance

Modeling expected variance ... Done!

hb$setGexpDev(verbose=TRUE) ## model necessary expression deviation to identify CNVs

Optimal deviance: 0.6496971

hb$calcGexpCnvBoundaries(init=TRUE, verbose=FALSE) ## HMM

Loading required package: rjags
Loading required package: coda
Linked to JAGS 4.3.0
Loaded modules: basemod,bugs

NULL

Double check what CNVs were identified

bgf <- hb$bound.genes.final
genes <- hb$genes
regions.genes <- range(genes[unlist(bgf)])
print(regions.genes)

GRanges object with 3 ranges and 0 metadata columns:
      seqnames             ranges strand
         <Rle>          <IRanges>  <Rle>
  [1]    chr12    861759-33049774      *
  [2]     chr2 75873909-242668893      *
  [3]     chr3   3168600-57307496      *
  -------
  seqinfo: 58 sequences from an unspecified genome; no seqlengths

The initial HMM step identified a number of candidate CNVs to test.

Now, we can use the bayesian model to derive the final posterior probability of each CNV in each cell.

Indeed, our initial HMM has identified a number of candidate CNVs to test. We can now retest all identified CNVs on all cells to derive the final posterior probability of each CNV in each cell.

hb$retestIdentifiedCnvs(retestBoundGenes = TRUE, retestBoundSnps = FALSE, verbose=FALSE)

## look at final results
results <- hb$summarizeResults(geneBased=TRUE, alleleBased=FALSE)
print(head(results[,1:5]))

Cluster cells on these posterior probabilities and visualize them as a heatmap.

trees <- hb$visualizeResults(geneBased=TRUE, alleleBased=FALSE, details=TRUE, margins=c(5,5))

We can again visualize our results, this time, ordering the cells based on their posterior probabilities of harboring CNVs.

## order cells
hc <- trees$hc
order <- hc$labels[hc$order]
## plot all chromosomes
hb$plotGexpProfile(cellOrder=hc$order)

We can compare this result with the one from the paper (Figure 2D,E):

Karaayvaz 2018 2D,E

We observe also a highlighted amplification in chr12, that was later confirmed by WES (Figure 2E)

## plot just identified cnvs
hb$plotGexpProfile(cellOrder=hc$order, region=hb$cnvs[['gene-based']][['amp']])

hb$plotGexpProfile(cellOrder=hc$order, region=hb$cnvs[['gene-based']][['del']])

Finally, we can also visualize the results from HoneyBadger into our UMAPs.

head(results)

prob_res<- data.frame(t(results[,8:ncol(results)]))
colnames(prob_res)<- paste0("prob_", results$seqnames, "_", results$start, "_", results$end)
length(intersect(rownames(prob_res), rownames(sce_cells_umap)))

[1] 273

sce_cells_umap<- cbind(sce_cells_umap,prob_res[rownames(sce_cells_umap),]) 
head(sce_cells_umap)

NA

ggplot(sce_cells_umap, aes(UMAP1, UMAP2, color=cell) )+
  geom_point()

ggplot(sce_cells_umap, aes(UMAP1, UMAP2, color=patient) )+
  geom_point()

sessionInfo()

LS0tCnRpdGxlOiAiSG9uZXlCYWRnZXIgb24gS2FyYWF5dmF6IGV0IGFsIDIwMTgiCmF1dGhvcjogIkFyaWVsIE1hZHJpZ2FsIgpkYXRlOiAnYHIgZm9ybWF0KFN5cy5EYXRlKCksICIlWS0lQi0lZCIpYCcKb3V0cHV0OgogIGh0bWxfbm90ZWJvb2s6CiAgICBkZl9wcmludDogcGFnZWQKICAgIGNvZGVfZm9sZGluZzogc2hvdwogICAgdG9jOiB5ZXMKICAgIHRvY19mbG9hdDoKICAgICAgY29sbGFwc2VkOiBubwogICAgICBzbW9vdGhfc2Nyb2xsOiBubwogIGh0bWxfZG9jdW1lbnQ6CiAgICB0b2M6IHllcwogICAgZGZfcHJpbnQ6IHBhZ2VkCi0tLQoKIyMgTG9hZGluZyBsaWJyYXJpZXMKCmBgYHtyfQpsaWJyYXJ5KEhvbmV5QkFER0VSKQpsaWJyYXJ5KGJpb21hUnQpCmxpYnJhcnkocGhlYXRtYXApCmxpYnJhcnkodXdvdCkKbGlicmFyeShnZ2ZvcnRpZnkpCmBgYAoKIyMgTG9hZGluZyBkYXRhCgpSZWFkaW5nIGRhdGEgZnJvbSBHRU8uIAoKYGBge3J9CmV4cF9ub3JtPC0gcmVhZC50YWJsZSgiZGF0YS9SU0VNX1RQTV8yNDBfTm9ybWFsQ2VsbHMudHh0IikKZXhwX2NhbmNlcjwtIHJlYWQudGFibGUoImRhdGEvR1NFMTE4Mzg5X3RwbV9yc2VtLnR4dCIpCm1ldGFfY2VsbHR5cGVzPC0gcmVhZC50YWJsZSgiZGF0YS9jZWxsX3R5cGVzX3RhYl9TOS50eHQiLCBzdHJpbmdzQXNGYWN0b3JzID0gRkFMU0UsIHJvdy5uYW1lcz0yKQptZXRhX2NlbGx0eXBlc1ssMV08LSBOVUxMCmRpbShtZXRhX2NlbGx0eXBlcykKbmFtZXMobWV0YV9jZWxsdHlwZXMpPC0gImNlbGwiCmxlbmd0aChpbnRlcnNlY3Qocm93bmFtZXMobWV0YV9jZWxsdHlwZXMpLCBjb2xuYW1lcyhleHBfY2FuY2VyKSkpCmBgYAoKVGFraW5nIGEgbG9vayBhdCB0aGUgZGF0YQoKYGBge3J9CmRpbShleHBfbm9ybSkKaGVhZChleHBfbm9ybVssMToxMF0pCmBgYAoKYGBge3J9CmRpbShleHBfY2FuY2VyKQpoZWFkKGV4cF9jYW5jZXJbLDE6MTBdKQpgYGAKCmBgYHtyfQpleHBfY2FuY2VyPC0gZXhwX2NhbmNlclssIHJvd25hbWVzKG1ldGFfY2VsbHR5cGVzKV0KYGBgCgoKYGBge3J9Cmhpc3QoY29sU3VtcyhleHBfY2FuY2VyKSkKaGlzdChjb2xTdW1zKGV4cF9ub3JtKSkKYGBgCiMjIEZpbHRlcmluZyBmb3IgZ2VuZXMKCkluIHRoaXMgY2FzZSB3ZSBhcmUgZ29pbmcgdG8gdXNlIGEgVFBNIGZpbHRlciBvZiAxMCBmb3IgYm90aCB0aGUgcmVmZXJlbmNlIGFuZCB0ZXN0IGRhdGFzZXRzLiAKCmBgYHtyfQptZWFuX2dlbmVfbm9ybTwtcm93TWVhbnMoZXhwX25vcm0pCnN1bW1hcnkobWVhbl9nZW5lX25vcm0pCmxlbmd0aCh3aGljaChtZWFuX2dlbmVfbm9ybT4gMTApKQpleHBfbm9ybTwtIGV4cF9ub3JtW21lYW5fZ2VuZV9ub3JtPjEwLF0KYGBgCgpgYGB7cn0KbWVhbl9nZW5lX2NhbmNlcjwtIHJvd01lYW5zKGV4cF9jYW5jZXIpCnN1bW1hcnkobWVhbl9nZW5lX2NhbmNlcikKbGVuZ3RoKHdoaWNoKG1lYW5fZ2VuZV9jYW5jZXI+IDEwKSkKZXhwX2NhbmNlcjwtIGV4cF9jYW5jZXJbbWVhbl9nZW5lX2NhbmNlcj4xMCxdCmBgYAoKYGBge3J9CmludGVyX2dlbmVzPC0gaW50ZXJzZWN0KHJvd25hbWVzKGV4cF9jYW5jZXIpLCByb3duYW1lcyhleHBfbm9ybSkpCmxlbmd0aChpbnRlcl9nZW5lcykKZXhwX2NhbmNlcjwtIGV4cF9jYW5jZXJbaW50ZXJfZ2VuZXMsXQpleHBfbm9ybSA8LSBleHBfbm9ybVtpbnRlcl9nZW5lcyxdCmBgYAojIyBHZW5lcmF0aW5nIFVNQVAgb2YgdGhlIGRhdGEKCmBgYHtyfQpzZXQuc2VlZCg0MykKc2NlX2NlbGxzX3VtYXAgPC0gdW1hcCh0KGV4cF9jYW5jZXIpLCAgICAgICAgICAgICAgICAgICAgICAgCiAgICAgICAgICAgICAgICAgICAgICAgbl9jb21wb25lbnRzID0gMiwKICAgICAgICAgICAgICAgICAgICAgICAjYT0xLAogICAgICAgICAgICAgICAgICAgICAgICNiPTAuNiwKICAgICAgICAgICAgICAgICAgICAgICBtaW5fZGlzdD0wLjAxLAogICAgICAgICAgICAgICAgICAgICAgIG1ldHJpYz0iY29zaW5lIiwKICAgICAgICAgICAgICAgICAgICAgICB2ZXJib3NlID0gVFJVRSwKICAgICAgICAgICAgICAgICAgICAgICBuX3RocmVhZHMgPSBtYXgoMSwgUmNwcFBhcmFsbGVsOjpkZWZhdWx0TnVtVGhyZWFkcygpLzIpICkKY29sbmFtZXMoc2NlX2NlbGxzX3VtYXApPC0gYygiVU1BUDEiLCAiVU1BUDIiKQpyb3duYW1lcyhzY2VfY2VsbHNfdW1hcCk8LSBjb2xuYW1lcyhleHBfY2FuY2VyKQpzY2VfY2VsbHNfdW1hcDwtIGNiaW5kKHNjZV9jZWxsc191bWFwLCBtZXRhKQoKaGVhZChzY2VfY2VsbHNfdW1hcCkKCmBgYAoKYGBge3J9CmdncGxvdChzY2VfY2VsbHNfdW1hcCwgYWVzKFVNQVAxLCBVTUFQMiwgY29sb3I9Y2VsbCkgKSsKICBnZW9tX3BvaW50KCkKYGBgCgoKIyMgQ3JlYXRpbmcgbWV0YSBkYXRhCgpgYGB7cn0KbWV0YTwtIGRhdGEuZnJhbWUoIHJvdy5uYW1lcz1jb2xuYW1lcyhleHBfY2FuY2VyKSwgCiAgICAgICAgICAgICAgICAgICBwYXRpZW50PSBzYXBwbHkoY29sbmFtZXMoZXhwX2NhbmNlciksIGZ1bmN0aW9uKFgpIHVubGlzdCggc3Ryc3BsaXQoWCwgIl8iKSlbMV0pLAogICAgICAgICAgICAgICAgICAgcGxhdGU9IHNhcHBseShjb2xuYW1lcyhleHBfY2FuY2VyKSwgZnVuY3Rpb24oWCkgdW5saXN0KCBzdHJzcGxpdChYLCAiXyIpKVsyXSksCiAgICAgICAgICAgICAgICAgICB3ZWxsID0gc2FwcGx5KGNvbG5hbWVzKGV4cF9jYW5jZXIpLCBmdW5jdGlvbihYKSB1bmxpc3QoIHN0cnNwbGl0KFgsICJfIikpWzNdKSwKICAgICAgICAgICAgICAgICAgIHN0cmluZ3NBc0ZhY3RvcnM9RkFMU0UpCmxlbmd0aChpbnRlcnNlY3Qocm93bmFtZXMobWV0YSksIHJvd25hbWVzKG1ldGFfY2VsbHR5cGVzKSkpCm1ldGE8LSBjYmluZChtZXRhLCBtZXRhX2NlbGx0eXBlc1tyb3duYW1lcyhtZXRhKSwsZHJvcD1GQUxTRV0pCmhlYWQobWV0YSkKCmBgYAoKCiMjIFJ1bm5pbmcgSG9uZXlCYWRnZXIgYW5hbHlzaXMgZm9yIFBUMDM5CgpgYGB7cn0KbWFydC5vYmogPC0gdXNlTWFydChiaW9tYXJ0ID0gImVuc2VtYmwiLCBkYXRhc2V0ID0gJ2hzYXBpZW5zX2dlbmVfZW5zZW1ibCcsIGhvc3QgPSAiZmViMjAxNC5hcmNoaXZlLmVuc2VtYmwub3JnIikgIyMgdmVyc2lvbiB1c2VkIGluIG1hbnVzY3JpcHQKYGBgCgpgYGB7cn0KUFQwMzlfbWV0YTwtIG1ldGFbbWV0YSRwYXRpZW50ICA9PSAiUFQwMzkiLF0KIyMgU3Vic2V0dGluZyBjZWxscwpzZXQuc2VlZCg0MykKI2NlbGxzX2tlZXA8LSBzYW1wbGUocm93bmFtZXMoUFQwMzlfbWV0YSksIDEwMCkKY2VsbHNfa2VlcDwtIHJvd25hbWVzKFBUMDM5X21ldGEpClBUMDM5X21ldGE8LSBQVDAzOV9tZXRhW2NlbGxzX2tlZXAsXQpoZWFkKFBUMDM5X21ldGEpClBUMDM5X2V4cDwtIGV4cF9jYW5jZXJbLGNlbGxzX2tlZXBdCmBgYAoKTWFraW5nIGEgbmV3IEhvbmV5QmFkZ2VyIG9iamVjdAoKYGBge3J9CmhiIDwtIG5ldygnSG9uZXlCQURHRVInLCBuYW1lPSdQVDAzOScpCgpoYiRzZXRHZXhwTWF0cyhsb2cyKGFzLm1hdHJpeChQVDAzOV9leHApKzEpLCBsb2cyKGV4cF9ub3JtKzEpLCBtYXJ0Lm9iaiwgZmlsdGVyPUZBTFNFLCBzY2FsZT1UUlVFLCB2ZXJib3NlPVRSVUUpCmBgYApgYGB7ciwgd2lkdGg9MTUsIGhlaWdodD0xMH0KaGIkcGxvdEdleHBQcm9maWxlKCkgIyMgaW5pdGlhbCB2aXN1YWxpemF0aW9uCmBgYApgYGB7cn0KaGIkc2V0TXZGaXQodmVyYm9zZT1UUlVFKSAjIyBtb2RlbCB2YXJpYW5jZQoKaGIkc2V0R2V4cERldih2ZXJib3NlPVRSVUUpICMjIG1vZGVsIG5lY2Vzc2FyeSBleHByZXNzaW9uIGRldmlhdGlvbiB0byBpZGVudGlmeSBDTlZzCmBgYApgYGB7cn0KaGIkY2FsY0dleHBDbnZCb3VuZGFyaWVzKGluaXQ9VFJVRSwgdmVyYm9zZT1GQUxTRSkgIyMgSE1NCmBgYApEb3VibGUgY2hlY2sgd2hhdCBDTlZzIHdlcmUgaWRlbnRpZmllZAoKYGBge3J9CmJnZiA8LSBoYiRib3VuZC5nZW5lcy5maW5hbApnZW5lcyA8LSBoYiRnZW5lcwpyZWdpb25zLmdlbmVzIDwtIHJhbmdlKGdlbmVzW3VubGlzdChiZ2YpXSkKcHJpbnQocmVnaW9ucy5nZW5lcykKCmBgYApUaGUgaW5pdGlhbCBITU0gc3RlcCBpZGVudGlmaWVkIGEgbnVtYmVyIG9mIGNhbmRpZGF0ZSBDTlZzIHRvIHRlc3QuIAoKTm93LCB3ZSBjYW4gdXNlIHRoZSBiYXllc2lhbiBtb2RlbCB0byBkZXJpdmUgdGhlIGZpbmFsIHBvc3RlcmlvciBwcm9iYWJpbGl0eSBvZiBlYWNoIENOViBpbiBlYWNoIGNlbGwuIAoKSW5kZWVkLCBvdXIgaW5pdGlhbCBITU0gaGFzIGlkZW50aWZpZWQgYSBudW1iZXIgb2YgY2FuZGlkYXRlIENOVnMgdG8gdGVzdC4gV2UgY2FuIG5vdyByZXRlc3QgYWxsIGlkZW50aWZpZWQgQ05WcyBvbiBhbGwgY2VsbHMgdG8gZGVyaXZlIHRoZSBmaW5hbCBwb3N0ZXJpb3IgcHJvYmFiaWxpdHkgb2YgZWFjaCBDTlYgaW4gZWFjaCBjZWxsLgoKYGBge3J9CmhiJHJldGVzdElkZW50aWZpZWRDbnZzKHJldGVzdEJvdW5kR2VuZXMgPSBUUlVFLCByZXRlc3RCb3VuZFNucHMgPSBGQUxTRSwgdmVyYm9zZT1GQUxTRSkKYGBgCgpgYGB7cn0KIyMgbG9vayBhdCBmaW5hbCByZXN1bHRzCnJlc3VsdHMgPC0gaGIkc3VtbWFyaXplUmVzdWx0cyhnZW5lQmFzZWQ9VFJVRSwgYWxsZWxlQmFzZWQ9RkFMU0UpCnByaW50KGhlYWQocmVzdWx0c1ssMTo1XSkpCmBgYAoKQ2x1c3RlciBjZWxscyBvbiB0aGVzZSBwb3N0ZXJpb3IgcHJvYmFiaWxpdGllcyBhbmQgdmlzdWFsaXplIHRoZW0gYXMgYSBoZWF0bWFwLgogCmBgYHtyfQp0cmVlcyA8LSBoYiR2aXN1YWxpemVSZXN1bHRzKGdlbmVCYXNlZD1UUlVFLCBhbGxlbGVCYXNlZD1GQUxTRSwgZGV0YWlscz1UUlVFLCBtYXJnaW5zPWMoNSw1KSkKYGBgCgpXZSBjYW4gYWdhaW4gdmlzdWFsaXplIG91ciByZXN1bHRzLCB0aGlzIHRpbWUsIG9yZGVyaW5nIHRoZSBjZWxscyBiYXNlZCBvbiB0aGVpciBwb3N0ZXJpb3IgcHJvYmFiaWxpdGllcyBvZiBoYXJib3JpbmcgQ05Wcy4KCmBgYHtyfQojIyBvcmRlciBjZWxscwpoYyA8LSB0cmVlcyRoYwpvcmRlciA8LSBoYyRsYWJlbHNbaGMkb3JkZXJdCiMjIHBsb3QgYWxsIGNocm9tb3NvbWVzCmhiJHBsb3RHZXhwUHJvZmlsZShjZWxsT3JkZXI9aGMkb3JkZXIpCmBgYAoKV2UgY2FuIGNvbXBhcmUgdGhpcyByZXN1bHQgd2l0aCB0aGUgb25lIGZyb20gdGhlIHBhcGVyIChGaWd1cmUgMkQsRSk6CgohW0thcmFheXZheiAyMDE4IDJELEVdKEthcmFheXZhel8yMDE4XzJERS5wbmcpCgpXZSBvYnNlcnZlIGFsc28gYSBoaWdobGlnaHRlZCBhbXBsaWZpY2F0aW9uIGluIGNocjEyLCB0aGF0IHdhcyBsYXRlciBjb25maXJtZWQgYnkgV0VTIChGaWd1cmUgMkUpCgpgYGB7cn0KIyMgcGxvdCBqdXN0IGlkZW50aWZpZWQgY252cwpoYiRwbG90R2V4cFByb2ZpbGUoY2VsbE9yZGVyPWhjJG9yZGVyLCByZWdpb249aGIkY252c1tbJ2dlbmUtYmFzZWQnXV1bWydhbXAnXV0pCmBgYAoKCmBgYHtyfQpoYiRwbG90R2V4cFByb2ZpbGUoY2VsbE9yZGVyPWhjJG9yZGVyLCByZWdpb249aGIkY252c1tbJ2dlbmUtYmFzZWQnXV1bWydkZWwnXV0pCmBgYAoKRmluYWxseSwgd2UgY2FuIGFsc28gdmlzdWFsaXplIHRoZSByZXN1bHRzIGZyb20gSG9uZXlCYWRnZXIgaW50byBvdXIgVU1BUHMuIAoKYGBge3J9CmhlYWQocmVzdWx0cykKcHJvYl9yZXM8LSBkYXRhLmZyYW1lKHQocmVzdWx0c1ssODpuY29sKHJlc3VsdHMpXSkpCmNvbG5hbWVzKHByb2JfcmVzKTwtIHBhc3RlMCgicHJvYl8iLCByZXN1bHRzJHNlcW5hbWVzLCAiXyIsIHJlc3VsdHMkc3RhcnQsICJfIiwgcmVzdWx0cyRlbmQpCmxlbmd0aChpbnRlcnNlY3Qocm93bmFtZXMocHJvYl9yZXMpLCByb3duYW1lcyhzY2VfY2VsbHNfdW1hcCkpKQpzY2VfY2VsbHNfdW1hcDwtIGNiaW5kKHNjZV9jZWxsc191bWFwLHByb2JfcmVzW3Jvd25hbWVzKHNjZV9jZWxsc191bWFwKSxdKSAKaGVhZChzY2VfY2VsbHNfdW1hcCkKCmBgYApgYGB7cn0KZ2dwbG90KHNjZV9jZWxsc191bWFwLCBhZXMoVU1BUDEsIFVNQVAyLCBjb2xvcj1jZWxsKSApKwogIGdlb21fcG9pbnQoKQpgYGAKCmBgYHtyfQpnZ3Bsb3Qoc2NlX2NlbGxzX3VtYXAsIGFlcyhVTUFQMSwgVU1BUDIsIGNvbG9yPXBhdGllbnQpICkrCiAgZ2VvbV9wb2ludCgpCmBgYCAgIAoKYGBge3IsIGZpZy53aWR0aD0yMCwgZmlnLmhlaWdodD04fQpnZ3Bsb3Qoc2NlX2NlbGxzX3VtYXAsIGFlcyhVTUFQMSwgVU1BUDIsIGNvbG9yPXByb2JfY2hyMTJfODYxNzU5XzMzMDQ5Nzc0KSApKwogIGdlb21fcG9pbnQoKSsKICBzY2FsZV9jb2xvcl92aXJpZGlzX2MoKQpgYGAKCgpgYGB7cn0Kc2Vzc2lvbkluZm8oKQpgYGA=

Linking transcriptional and genetic tumor heterogeneity through allele analysis of single-cell RNA-seq data

2020-11-18T00:00:00+00:00

In our first session we will discuss “Linking transcriptional and genetic tumor heterogeneity through allele analysis of single-cell RNA-seq data” with first author PhD Jean Fan, in this article Jean and collaborators develop HONEYBadger R package ….

This article can be downloaded here The code of HoneyBadger can be found here

How to run Honey BADGER?

2020-11-04T00:00:00+00:00

Fish is really a good shell that I’m using everyday since two years now. Unfortunately there is not much guides about how to program using it. That’s sad because fish has a lot of advantages over bash to write scripts: cleaner and easier to remember syntax, good variables escaping, etc.. So I decided to write a small tutorial on the subject.

By “programming” I mean making small program that can realize simple mathematical operations, apply algorithms, etc… Yes, as strange as it way seem even when reading the full fish documentation it’s still easier to make such basic things in C rather than fish or bash. That is, in my opinion, because that documentation (which is quite good indeed) mainly speaks about an interactive shell rather than about a programming language. This article aims to correct that point.

To start, install fish, create a small .fish file, put a #! /usr/bin/fish on top of it, make it executable and start to read the guide.

Loading libraries

library(HoneyBADGER)
library(biomaRt)

The set keyword takes as first argument the name of the variable. All the remaining arguments will constitute its value. If there are multiple ones the resulting variable will be a list.

To use that variable, use the $ prefix:

echo $variable_name

Mathematical operations

The dedicated command for mathematical operations in fish is math. Example:

math "1+2"
# 3

math concatenates all its arguments with a space between them before evaluating the complete expression. So math 1+2, math 1 + 2 and math 1+ 2 are all equivalent. But since many operators can also be used in the fish syntax (like <, >, |, *,…) it is usually easier to put everything in quotes.

math supports most C operators and outputs the result on stdout.

To play a little with mathematical operations and variables we will also need to use command substitution, which is realized in fish using parentheses:

set x 1
set y (math "$x*3")
set y (math "$y+1")
echo $y
# 4

If

The if in fish is quite easy to remember compared to bash:

if some_command_returning_true_or_false
    echo "command returned true"
else
    echo "command returned false"
end

There is no specific delimiter to mark the beginning of command blocks except the carriage return. The end of the structure is marked with the end keyword, like with all other command structures.

What is a little bit more tricky is the condition used for the if: it’s a command whose returning status should be 0 (for true) or any other value (for false). Since the math keyword returns its value on stdout and not as a status, how to use it with if? This is the trick: the returning status of math will be false if the calculated value is equal to 0 and true for any other value. So the following code will work:

if math "1==1"
    echo "1 equals 1"
else
    echo "1 is not equal to 1"
end

Even if it works as expected, there is still one little problem. It will output this when executed:

1
1 equals 1

That is because the math command will perform as it is supposed to and output its value on stdout. To avoid this, we can redirect its output to /dev/null:

if math "1==1" > /dev/null
    echo "1 equals 1"
else
    echo "1 is not equal to 1"
end

As a side note, the test command can also be useful for expression evaluation in if. It can perform checks on files and strings.

While

set x 0
while math "$x<10" > /dev/null
    echo $x
    set x (math "$x+1")
end

Now that we explained the if, the while should be quite trivial isn’t it?

Lists

As explained before, when we assign multiple values to a variable it will create a list. The different elements of that list can be accessed using the [] syntax:

set myvar "apple" "banana" "apple pie"
echo $myvar[1]
# apple
echo $myvar[2]
# banana
echo $myvar[3]
# apple pie
echo $myvar[1..2]
# apple banana

When passing a whole list as an argument to a command fish will expand the elements of the list as multiple arguments for the command. This makes fish infinitely superior to bash by making it, at least, somewhat possible to make scripts handling correctly spaces in the file names.

set myfiles "file 1.txt" "file 2.txt"
rm $myfiles
# yes, for the first time in the whole Unix history this will work as expected

Anyway, to get the size of a list you can use the count operation:

count $myvar
# 3

Lists in fish are technically immutable, but you can construct a new list and assign the new value to the same variable:

set myvar $myvar tomato

To iterate on list elements you can also use the for ... in structure:

for el in $myvar
    echo "Element value: $el"
end
#Element value: apple
#Element value: banana
#Element value: apple pie
#Element value: tomato

Parsing command line arguments

When creating complex scripts using fish it can be useful to parse command line arguments. Most tutorials about this using bash will explain the usage of the getopts command. That command is specific to bash so it is not usable in fish, fortunately we can use an alternative which is the Unix standard getopt command (notice the only difference is the “s”).

Here is a complete example of usage of a command line arguments parsing program in fish. It seems long and boring but once we get used to it it becomes quite fast to write.

function help_exit
    echo "Usage:  [options] arguments..."
    echo "Arguments:"
    echo "-a : Do something"
    echo "-d : Do something else"
    echo "-c stuff : Do someting with stuff"
    exit 1
end

set args (getopt -s sh abc: $argv); or help_exit
set args (fish -c "for el in $args; echo \$el; end")

set i 1
while true
    switch $args[$i]
        case "-a"
            echo "argument a is specified"
        case "-b"
            echo "argument b is specified"
        case "-c"
            set i (math "$i + 1")
            echo "value of argument c is" $args[$i]
        case "--"
            break
    end
    set i (math "$i + 1")
end
set pargs
if math "$i <" (count $args) > /dev/null
    set pargs $args[(math "$i + 1")..-1]
end

echo "positional arguments:" $pargs

The most interesting part of this example is the call to getopt. getopt‘s first positional argument is the options specifier (here abc:). That options specifier is a list of characters where each character is a possible option for our program (only short options with only one character are supported with this usage). If an option is followed by a colon (;) it means that option takes a parameter. The rest of the positional arguments for getopt are the arguments to parse (in fish it’s the $argv list).

The whole purspose of getopt is to analyze Unix-style command line arguments and rewrite them in a more standardized format which is easy to parse with a simple while. By using the -s sh option of getopt and using a little trick by invoking a new fish interpreter I create a nice list where each arguments are clearly indicated. As example, if I use these arguments:

-ac tmp.fs hello world

I will obtain these elements in args:

"-a" "-c" "tmp.fs" "--" "hello" "world"

getopt was smart enough separate the -a and -c arguments and determine that tmp.fs was a parameter of -c instead of a positional argument. -- always indicate the beginning of the positional arguments. getopt also return false if it detected an error during the parsing, in which case we generally exit the program after printing the help.

The End

That’s all for now, if you have some ideas about additional stuff to explain in this guide don’t hesitate to post some comments. Just for fun, here is an implementation of 99 bottles of beer on the wall in fish:

#! /usr/bin/fish

for quant in (seq 99 -1 1)
    if math "$quant > 1" > /dev/null
        echo "$quant bottles of beer on the wall, $quant bottles of beer."
        if math "$quant > 2" > /dev/null
            set suffix (math "$quant - 1")
            set suffix "$suffix bottles of beer on the wall."
        else
            set suffix "1 bottle of beer on the wall."
        end
    else if math "$quant == 1" > /dev/null
        echo "1 bottle of beer on the wall, 1 bottle of beer."
        set suffix "no more beer on the wall!"
    end
    echo "Take one down, pass it around, $suffix"
    echo "--"
end

PJC - Computational Biology

Honeybadger_karaayvaz.nb

HoneyBadger on Karaayvaz et al 2018

Ariel Madrigal

2020-November-17

Loading libraries

Loading data

Filtering for genes

Generating UMAP of the data

Creating meta data

Running HoneyBadger analysis for PT039

Linking transcriptional and genetic tumor heterogeneity through allele analysis of single-cell RNA-seq data

How to run Honey BADGER?

Loading libraries

Mathematical operations

If

While

Lists

Parsing command line arguments

The End