FreeHi-C Spike-in Simulations for Benchmarking Differential Chromatin Interaction Detection

Abstract

High-throughput genome-wide chromatin conformation capture assay (Hi-C) is routinely used to profile long-range genomic interactions and three-dimensional organization of genomes. A key application of Hi-C is the comparative analysis of genomic interactions across different time points, cellular conditions, or multiple stimuli. While operating characteristics of methods for Hi-C data processing such as normalization, pairwise interaction and higher-order organization detection have been relatively well studied, properties of methods for differential chromatin interaction detection are less investigated. We have recently developed FreeHi-C to enable data-driven non-parametric simulations from Hi-C experiments. Here, we extend FreeHi-C with a user/data-driven spike-in module to facilitate comparisons of differential chromatin interaction detection methods where the ground truth differential chromatin interactions are known under a wide variety of settings. We use FreeHi-C to benchmark four differential chromatin interaction detection methods, namely HiCcompare, multiHiCcompare, diffHic, and Selfish, using three comparative analysis settings with different sequencing depths and spike-in proportions. This comparison reveals distinguished performances in terms of the standard metrics such as the false discovery rate control, detection power, significance order, precision-recall curve, and receiver operating characteristic curve as well as overall genomic properties of the types of differential chromatin interactions detectable by each method. Furthermore, it highlights the lack of power for all methods in small replication settings.