Parsimonious module inference in large networks
- PMID: 25167049
- DOI: 10.1103/PhysRevLett.110.148701
Parsimonious module inference in large networks
Abstract
We investigate the detectability of modules in large networks when the number of modules is not known in advance. We employ the minimum description length principle which seeks to minimize the total amount of information required to describe the network, and avoid overfitting. According to this criterion, we obtain general bounds on the detectability of any prescribed block structure, given the number of nodes and edges in the sampled network. We also obtain that the maximum number of detectable blocks scales as sqrt[N], where N is the number of nodes in the network, for a fixed average degree ⟨k⟩. We also show that the simplicity of the minimum description length approach yields an efficient multilevel Monte Carlo inference algorithm with a complexity of O(τNlogN), if the number of blocks is unknown, and O(τN) if it is known, where τ is the mixing time of the Markov chain. We illustrate the application of the method on a large network of actors and films with over 10(6) edges, and a dissortative, bipartite block structure.
Similar articles
-
Efficient Monte Carlo and greedy heuristic for the inference of stochastic block models.Phys Rev E Stat Nonlin Soft Matter Phys. 2014 Jan;89(1):012804. doi: 10.1103/PhysRevE.89.012804. Epub 2014 Jan 13. Phys Rev E Stat Nonlin Soft Matter Phys. 2014. PMID: 24580278
-
Robust full Bayesian learning for radial basis networks.Neural Comput. 2001 Oct;13(10):2359-407. doi: 10.1162/089976601750541831. Neural Comput. 2001. PMID: 11571002
-
Nonparametric Bayesian inference of the microcanonical stochastic block model.Phys Rev E. 2017 Jan;95(1-1):012317. doi: 10.1103/PhysRevE.95.012317. Epub 2017 Jan 17. Phys Rev E. 2017. PMID: 28208453
-
MODELING SOCIAL NETWORKS FROM SAMPLED DATA.Ann Appl Stat. 2010;4(1):5-25. doi: 10.1214/08-AOAS221. Ann Appl Stat. 2010. PMID: 26561513 Free PMC article.
-
Biological Network Inference and analysis using SEBINI and CABIN.Methods Mol Biol. 2009;541:551-76. doi: 10.1007/978-1-59745-243-4_24. Methods Mol Biol. 2009. PMID: 19381531 Review.
Cited by
-
Quantifying metadata relevance to network block structure using description length.Commun Phys. 2024;7(1):331. doi: 10.1038/s42005-024-01819-y. Epub 2024 Oct 11. Commun Phys. 2024. PMID: 39398491 Free PMC article.
-
Efficiently inferring community structure in bipartite networks.Phys Rev E Stat Nonlin Soft Matter Phys. 2014 Jul;90(1):012805. doi: 10.1103/PhysRevE.90.012805. Epub 2014 Jul 10. Phys Rev E Stat Nonlin Soft Matter Phys. 2014. PMID: 25122340 Free PMC article.
-
A Shadowing Problem in the Detection of Overlapping Communities: Lifting the Resolution Limit through a Cascading Procedure.PLoS One. 2015 Oct 13;10(10):e0140133. doi: 10.1371/journal.pone.0140133. eCollection 2015. PLoS One. 2015. PMID: 26461919 Free PMC article.
-
A network approach to topic models.Sci Adv. 2018 Jul 18;4(7):eaaq1360. doi: 10.1126/sciadv.aaq1360. eCollection 2018 Jul. Sci Adv. 2018. PMID: 30035215 Free PMC article.
-
Cross-validation estimate of the number of clusters in a network.Sci Rep. 2017 Jun 12;7(1):3327. doi: 10.1038/s41598-017-03623-x. Sci Rep. 2017. PMID: 28607441 Free PMC article.
LinkOut - more resources
Full Text Sources
Other Literature Sources