Commit af64085d authored by Tiago Peixoto's avatar Tiago Peixoto
Browse files

doctests: fix double execution and references

parent 9b01da33
Pipeline #945 passed with stage
in 43 minutes and 12 seconds
......@@ -12,9 +12,9 @@ it might make sense to search exclusively for assortative communities
[lizhi-statistical-2020]_. A version of the SBM that is constrained in
this way is called the "planted partition model", which can be inferred
with graph-tool using
:class:`~graph_tool.inference.planted_partition.PPBlockState`. This
:class:`~graph_tool.inference.PPBlockState`. This
class behaves just like
:class:`~graph_tool.inference.blockmodel.BlockState`, therefore all
:class:`~graph_tool.inference.BlockState`, therefore all
algorithms described in this documentation work in the same way. Below
we show how this model can be inferred for the football network
considered previously
......@@ -81,8 +81,8 @@ and any of the other discrete distributions for the magnitude,
The support for weighted networks is activated by passing the parameters
``recs`` and ``rec_types`` to
:class:`~graph_tool.inference.blockmodel.BlockState` (or
:class:`~graph_tool.inference.BlockState` (or
that specify the edge covariates (an edge
:class:`~graph_tool.PropertyMap`) and their types (a string from the
table above), respectively. Note that these parameters expect *lists*,
......@@ -6,9 +6,9 @@ representing distinct types if interactions
[peixoto-inferring-2015]_. Extensions to the SBM may be defined for such
data, and they can be inferred using the exact same interface shown
above, except one should use the
class, instead of
:class:`~graph_tool.inference.blockmodel.BlockState`. This class takes
:class:`~graph_tool.inference.BlockState`. This class takes
two additional parameters: the ``ec`` parameter, that must correspond to
an edge :class:`~graph_tool.PropertyMap` with the layer/covariate values
on the edges, and the Boolean ``layers`` parameter, which if ``True``
......@@ -16,10 +16,10 @@ specifies a layered model, otherwise one with categorical edge
covariates (not to be confused with the weighted models in
Sec. :ref:`weights`).
If we use :func:`~graph_tool.inference.minimize.minimize_blockmodel_dl`, this can
If we use :func:`~graph_tool.inference.minimize_blockmodel_dl`, this can
be achieved simply by passing the option ``layers=True`` as well as the
appropriate value of ``state_args``, which will be propagated to
:class:`~graph_tool.inference.layered_blockmodel.LayeredBlockState`'s constructor.
:class:`~graph_tool.inference.LayeredBlockState`'s constructor.
As an example, let us consider a social network of tribes, where two
types of interactions were recorded, amounting to either friendship or
......@@ -4,8 +4,8 @@ Inferring the best partition
The simplest and most efficient approach is to find the best
partition of the network by maximizing Eq. :eq:`model-posterior`
according to some version of the model. This is obtained via the
functions :func:`~graph_tool.inference.minimize.minimize_blockmodel_dl` or
:func:`~graph_tool.inference.minimize.minimize_nested_blockmodel_dl`, which
functions :func:`~graph_tool.inference.minimize_blockmodel_dl` or
:func:`~graph_tool.inference.minimize_nested_blockmodel_dl`, which
employs an agglomerative multilevel `Markov chain Monte Carlo (MCMC)
<>`_ algorithm
......@@ -41,7 +41,7 @@ We then fit the degree-corrected model by calling:
state = gt.minimize_blockmodel_dl(g)
This returns a :class:`~graph_tool.inference.blockmodel.BlockState` object that
This returns a :class:`~graph_tool.inference.BlockState` object that
includes the inference results.
.. note::
......@@ -59,12 +59,12 @@ includes the inference results.
and select the partition with the largest posterior probability of
Eq. :eq:`model-posterior`, or equivalently, the minimum description
length of Eq. :eq:`model-dl`. The description length of a fit can be
obtained with the :meth:`~graph_tool.inference.blockmodel.BlockState.entropy`
obtained with the :meth:`~graph_tool.inference.BlockState.entropy`
method. See also Sec. :ref:`sec_model_selection` below.
We may perform a drawing of the partition obtained via the
:mod:`~graph_tool.inference.blockmodel.BlockState.draw` method, that functions as a
:mod:`~graph_tool.inference.BlockState.draw` method, that functions as a
convenience wrapper to the :func:`~graph_tool.draw.graph_draw` function
.. testcode:: football
......@@ -83,7 +83,7 @@ which yields the following image.
We can obtain the group memberships as a
:class:`~graph_tool.PropertyMap` on the vertices via the
:mod:`~graph_tool.inference.blockmodel.BlockState.get_blocks` method:
:mod:`~graph_tool.inference.BlockState.get_blocks` method:
.. testcode:: football
......@@ -105,7 +105,7 @@ which yields:
We may also access the matrix of edge counts between groups via
.. testcode:: football
......@@ -143,7 +143,7 @@ Hierarchical partitions
The inference of the nested family of SBMs is done in a similar manner,
but we must use instead the
:func:`~graph_tool.inference.minimize.minimize_nested_blockmodel_dl` function. We
:func:`~graph_tool.inference.minimize_nested_blockmodel_dl` function. We
illustrate its use with the neural network of the `C. elegans
<>`_ worm:
......@@ -169,10 +169,10 @@ A hierarchical fit of the degree-corrected model is performed as follows.
state = gt.minimize_nested_blockmodel_dl(g)
The object returned is an instance of a
:class:`~graph_tool.inference.nested_blockmodel.NestedBlockState` class, which
:class:`~graph_tool.inference.NestedBlockState` class, which
encapsulates the results. We can again draw the resulting hierarchical
clustering using the
:meth:`~graph_tool.inference.nested_blockmodel.NestedBlockState.draw` method:
:meth:`~graph_tool.inference.NestedBlockState.draw` method:
.. testcode:: celegans
......@@ -193,12 +193,12 @@ clustering using the
.. note::
If the ``output`` parameter to
:meth:`~graph_tool.inference.nested_blockmodel.NestedBlockState.draw` is omitted, an
:meth:`~graph_tool.inference.NestedBlockState.draw` is omitted, an
interactive visualization is performed, where the user can re-order
the hierarchy nodes using the mouse and pressing the ``r`` key.
A summary of the inferred hierarchy can be obtained with the
:meth:`~graph_tool.inference.nested_blockmodel.NestedBlockState.print_summary` method,
:meth:`~graph_tool.inference.NestedBlockState.print_summary` method,
which shows the number of nodes and groups in all levels:
.. testcode:: celegans
......@@ -214,8 +214,8 @@ which shows the number of nodes and groups in all levels:
l: 4, N: 1, B: 1
The hierarchical levels themselves are represented by individual
:meth:`~graph_tool.inference.blockmodel.BlockState` instances obtained via the
:meth:`~graph_tool.inference.nested_blockmodel.NestedBlockState.get_levels()` method:
:meth:`~graph_tool.inference.BlockState` instances obtained via the
:meth:`~graph_tool.inference.NestedBlockState.get_levels()` method:
.. testcode:: celegans
......@@ -254,8 +254,8 @@ Refinements using merge-split MCMC
The agglomerative algorithm behind
:func:`~graph_tool.inference.minimize.minimize_blockmodel_dl` and
:func:`~graph_tool.inference.minimize.minimize_nested_blockmodel_dl` has
:func:`~graph_tool.inference.minimize_blockmodel_dl` and
:func:`~graph_tool.inference.minimize_nested_blockmodel_dl` has
a log-linear complexity on the size of the network, and it usually works
very well in finding a good estimate of the optimum
partition. Nevertheless, it's often still possible to find refinements
......@@ -287,10 +287,10 @@ to the above minimization for the `C. elegans` network is the following:
Whenever possible, this procedure should be repeated several times, and
the result with the smallest description length (obtained via the
:meth:`~graph_tool.inference.blockmodel.BlockState.entropy` method)
:meth:`~graph_tool.inference.BlockState.entropy` method)
should be chosen. In more demanding situations, better results still can
be obtained, at the expense of a longer computation time, by using the
:meth:`~graph_tool.inference.mcmc.mcmc_anneal` function, which
:meth:`~graph_tool.inference.mcmc_anneal` function, which
implements `simulated annealing
......@@ -52,14 +52,14 @@ The computation of the posterior entropy :math:`\mathcal{S}`, however,
is significantly more difficult, since it involves measuring the precise
value of :math:`q(\boldsymbol b)`. A direct "brute force" computation of
:math:`\mathcal{S}` is implemented via
and :func:`~graph_tool.inference.blockmodel.microstate_entropy`, however
and :func:`~graph_tool.inference.microstate_entropy`, however
this is only feasible for very small networks. For larger networks, we
are forced to perform approximations. One possibility is to employ the
method described in [peixoto-revealing-2021]_, based on fitting a
mixture "random label" model to the posterior distribution, which allows
us to compute its entropy. In graph-tool this is done by using
:class:`~graph_tool.inference.partition_modes.ModeClusterState`, as we
:class:`~graph_tool.inference.ModeClusterState`, as we
show in the example below.
.. testsetup:: model-evidence
......@@ -46,7 +46,7 @@ which do not depend on the normalization constant.
The values :math:`P(\delta \boldsymbol A | \boldsymbol A, \boldsymbol b)`
can be computed with
:meth:`~graph_tool.inference.blockmodel.BlockState.get_edges_prob`. Hence, we can
:meth:`~graph_tool.inference.BlockState.get_edges_prob`. Hence, we can
compute spurious/missing edge probabilities just as if we were
collecting marginal distributions when doing model averaging.
......@@ -62,11 +62,11 @@ generally can lead to substantial improvements
In graph-tool there is support for reconstruction with the above
framework for three measurement processes: 1. Repeated measurements with
uniform errors (via
:class:`~graph_tool.inference.uncertain_blockmodel.MeasuredBlockState`), 2. Repeated
:class:`~graph_tool.inference.MeasuredBlockState`), 2. Repeated
measurements with heterogeneous errors (via
and 3. Extraneously obtained edge probabilities (via
which we describe in the following.
In addition, it is also possible to reconstruct networks from observed
......@@ -125,7 +125,7 @@ In this situation the priors :math:`P(p|\alpha=1,\beta=1)` and
Below, we illustrate how the reconstruction can be performed with a
simple example, using
.. testsetup:: measured
......@@ -506,7 +506,7 @@ into the scheme of Eq. :eq:`posterior-reconstruction` by considering
the data to be the observed simple graph,
:math:`\boldsymbol{\mathcal{D}} = \boldsymbol G`. We proceed in same
way as in the previous reconstruction scenarios, but using instead
For example, in the following we will obtain the community structure and
latent multiedges of a network of political books:
......@@ -574,7 +574,7 @@ community structure [peixoto-disentangling-2021]_. This approach can be
used to separate the effects of triangle formation from node homophily,
which are typically conflated. We proceed in same way as in the previous
reconstruction scenarios, but using instead
For example, in the following we will obtain the community structure and
latent closure edges of a network of political books:
......@@ -652,7 +652,7 @@ latent closure edges of a network of political books:
Triadic closure can also be used to perform uncertain network
reconstruction, using
in a manner analogous to what was done in :ref:`measured_networks`:
.. testsetup:: measured-closure
......@@ -9,15 +9,15 @@ cost. In such situations, we are required to infer the network of
interactions from the observed functional behavior
[peixoto-network-2019]_. In graph-tool this can be done for epidemic
spreading (via
for the kinetic Ising model (via
and for the equilibrium Ising model (via
We consider the general reconstruction framework outlined above, where
the observed data :math:`\mathcal{D}` in
Eq. :eq:`posterior-reconstruction` are a time-series generated by some
......@@ -26,11 +26,11 @@ groups being used in the model, and hence is suitable for use on very
large networks.
In order to perform such moves, one needs again to operate with
:class:`~graph_tool.inference.blockmodel.BlockState` or
:class:`~graph_tool.inference.BlockState` or
instances, and calling either their
:meth:`~graph_tool.inference.blockmodel.BlockState.mcmc_sweep` or
:meth:`~graph_tool.inference.BlockState.mcmc_sweep` or
methods. The former implements a simpler MCMC where a single node is
moved at a time, where the latter is a more efficient version that
performs merges and splits [peixoto-merge-split-2020]_, which should be
......@@ -65,7 +65,7 @@ from a random partition into 20 groups
Although the above is sufficient to implement sampling from the
posterior, there is a convenience function called
:func:`~graph_tool.inference.mcmc.mcmc_equilibrate` that is intend to
:func:`~graph_tool.inference.mcmc_equilibrate` that is intend to
simplify the detection of equilibration, by keeping track of the maximum
and minimum values of description length encountered and how many sweeps
have been made without a "record breaking" event. For example,
......@@ -81,14 +81,14 @@ Note that the value of ``wait`` above was made purposefully low so that
the output would not be overly long. The most appropriate value requires
experimentation, but a typically good value is ``wait=1000``.
The function :func:`~graph_tool.inference.mcmc.mcmc_equilibrate` accepts
The function :func:`~graph_tool.inference.mcmc_equilibrate` accepts
a ``callback`` argument that takes an optional function to be invoked
after each call to
:meth:`~graph_tool.inference.blockmodel.BlockState.multiflip_mcmc_sweep`. This
:meth:`~graph_tool.inference.BlockState.multiflip_mcmc_sweep`. This
function should accept a single parameter which will contain the actual
:class:`~graph_tool.inference.blockmodel.BlockState` instance. We will
:class:`~graph_tool.inference.BlockState` instance. We will
use this in the example below to collect the posterior vertex marginals
(via :class:`~graph_tool.inference.partition_modes.PartitionModeState`,
(via :class:`~graph_tool.inference.PartitionModeState`,
which disambiguates group labels [peixoto-revealing-2021]_), i.e. the
posterior probability that a node belongs to a given group:
......@@ -170,7 +170,7 @@ Hierarchical partitions
We can also perform model averaging using the nested SBM, which will
give us a distribution over hierarchies. The whole procedure is fairly
analogous, but now we make use of
:class:`~graph_tool.inference.nested_blockmodel.NestedBlockState` instances.
:class:`~graph_tool.inference.NestedBlockState` instances.
Here we perform the sampling of hierarchical partitions using the same
network as above.
......@@ -201,18 +201,18 @@ network as above.
.. warning::
When using
:class:`~graph_tool.inference.nested_blockmodel.NestedBlockState`, a
:class:`~graph_tool.inference.NestedBlockState`, a
single call to
performs ``niter`` sweeps at each hierarchical level once. This means
that in order for the chain to equilibrate, we need to call these
functions several times, i.e. it is not enough to call it once with a
large value of ``niter``.
Similarly to the the non-nested case, we can use
:func:`~graph_tool.inference.mcmc.mcmc_equilibrate` to do most of the boring
:func:`~graph_tool.inference.mcmc_equilibrate` to do most of the boring
work, and we can now obtain vertex marginals on all hierarchical levels:
.. testcode:: nested-model-averaging
......@@ -332,7 +332,7 @@ Characterizing the posterior distribution
The posterior distribution of partitions can have an elaborate
structure, containing multiple possible explanations for the data. In
order to summarize it, we can infer the modes of the distribution using
:class:`~graph_tool.inference.partition_modes.ModeClusterState`, as
:class:`~graph_tool.inference.ModeClusterState`, as
described in [peixoto-revealing-2021]_. This amounts to identifying
clusters of partitions that are very similar to each other, but
sufficiently different from those that belong to other
......@@ -25,95 +25,3 @@
.. automodule:: graph_tool.inference.base_states
.. automodule:: graph_tool.inference.blockmodel
.. autoclass:: graph_tool.inference.blockmodel.PartitionHist
.. autoclass:: graph_tool.inference.blockmodel.BlockPairHist
.. automodule:: graph_tool.inference.overlap_blockmodel
.. automodule:: graph_tool.inference.layered_blockmodel
.. automodule:: graph_tool.inference.nested_blockmodel
.. automodule:: graph_tool.inference.uncertain_blockmodel
.. automodule:: graph_tool.inference.latent_layers
.. automodule:: graph_tool.inference.latent_multigraph
.. automodule:: graph_tool.inference.mcmc
.. automodule:: graph_tool.inference.minimize
.. automodule:: graph_tool.inference.partition_modes
.. automodule:: graph_tool.inference.partition_centroid
.. automodule:: graph_tool.inference.planted_partition
.. automodule:: graph_tool.inference.blockmodel_em
.. automodule:: graph_tool.inference.util
.. automodule:: graph_tool.inference.modularity
.. automodule:: graph_tool.inference.clique_decomposition
.. automodule:: graph_tool.inference.norm_cut
......@@ -1635,7 +1635,7 @@ def draw_hierarchy(state, pos=None, layout="radial", beta=0.8, node_weight=None,
state : :class:`~graph_tool.inference.nested_blockmodel.NestedBlockState`
state : :class:`~graph_tool.inference.NestedBlockState`
Nested block state to be drawn.
pos : :class:`~graph_tool.VertexPropertyMap` (optional, default: ``None``)
If supplied, this specifies a vertex property map with the positions of
......@@ -230,6 +230,7 @@ __all__ = ["minimize_blockmodel_dl",
......@@ -123,7 +123,7 @@ class MCMCState(ABC):
made for each node.
entropy_args : ``dict`` (optional, default: ``{}``)
Entropy arguments, with the same meaning and defaults as in
allow_vacate : ``bool`` (optional, default: ``True``)
Allow groups to be vacated.
sequential : ``bool`` (optional, default: ``True``)
......@@ -234,7 +234,7 @@ class MultiflipMCMCState(ABC):
made for each node, on average.
entropy_args : ``dict`` (optional, default: ``{}``)
Entropy arguments, with the same meaning and defaults as in
verbose : ``bool`` (optional, default: ``False``)
If ``verbose == True``, detailed information will be displayed.
......@@ -375,7 +375,7 @@ class MultilevelMCMCState(ABC):
entropy_args : ``dict`` (optional, default: ``{}``)
Entropy arguments, with the same meaning and defaults as in
verbose : ``bool`` (optional, default: ``False``)
If ``verbose == True``, detailed information will be displayed.
......@@ -460,7 +460,7 @@ class GibbsMCMCState(ABC):
made for each node.
entropy_args : ``dict`` (optional, default: ``{}``)
Entropy arguments, with the same meaning and defaults as in
allow_new_group : ``bool`` (optional, default: ``True``)
Allow the number of groups to increase and decrease.
sequential : ``bool`` (optional, default: ``True``)
......@@ -525,8 +525,8 @@ class MulticanonicalMCMCState(ABC):
m_state : :class:`~graph_tool.inference.mcmc.MulticanonicalState`
:class:`~graph_tool.inference.mcmc.MulticanonicalState` instance
m_state : :class:`~graph_tool.inference.MulticanonicalState`
:class:`~graph_tool.inference.MulticanonicalState` instance
containing the current state of the Wang-Landau run.
multiflip : ``bool`` (optional, default: ``False``)
If ``True``, ``multiflip_mcmc_sweep()`` will be used, otherwise
......@@ -613,7 +613,7 @@ class ExhaustiveSweepState(ABC):
entropy_args : ``dict`` (optional, default: ``{}``)
Entropy arguments, with the same meaning and defaults as in
callback : callable object (optional, default: ``None``)
Function to be called for each partition, with three arguments ``(S,
S_min, b_min)`` corresponding to the the current entropy value, the
......@@ -570,7 +570,7 @@ class BlockState(MCMCState, MultiflipMCMCState, MultilevelMCMCState,
def get_block_state(self, b=None, vweight=False, **kwargs):
r"""Returns a :class:`~graph_tool.inference.blockmodel.BlockState` corresponding
r"""Returns a :class:`~graph_tool.inference.BlockState` corresponding
to the block graph (i.e. the blocks of the current state become the
nodes). The parameters have the same meaning as the in the
constructor. If ``vweight == True`` the nodes of the block state are
......@@ -756,7 +756,7 @@ class BlockState(MCMCState, MultiflipMCMCState, MultilevelMCMCState,
return self.b
def get_state(self):
"""Alias to :meth:`~graph_tool.inference.blockmodel.BlockState.get_blocks`."""
"""Alias to :meth:`~graph_tool.inference.BlockState.get_blocks`."""
return self.get_blocks()
def set_state(self, b):
......@@ -1122,7 +1122,7 @@ class BlockState(MCMCState, MultiflipMCMCState, MultilevelMCMCState,
def virtual_vertex_move(self, v, s, **kwargs):
r"""Computes the entropy difference if vertex ``v`` is moved to block ``s``. The
remaining parameters are the same as in
return self._state.virtual_move(int(v), self.b[v], s,
......@@ -1190,7 +1190,7 @@ class BlockState(MCMCState, MultiflipMCMCState, MultilevelMCMCState,
def get_move_prob(self, v, s, c=1., d=.1, reverse=False):
r"""Compute the log-probability of a move proposal for vertex ``v`` to block ``s``
according to sampling parameters ``c`` and ``d``, as obtained with
:meth:`graph_tool.inference.blockmodel.BlockState.sample_vertex_move`. If ``reverse
:meth:`graph_tool.inference.BlockState.sample_vertex_move`. If ``reverse
== True``, the reverse probability of moving the node back from block
``s`` to its current one is obtained.
......@@ -1215,7 +1215,7 @@ class BlockState(MCMCState, MultiflipMCMCState, MultilevelMCMCState,
(with missing edges added and spurious edges deleted).
The values in ``entropy_args`` are passed to
:meth:`~graph_tool.inference.blockmodel.BlockState.entropy()` to
:meth:`~graph_tool.inference.BlockState.entropy()` to
calculate the log-probability.
......@@ -1371,7 +1371,7 @@ class BlockState(MCMCState, MultiflipMCMCState, MultilevelMCMCState,
the endpoints of each node have been assigned to a given block pair.
This should be called multiple times, e.g. after repeated runs of the
:meth:`graph_tool.inference.blockmodel.BlockState.mcmc_sweep` function.
:meth:`graph_tool.inference.BlockState.mcmc_sweep` function.
......@@ -1422,7 +1422,7 @@ class BlockState(MCMCState, MultiflipMCMCState, MultilevelMCMCState,
node was assigned to a given block.
This should be called multiple times, e.g. after repeated runs of the
:meth:`graph_tool.inference.blockmodel.BlockState.mcmc_sweep` function.
:meth:`graph_tool.inference.BlockState.mcmc_sweep` function.
......@@ -1488,11 +1488,11 @@ class BlockState(MCMCState, MultiflipMCMCState, MultilevelMCMCState,
r"""Collect a histogram of partitions.
This should be called multiple times, e.g. after repeated runs of the
:meth:`graph_tool.inference.blockmodel.BlockState.mcmc_sweep` function.
:meth:`graph_tool.inference.BlockState.mcmc_sweep` function.
h : :class:`~graph_tool.inference.blockmodel.PartitionHist` (optional, default: ``None``)
h : :class:`~graph_tool.inference.PartitionHist` (optional, default: ``None``)
Partition histogram. If not provided, an empty histogram will be created.
update : float (optional, default: ``1``)
Each call increases the current count by the amount given by this
......@@ -1503,7 +1503,7 @@ class BlockState(MCMCState, MultiflipMCMCState, MultilevelMCMCState,
h : :class:`~graph_tool.inference.blockmodel.PartitionHist` (optional, default: ``None``)
h : :class:`~graph_tool.inference.PartitionHist` (optional, default: ``None``)
Updated Partition histogram.
......@@ -1677,7 +1677,7 @@ def bethe_entropy(g, p):
Vertex property map with vector-type values, storing the accumulated
block membership counts. These are the node marginals, as would be
returned by the
......@@ -1754,7 +1754,7 @@ def microstate_entropy(h, unlabel=True):
h : :class:`~graph_tool.inference.blockmodel.PartitionHist` (optional, default: ``None``)
h : :class:`~graph_tool.inference.PartitionHist` (optional, default: ``None``)
Partition histogram.
unlabel : bool (optional, default: ``True``)
If ``True``, a canonical labelling of the groups will be used, so that
......@@ -38,7 +38,7 @@ class EMBlockState(object):
Graph to be modelled.
B : ``int``
Number of blocks (or vertex groups).
init_state : :class:`~graph_tool.inference.blockmodel.BlockState` (optional, default: ``None``)
init_state : :class:`~graph_tool.inference.BlockState` (optional, default: ``None``)
Optional block state used for initialization.
......@@ -214,7 +214,7 @@ def em_infer(state, max_iter=1000, max_e_iter=1, epsilon=1e-3,
state : model state
State object, e.g. of type :class:`graph_tool.inference.blockmodel_em.EMBlockState`.