GitLab

The whole histogram code has been redone, and the code has been
simplified. The three-point vertex-edge-vertex correlation has been
scrapped, since it's not frequently used, and would make compilation
even more expensive.

This also adds some missing files to the generation routine.

It is now much simpler, and works better.

Now graphml files properly contain all the supported value types, which
are all perfectly preserved when read (floating point data is now saved
in hexadecimal format). Several other improvements were made, such as
the ability to read and write to python file-like objects.

It is also now possible to have arbitrary python object properties, and
store them persistently (which is done internally with the pickling
interface).

vector<bool> was totally abolished, since its implementation is quite
broken. See: http://www.gotw.ca/publications/N1211.pdf and
http://www.gotw.ca/publications/N1185.pdf Now a uint8_t (aka. char) is
used in graph properties instead of a bool.

Graph types can now be fully pickled (this may not be feasible
memory-wise if the graph is too large, since the whole XML
representation is dumped to a string before it is saved on disc).

This commit splits libraph_tool into different libraries:
 
   - libgraph_tool_core
   - libgraph_tool_clustering (*)
   - libgraph_tool_community (*)
   - libgraph_tool_correlations (*)
   - libgraph_tool_distance (*)
   - libgraph_tool_generation (*)
   - libgraph_tool_layout (*)
   - libgraph_tool_misc (*)
   - libgraph_tool_stats (*)

It also adds the python sub-module 'test', which provides extensive unit
testing of the core functionality. The core library is fully functional
and all test pass successfully.

(*) -> module needs to be ported to new refactoring, and does not yet build

This is a huge commit which completely refactors the metaprogramming
engine which generates and selects (at run time) the graph view type and
the desired algorithm implementation (template instantiation) that runs
on it.

Things are laid out now as following. There exists a main underlying
graph type (GraphInterface::multigraph_t) and several other template
classes that mask it some way or another, in a hierarchic fashion:

     multigraph_t -> filtered_graph (edges only, vertices only, both)
         |                               |           |           |
         |                               |           |           |
         |-------(reversed_graph)--------|-----------|-----------|
         |                               |           |           |
         \------(UndirectedAdaptor)------------------------------/

The filtered_graph filters out edges and/or vertices from the graph
based on some scalar boolean property. The reversed_graph reversed the
direction of the edges and, finally, the UndirectedAdaptor treats the
original directed graphs as undirected, transversing the in- and
out-edges of each vertex indifferently. Thus, the total number of graph
view types is 12. (The option --disable-graph-filtering can be passed to
the configure script, which will disable graph filtering altogether and
bring the total number down to 3, to reduce compile time and memory
usage)

In general, some specific algorithm, implemented as a template function
object, needs to be instantiated for each of those types. Furthermore,
the algorithm may also depend on other types, such as specific
property_maps. Thus, the following scheme is used:

    struct my_algorithm // algorithm to be implemented
    {
        template <class Graph, class PropertyMap>
        void operator()(Graph *g, PropertyMap p, double& result) const
        {
            // ...
        }
    };

    // in order for the above code to be instantiated at compile time
    // and selected at run time, the run_action template function object
    // is used from a member function of the GraphInterface class:

    double GraphInterface::MyAlgorithm(string prop_name)
    {
        double result;
        boost::any vprop = prop(property, _vertex_index, _properties);
        run_action<>()(*this, bind<void>(my_algorithm(), _1, _2,
                                         var(result)),
                       vertex_scalar_properties())(vprop);
        return result;
    }

The whole code was changed to reflect this scheme, but now things are
more centralized and less ad-hoc code needed to be
written. Unfortunately, due to GCC's high memory usage during template
instantiations, some of the code (namely all the degree correlation
things) had to be split in multiple compilation units... Maybe this will
change in the future if GCC gets optimized.

This commit also touches other parts of code. More specifically, the way
filtering gets done is very different. Now we only filter on boolean
properties, and with the above scheme, the desired implementation runs
with the correct chosen type, and no implicit type conversions should
ever happen, which would have a bad impact on performance.

Everything was moved into the graph_tool directory, which should have,
after installation, the following structure:

graph_tool/                          Top-level package
      __init__.py                    Initialize the graph_tool package
      libgraph_tool.so               libgraph_tool submodule
      libgraph_tool.la               libtool file for libgraph_tool
      include/                       C++ include files for dynamically
                                     compiled code

Some visibility voodoo is necessary to ensure that RTTI work properly
across DSO boundaries, which is necessary to properly handle dynamic
property maps and such. Additionally, the action template must have a
different name each time the action code changes, to avoid additional
weirdness.

This also adds some other utility functions/typedefs to make writing
"inline" c++ code easier.

It is now possible to run arbitrary "inline" C++ code from python, using
scipy.weave, as such:

        g = graph_tool.Graph()
        g.load("foo.xml")
        value = 2.0
        g.run_action('cout << num_vertices(g) << " " << value << endl;',
                     ["value"]);

The code gets compiled the first time, and is reused after that. Python
local and global variables can be passed to C++, as shown above, by
specifying a list of passed variables as the second argument.

The graph object passed to the edit function is now a instance of the
Graph class which wraps GraphInterface, not a GraphInterface instance.

Move remaining parallelism check out of the outer/inner loops and into the parallel_check() function.

3) missing parallel edge check in uncorrelated strategy
2) "_in()" for target and source of "target swap edge"
1) order of arguments of swap_edge_triad::parallel_check

melodrama time...
"premature optimization is the root of all evil"
"careless abstraction are the fruits of its tree"

The check for parallel edges was cleaned up and abstracted into a
function, and other minor things were changed. Code comments were added
to places where it might have been hard to understand.

Property maps can now be obtained as such:

         weight = g.edge_properties['weight']
         print weight[v] # v is a Vertex object
         weight[v] = 2.0
         # and so on...

The list of properties is obtained from g.vertex_properties,
g.edge_properties and g.graph_properties, which can be read as
dictionaries. The last can be set also, as such:

         g.graph_properties = {foo:"bar", baz:42}

Functions to add and remove vertices or adges were also added
(add_{vertex|edge}, remove_{vertex|edgge}).

Vertex and Edge types can also now be printed for convenience, as such:

       for v in g.vertices():
           print v
       for e in g.edges():
           print e

which results, for example, in:
0
1
2
3
4
5
6
(0,1)
(1,2)
(2,6)
(3,4)
(4,5)
(4,2)
(5,6)
(6,1)

(this also adds the forgotten graph_tool.py file, which was previously
on .gitignore)

Major changes to both correlated and uncorrelated rewiring code.
Both cases are now functional and use iterative shuffling.
Uncorrelated picks edges to rewire directly, correlated picks
vertices in order to choose edges.

Vertices and edges can be accessed from the graph class, as such:

    import graph_tool
    g = graph_tool.Graph()
    for v in g.vertices():
        for e in v.out_edges():
           # do something...

Additionally, the --edit-{vertex|edge|graph}-property was ported to the
new interface, and is working again, as it used to.

The Vertex and Edge class no longer have the 'get_property' and
'set_property' method. They'll be replaced by a new method of accessing
property maps.

This fixes a bug where properties with 'long long' types were not handled
properly through the python interface.

Several changes in the random rewiring code, making it fully
functional for the correlated case.
The reshuffling operations lag in efficiency, even though already
the algorithm runs in little time.

Line breaks at column 80 were added, and all trailing whitespace was deleted. Code
comments were modified and some more were added.

It should be 'size_t' instead of 'long'.

This fixes the same problem as commit 6663be71, but which arises
when a filter range boundary is -inf (such as "<=100").

    
When only one of vertex or edge filtering was disabled, the allowed
range of the disabled filter was set to
]numeric_limits<double>::min(), numeric_limits<double>::max()[, and
the selected filtering property was the respective index. But
according to the STL documentation from GCC,
numeric_limits<>::min() returns:
    
   "The minimum finite value, or for floating types with
    denormalization, the minimum positive normalized value."
    
which is always positive for double (!), thus introducing a weird
regression, where the first vertex (index 0) is always filtered out if
only the edge filter is active, and vice-versa.

It's still buggy (of course), but now we'll try to fix it in it's own
 branch.

This reverts commit 1cdbdeef.

The Histogram type 'hist_t' should have a 'double' value, so that
counts of 1/2 can be added when it's known things are going to be
counted twice.

(and I don't know what that 'double_t' thing was doing there)

Implements extended_clustering for directed graphs, based on the fraction of possible directed cycles (out-in edges combination).
The old behavior is still present since it was equivalent to treating the directed graph as undirected (use --undirected option).

    
Filtered vertices and edges can be permanently removed from the graph
with --purge-vertices and --purge-edges, respectively. The edge or
vertex filter is automatically removed, afterwards. This is useful if
maximum speed is necessary, and saving and reloading the graph without
filtering is not desired.
    
(this commit also removes some trailing whitespaces)

This will increase memory usage, but guarantees, across different
platforms, that 'long' will always hold larger values than 'int'.

Remove graph rewiring for now, since it's quite buggy. It will be
re-commited when ready.

Complete overhaul of command line parsing, and support for loading graph-tool as whole as a python module
    
The command line parsing was completely rewritten. It now supports
better parsing of sub-options, with type checking and grouping
support. Error reporting was also significantly improved, and it now
warns of invalid options and option values, before the option is
executed. Some syntax has changed, such as range filtering:
--[vertex|edge]-range-filter was replaced by
--exclude-[vertex|edge]-range and --keep-[vertex|edge]-range, which
should have a clearer meaning. Ranges can also be specified now by
comparison operators (>,<,>=,<=,=), such as ">=10", to indicate a
range of (10, inf). In addition, ranges can now be easily open or
closed at either end, by suffixing the specific end with '*', to
indicate it is closed, ex: "10 700*" means (10,700].
    
The graph-tool script can now be loaded as a python module (it must be
renamed first to 'something.py'). All the command line options (except
'for' and 'history' which become irrelevant) are available as
functions, with full description and optional parameter support. In
addition, pure function objects can be given as parameters where
expressions are asked, instead of strings and files, which enables
convenient extension of graph-tool.

for undirected graphs

Simplify range filtering, and definitely remove python filtering
    
Simplify range filtering of vertices and edges, by always filtering
both at once, even if all vertices or edges are being considered. This
severely reduces compilation time and memory, at a small potential
cost in run-time speed, which will probably be overshadowed by other
things, such as dynamic_map look-ups ("premature optimization is the
root of all evil"). Also, remove python-filtering, since, in the end,
it is just code bloat, since it is quite slow for most uses and can be
replaced, generally, by python property editing + range filtering.