Commit 3decdbe5 authored by Tiago Peixoto's avatar Tiago Peixoto
Browse files

Add README.md

parent f77920c4
# Netzschleuder
Netzschleuder is a network data catalogue and repository.
The code here can be used to fully replicate the 'official' instance at
<https://networks.skewed.de>, including the upstream data downloading,
parsing and processing.
## Dependencies
The following Python packages are needed to use this code:
* [graph-tool](https://graph-tool.skewed.de)
* [flask](https://flask.palletsprojects.com)
* [pillow](https://pillow.readthedocs.io)
* [google-api-python-client](https://github.com/googleapis/google-api-python-client)
* [google-auth-oauthlib](https://google-auth-oauthlib.readthedocs.io)
* [python-zstandard](https://python-zstandard.readthedocs.io)
* [openpyxl](https://openpyxl.readthedocs.io)
* [pandas](https://pandas.pydata.org)
To render the website you need the following stored inside the
``./static`` directory:
* [ink](http://ink.sapo.pt)
* [mathjax](https://www.mathjax.org)
## Usage
The full repository is populated by running simply:
```
./do_all.py
```
The above command takes an optional space-separate list of entries as
arguments, in which case only the specified entries are processed.
The processing pipeline consists of the steps ``fetch``, ``parse``,
``convert``, ``analyze``, and ``draw`` taken in sequence. Each of these
steps can also be run in isolation by running the corresponding python
script, e.g. ``fetch.py``.
The processed results will be stored inside a ``cache`` sub-directory. A
local version of the website can be run by evoking ``./net_site.py``.
### Google drive authentication
Some entries require the use of google drive API for fetching. You will
need authenticate with a google account the first time you run the above
scripts, and your credentials will be stored in a local file.
## License
Netzschleuder is free software, you can redistribute it and/or modify it
under the terms of the GNU General Public License, version 3 or
above. See COPYING for details.
This program is distributed in the hope that it will be useful, but
WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment