diff --git a/README.md b/README.md new file mode 100644 index 0000000000000000000000000000000000000000..eb4dc79d5c0d878ad73f902426e640c7e21d51ef --- /dev/null +++ b/README.md @@ -0,0 +1,60 @@ +# Netzschleuder + +Netzschleuder is a network data catalogue and repository. + +The code here can be used to fully replicate the 'official' instance at +, including the upstream data downloading, +parsing and processing. + +## Dependencies + +The following Python packages are needed to use this code: + +* [graph-tool](https://graph-tool.skewed.de) +* [flask](https://flask.palletsprojects.com) +* [pillow](https://pillow.readthedocs.io) +* [google-api-python-client](https://github.com/googleapis/google-api-python-client) +* [google-auth-oauthlib](https://google-auth-oauthlib.readthedocs.io) +* [python-zstandard](https://python-zstandard.readthedocs.io) +* [openpyxl](https://openpyxl.readthedocs.io) +* [pandas](https://pandas.pydata.org) + +To render the website you need the following stored inside the +``./static`` directory: + +* [ink](http://ink.sapo.pt) +* [mathjax](https://www.mathjax.org) + +## Usage + +The full repository is populated by running simply: +``` +./do_all.py +``` + +The above command takes an optional space-separate list of entries as +arguments, in which case only the specified entries are processed. + +The processing pipeline consists of the steps ``fetch``, ``parse``, +``convert``, ``analyze``, and ``draw`` taken in sequence. Each of these +steps can also be run in isolation by running the corresponding python +script, e.g. ``fetch.py``. + +The processed results will be stored inside a ``cache`` sub-directory. A +local version of the website can be run by evoking ``./net_site.py``. + +### Google drive authentication + +Some entries require the use of google drive API for fetching. You will +need authenticate with a google account the first time you run the above +scripts, and your credentials will be stored in a local file. + +## License + +Netzschleuder is free software, you can redistribute it and/or modify it +under the terms of the GNU General Public License, version 3 or +above. See COPYING for details. + +This program is distributed in the hope that it will be useful, but +WITHOUT ANY WARRANTY; without even the implied warranty of +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.