Pickle Objects
Hi, during my work with graph-tool, I experienced a problem with the graph.save() method. My vertexes have a vertex property of type python::object. Those objects can not be pickled with the standard pickle protocol (protocol 0). The reason for that is, that the objects include the "dnspython" package which uses sockets. So I get the following error stack trace:
Traceback (most recent call last):
File "Main.py", line 427, in <module>
Main().control()
File "Main.py", line 77, in control
self.saveGraph()
File "Main.py", line 409, in saveGraph
self.graph.save(input)
File "/usr/lib/python2.7/dist-packages/graph_tool/__init__.py", line 1362, in save
self.__graph.WriteToFile(file_name, None, file_format, props)
File "/usr/lib/python2.7/dist-packages/graph_tool/io.py", line 62, in pickler
pickle.dump(obj, sstream)
File "/usr/lib/python2.7/pickle.py", line 1370, in dump
Pickler(file, protocol).dump(obj)
File "/usr/lib/python2.7/pickle.py", line 224, in dump
self.save(obj)
File "/usr/lib/python2.7/pickle.py", line 331, in save
self.save_reduce(obj=obj, *rv)
File "/usr/lib/python2.7/pickle.py", line 419, in save_reduce
save(state)
File "/usr/lib/python2.7/pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "/usr/lib/python2.7/pickle.py", line 649, in save_dict
self._batch_setitems(obj.iteritems())
File "/usr/lib/python2.7/pickle.py", line 663, in _batch_setitems
save(v)
File "/usr/lib/python2.7/pickle.py", line 306, in save
rv = reduce(self.proto)
File "/usr/lib/python2.7/copy_reg.py", line 77, in _reduce_ex
raise TypeError("a class that defines __slots__ without "
TypeError: a class that defines __slots__ without defining __getstate__ cannot be pickled
I fixed that issue by changing the method "pickler" in io.py from:
def pickler(stream, obj):
sstream = StringIO()
pickle.dump(obj, sstream)
stream.write(base64.b64encode(sstream.getvalue()))
to:
def pickler(stream, obj):
sstream = StringIO()
pickle.dump(obj, sstream, -1)
stream.write(base64.b64encode(sstream.getvalue()))
Doing so, pickle will use the highest protocol available. Since this fixed the problem, I was thinking if this change to the method would hurt anything and if it would make sense, to include this change in the next release of graph-tool. What do you think? I'm not sure, if I might be missing something.
Hope to hear from you soon,
Felix