Subclass object of pathlib.Path gets custom attributes lost after pickle.load

from pathlib import Path
import pickle
class P(type(Path())):
    def __init__(self, *args):
        super().__init__()
        self.a = ''
p = P()
p.a = 'x'
with open('xx', 'wb') as wf:
    pickle.dump(p, wf)
p1 = pickle.load(open('xx', 'rb'))
print(p1.a)            # here p1.a is ''

I am making a subclass of pathlib.Path and like to add some custom attributes to it. The problem is that the custom attribute gets lost after reloaded by pickle. How to solve this problem.

Other solutions I tried:

  • use __slots__, the same issue.
  • use composition instead of inheritance, then dispatch Path-like methods by implementing __getattr__. However, in this case, self.path is not initialized within pickle.load thus causing an endless call of __getattr__.
class File():
    def __init__(self, *args):
        self.path = Path(*args)
    def __getattr__(self, item):
        return getattr(self.path, item)
p = File('aaa')
p.exits()  # no error
with open('xx', 'wb') as wf:
    pickle.dump(p, wf)
p1 = pickle.load(open('xx', 'rb')) 
# RecursionError: maximum recursion depth exceeded. 
# This is due to call of self.path, in that moment, path is not in self.__dict__

Answer

One way to do it would be by using the copyreg module to associate a pickle support function to instance of your class as shown below. Note I also had to modify how your P class handles arguments — it no longer ignores them.

import copyreg
from pathlib import Path
import pickle


class P(type(Path())):
    def __init__(self, *args):
        super().__init__()
        self.a = args[0] if args else ''


def pickle_P(p):
    print("pickling a P instance...")
    return P, (p.a,)

copyreg.pickle(P, pickle_P)

p = P()
p.a = 'x'
q = P('y')

with open('xx', 'wb') as outp:
    pickle.dump(p, outp)
    pickle.dump(q, outp)

with open('xx', 'rb') as inp:
    p1 = pickle.load(inp)
    q1 = pickle.load(inp)

print('p1.a = {!r}'.format(p1.a))
print('q1.a = {!r}'.format(q1.a))

Output:

pickling a P instance...
pickling a P instance...
p1.a = 'x'
q1.a = 'y'

Leave a Reply

Your email address will not be published. Required fields are marked *