3 d()@sdZddlZddlZddlZddlZddlZddlZddlZejZ Gddde Z Gddde Z Gddde Z Gd d d e Zdd dZdS)zb Fast cryptographic hash of Python objects, with a special case for fast hashing of numpy arrays. Nc@seZdZdZddZdS)_ConsistentSetza Class used to ensure the hash of Sets is preserved whatever the order of its items. c CsByt||_Wn.ttjfk r<tdd|D|_YnXdS)Ncss|]}t|VqdS)N)hash).0er0/tmp/pip-build-v9q4h5k9/joblib/joblib/hashing.py %sz*_ConsistentSet.__init__..)sortedZ _sequence TypeErrordecimalInvalidOperation)selfZ set_sequencerrr__init__sz_ConsistentSet.__init__N)__name__ __module__ __qualname____doc__rrrrrrsrc@seZdZdZddZdS)_MyHashz7 Class used to hash objects that won't normally pickle cGs ||_dS)N)args)r rrrrr+sz_MyHash.__init__N)rrrrrrrrrr(src@seZdZdZdddZdddZdd Zd d Zd ej fd dZ e j j Z e e ee<e e ee<e e ee <e e eej<ddZddZee ee<d S)HasherzW A subclass of pickler, to do cryptographic hashing, rather than pickling. md5cCs0tj|_d}tj||j|dtj||_dS)N)protocol)ioBytesIOstreamPicklerrhashlibnew_hash)r hash_namerrrrr4s zHasher.__init__TcCsvy|j|Wn>tjk rL}z |jd||ff7_WYdd}~XnX|jj}|jj||rr|jjSdS)Nz"PicklingError while hashing %r: %r) dumppickle PicklingErrorrrgetvaluerupdate hexdigest)r objZ return_digestrdumpsrrrr=s  z Hasher.hashcCst|tjtijfr|t|dr*|jj}n|j}|j}t|tt krTt ||j}n(|dkrht ||}n|jj }t |||}t j ||dS)N__func__) isinstancetypes MethodTypetypepophasattrr)r__self__r"r __class__rsave)r r' func_nameinstclsrrrr2Hs    z Hasher.savecCs"t|ttfrdStj||dS)N)r*bytesstrrmemoize)r r'rrrr8[szHasher.memoizeNc Cst||d}|d=ytj||f|Wnltjk rtj||f|t|dd}|dkr|}|dkrn|j}tj|}t ||st |||YnXdS)N)namepackr:r__main__) dictr save_globalr"r#getattrrsysmodulesr/setattr)r r'r9r:kwargsmoduleZmy_namemodrrrr=fs    zHasher.save_globalc CsPytj|tt|Wn2tk rJtj|ttdd|DYnXdS)Ncss|]\}}t||fVqdS)N)r)rkvrrrrsz)Hasher._batch_setitems..)r_batch_setitemsiterr r )r itemsrrrrGs zHasher._batch_setitemscCstj|t|dS)N)rr2r)r Z set_itemsrrrsave_setszHasher.save_set)r)T)rrrrrrr2r8structr:r=rdispatchcopyr-lenobjectr"r!rGrJsetrrrrr/s      rc@s"eZdZdZd ddZddZdS) NumpyHasherz7 Special case the hasher for when numpy is loaded. rFcCs@||_tj||dddl}||_t|dr6|j|_nt|_dS)a Parameters ---------- hash_name: string The hash algorithm to be used coerce_mmap: boolean Make no difference between np.memmap and np.ndarray objects. )r rN getbuffer) coerce_mmaprrnumpynpr/rR _getbuffer memoryview)r r rSrUrrrrs   zNumpyHasher.__init__cCst||jjr|jj r|jfkr,|j}n&|jjr:|}n|jj rJ|j }n|j}|j j |j |j|jj|jrt||jjr|jj}n|j}|d|j|j|jff}n6t||jjr|j j djd|j j tj|dStj||dS)z Subclass the save method, to hash ndarray subclass, rather than pickling them. Off course, this is a total abuse of the Pickler class. ZHASHEDZ _HASHED_DTYPEzutf-8N)r*rUZndarrayZdtypeZ hasobjectshapeflattenflags c_contiguous f_contiguousTrr%rVviewZuint8rSZmemmapr1stridesencoder"r(rr2)r r'Zobj_c_contiguousklassrrrr2s&   zNumpyHasher.saveN)rF)rrrrrr2rrrrrQs rQrFcCsHd}||krtdj||dtjkr4t||d}n t|d}|j|S)al Quick calculation of a hash to identify uniquely Python objects containing numpy arrays. Parameters ----------- hash_name: 'md5' or 'sha1' Hashing algorithm used. sha1 is supposedly safer, but md5 is faster. coerce_mmap: boolean Make no difference between np.memmap and np.ndarray rsha1zAValid options for 'hash_name' are {}. Got hash_name={!r} instead.rT)r rS)r )rrb) ValueErrorformatr?r@rQrr)r'r rSZvalid_hash_namesZhasherrrrrs    r)rF)rr"rr?r+rKrr _PicklerrrOrrrrQrrrrrskZ