3 Ud0 @s ddlmZddlZddlmZmZmZmZmZm Z m Z ddl Z ddl mZddlmZddlmZddlZddlmZdd Zdeeee ed d dZde eeefe e eefe e eee eeeffe ee eeee edd ddZededdZdS)) defaultdictN)Any DefaultDictDictIterableListOptionalUnion)convert_json_to_lines)Scalar) deprecate) DataFramecCs2|ddk r|ddkr|S|dd}t|S)zJ Helper function that converts JSON lists to line delimited JSON. r[]r)r )srM/var/www/html/virt/lib64/python3.6/site-packages/pandas/io/json/_normalize.pyconvert_to_line_delimitss r.)prefixseplevel max_levelc Csd}t|tr|g}d}g}x|D]}tj|}x|jD]\} } t| tsTt| } |dkrb| } n ||| } t| t s|dk r||kr|dkr:|j| } | || <q:q:|j| } |jt| | ||d|q:W|j |q"W|r|dS|S)a A simplified json_normalize Converts a nested dict into a flat dict ("record"), unlike json_normalize, it does not attempt to extract a subset of the data. Parameters ---------- ds : dict or list of dicts prefix: the prefix, optional, default: "" sep : str, default '.' Nested records will generate names separated by sep, e.g., for sep='.', { 'foo' : { 'bar' : 0 } } -> foo.bar level: int, optional, default: 0 The number of levels in the json string. max_level: int, optional, default: None The max depth to normalize. .. versionadded:: 0.25.0 Returns ------- d - dict or list of dicts, matching `ds` Examples -------- IN[52]: nested_to_record(dict(flat1=1,dict1=dict(c=1,d=2), nested=dict(e=dict(c=1,d=2),d=2))) Out[52]: {'dict1.c': 1, 'dict1.d': 2, 'flat1': 1, 'nested.d': 2, 'nested.e.c': 1, 'nested.e.d': 2} FTrNr) isinstancedictcopydeepcopyitemsstrpopupdatenested_to_recordappend) ZdsrrrrZ singletonZnew_dsdZnew_dkvZnewkeyrrrr$s2,        r$raiser ) data record_pathmeta meta_prefix record_prefixerrorsrrreturnc stttftttftttfdddtttftttftdfdd t|trf| rft St|t rv|g}|dkrt dd|Drt | d }t |St|ts|g}|dkrg}nt|ts|g}d d |Dg gt t fd d Dd f dd ||id dt } dk r\|j fddd}xZjD]N\} } |dk r|| } | |krtd| dtj| tdj|| <qfW|S)a Normalize semi-structured JSON data into a flat table. Parameters ---------- data : dict or list of dicts Unserialized JSON objects. record_path : str or list of str, default None Path in each object to list of records. If not passed, data will be assumed to be an array of records. meta : list of paths (str or list of str), default None Fields to use as metadata for each record in resulting table. meta_prefix : str, default None If True, prefix records with dotted (?) path, e.g. foo.bar.field if meta is ['foo', 'bar']. record_prefix : str, default None If True, prefix records with dotted (?) path, e.g. foo.bar.field if path to records is ['foo', 'bar']. errors : {'raise', 'ignore'}, default 'raise' Configures error handling. * 'ignore' : will ignore KeyError if keys listed in meta are not always present. * 'raise' : will raise KeyError if keys listed in meta are not always present. sep : str, default '.' Nested records will generate names separated by sep. e.g., for sep='.', {'foo': {'bar': 0}} -> foo.bar. max_level : int, default None Max number of levels(depth of dict) to normalize. if None, normalizes all levels. .. versionadded:: 0.25.0 Returns ------- frame : DataFrame Normalize semi-structured JSON data into a flat table. Examples -------- >>> data = [{'id': 1, 'name': {'first': 'Coleen', 'last': 'Volk'}}, ... {'name': {'given': 'Mose', 'family': 'Regner'}}, ... {'id': 2, 'name': 'Faye Raker'}] >>> pandas.json_normalize(data) id name name.family name.first name.given name.last 0 1.0 NaN NaN Coleen NaN Volk 1 NaN NaN Regner NaN Mose NaN 2 2.0 Faye Raker NaN NaN NaN NaN >>> data = [{'id': 1, ... 'name': "Cole Volk", ... 'fitness': {'height': 130, 'weight': 60}}, ... {'name': "Mose Reg", ... 'fitness': {'height': 130, 'weight': 60}}, ... {'id': 2, 'name': 'Faye Raker', ... 'fitness': {'height': 130, 'weight': 60}}] >>> json_normalize(data, max_level=0) fitness id name 0 {'height': 130, 'weight': 60} 1.0 Cole Volk 1 {'height': 130, 'weight': 60} NaN Mose Reg 2 {'height': 130, 'weight': 60} 2.0 Faye Raker Normalizes nested data up to level 1. >>> data = [{'id': 1, ... 'name': "Cole Volk", ... 'fitness': {'height': 130, 'weight': 60}}, ... {'name': "Mose Reg", ... 'fitness': {'height': 130, 'weight': 60}}, ... {'id': 2, 'name': 'Faye Raker', ... 'fitness': {'height': 130, 'weight': 60}}] >>> json_normalize(data, max_level=1) fitness.height fitness.weight id name 0 130 60 1.0 Cole Volk 1 130 60 NaN Mose Reg 2 130 60 2.0 Faye Raker >>> data = [{'state': 'Florida', ... 'shortname': 'FL', ... 'info': {'governor': 'Rick Scott'}, ... 'counties': [{'name': 'Dade', 'population': 12345}, ... {'name': 'Broward', 'population': 40000}, ... {'name': 'Palm Beach', 'population': 60000}]}, ... {'state': 'Ohio', ... 'shortname': 'OH', ... 'info': {'governor': 'John Kasich'}, ... 'counties': [{'name': 'Summit', 'population': 1234}, ... {'name': 'Cuyahoga', 'population': 1337}]}] >>> result = json_normalize(data, 'counties', ['state', 'shortname', ... ['info', 'governor']]) >>> result name population state shortname info.governor 0 Dade 12345 Florida FL Rick Scott 1 Broward 40000 Florida FL Rick Scott 2 Palm Beach 60000 Florida FL Rick Scott 3 Summit 1234 Ohio OH John Kasich 4 Cuyahoga 1337 Ohio OH John Kasich >>> data = {'A': [1, 2]} >>> json_normalize(data, 'A', record_prefix='Prefix.') Prefix.0 0 1 1 2 Returns normalized data with columns prefixed with the given string. )jsspecr0cSs2|}t|tr&x|D] }||}qWn||}|S)zInternal function to pull field)rlist)r1r2resultfieldrrr _pull_fields   z$_json_normalize.._pull_fieldcsB||}t|ts>tj|r$g}nt|d|d|d|S)z Internal function to pull field for records, and similar to _pull_field, but require to return list. And will raise error if has non iterable value. z has non list value z for path z. Must be list or null.)rr3pdZisnull TypeError)r1r2r4)r6rr _pull_recordss   z&_json_normalize.._pull_recordsNcss |]}dd|jDVqdS)cSsg|]}t|tqSr)rr).0xrrr sz-_json_normalize...N)values)r:yrrr sz"_json_normalize..)rrcSs g|]}t|tr|n|gqSr)rr3)r:mrrrr< sz#_json_normalize..csg|]}j|qSr)join)r:val)rrrr<'src svt|tr|g}t|dkrxj|D]b}x8tD]*\}}|dt|kr2||d ||<q2W||d|dd||ddq"Wnx|D]}||d} fdd|D}jt|xtD]\}}|dt|kr||}n`y|||d}WnHtk rP} z*dkr.tj}ntd| d| WYdd} ~ XnX|j|qW j|qWdS) Nrr)rcs(g|] }t|tr t|dn|qS))rr)rrr$)r:r)rrrrr<7sz?_json_normalize.._recursive_extract..ignorez(Try running with errors='ignore' as key z is not always presentr) rrlenzipr%KeyErrornpnanextend) r*pathZ seen_metarobjrBkeyZrecsZmeta_vale) _metar6r9_recursive_extractr/lengthsr meta_keys meta_valsrecordsrrrrP)s4   *     z+_json_normalize.._recursive_extract)rcs |S)Nr)r;)r.rrUsz!_json_normalize..)columnszConflicting metadata name z, need distinguishing prefix )Zdtype)r)rr!rr rr rrr3r ranyr$rrenamer ValueErrorrHarrayobjectrepeat) r*r+r,r-r.r/rrr4r'r(r) rOr6r9rPr/rQrrRrSr.rTrr_json_normalizepsFw &   "'   r]zpandas.io.json.json_normalizez1.0.0zpandas.json_normalize)rrrN)NNNNr)rN) collectionsrrtypingrrrrrrr ZnumpyrHZpandas._libs.writersr Zpandas._typingr Zpandas.util._decoratorsr Zpandasr7r rr!intr$r]Zjson_normalizerrrrs0 $    NRl