3 d|@sdZddlZddlZddlmZddlmZddlmZddl m Z GdddZ dd d Z Gd d d e ZddZdddZddZddZddZdZdZdZedkredS)z Tools for reading and writing dependency trees. The input is assumed to be in Malt-TAB format (https://stp.lingfil.uu.se/~nivre/research/MaltXML.html). N) defaultdict)chain)pformat)Treec@seZdZdZd:ddZddZd d Zd d Zd dZddZ ddZ ddZ ddZ ddZ ddZed;ddZddZdd Zd!d"Zdd,d-Zd.d/Zd0d1Zd2d3Zd4d5Zd6d7Zd8d9ZdS)?DependencyGraphzQ A container for the nodes and labelled edges of a dependency structure. NFROOTcCsHtdd|_|jdjddddd|_|rD|j|||||ddS)aDependency graph. We place a dummy `TOP` node with the index 0, since the root node is often assigned 0 as its head. This also means that the indexing of the nodes corresponds directly to the Malt-TAB format, which starts at 1. If zero-based is True, then Malt-TAB-like input with node numbers starting at 0 and the root node assigned -1 (as produced by, e.g., zpar). :param str cell_separator: the cell separator. If not provided, cells are split by whitespace. :param str top_relation_label: the label by which the top relation is identified, for examlple, `ROOT`, `null` or `TOP`. c Ssdddddddttdd S)N) addresswordlemmactagtagfeatsheaddepsrel)rlistrr:/tmp/pip-build-v9q4h5k9/nltk/nltk/parse/dependencygraph.py<sz*DependencyGraph.__init__..rTOP)r r rN)cell_extractor zero_basedcell_separatortop_relation_label)rnodesupdateroot_parse)selftree_strrrrrrrr__init__#s zDependencyGraph.__init__cCs |j|=dS)zw Removes the node with the given address. References to this node in others will still exist. N)r)rrrrrremove_by_addressVsz!DependencyGraph.remove_by_addresscCsTxN|jjD]@}g}x.|dD]"}||kr6|j|q|j|qW||d<q WdS)zp Redirects arcs to any of the nodes in the originals list to the redirect node address. rN)rvaluesappend)rZ originalsredirectnodeZnew_depsdeprrr redirect_arcs]s zDependencyGraph.redirect_arcscCs@|j|d}|j|dj|g|j|d|j|dS)zw Adds an arc from the node specified by head_address to the node specified by the mod address. rrN)r setdefaultr#)rZ head_addressZ mod_addressrelationrrradd_arckszDependencyGraph.add_arccCsvxp|jjD]b}x\|jjD]N}|d|dkr|ddkr|d}|dj|g|d|j|dqWq WdS)zr Fully connects all non-root nodes. All nodes are set to be dependents of the root node. rrrrN)rr"r(r#)rnode1node2r)rrr connect_graphus zDependencyGraph.connect_graphcCs |j|S)z'Return the node with the given address.)r)r node_addressrrrget_by_addressszDependencyGraph.get_by_addresscCs ||jkS)zq Returns true if the graph contains a node with the given node address, false otherwise. )r)rr.rrrcontains_addresssz DependencyGraph.contains_addresscCsd}|d7}|d7}xt|jjdddD]~}|dj|d|d|d 7}xZ|d jD]J\}}x@|D]8}|d k r|d j|d||7}qj|d j|d|7}qjWq\Wq,W|d7}|S)aReturn a dot representation suitable for using with Graphviz. >>> dg = DependencyGraph( ... 'John N 2\n' ... 'loves V 0\n' ... 'Mary N 2' ... ) >>> print(dg.to_dot()) digraph G{ edge [dir=forward] node [shape=plaintext] 0 [label="0 (None)"] 0 -> 2 [label="ROOT"] 1 [label="1 (John)"] 2 [label="2 (loves)"] 2 -> 1 [label=""] 2 -> 3 [label=""] 3 [label="3 (Mary)"] } z digraph G{ zedge [dir=forward] znode [shape=plaintext] cSs|dS)Nrr)vrrrrsz(DependencyGraph.to_dot..)keyz {} [label="{} ({})"]rr rNz {} -> {} [label="{}"]z {} -> {} z })sortedrr"formatitems)rsr%rrr&rrrto_dots   zDependencyGraph.to_dotcCs|j}t|S)a7Show SVG representation of the transducer (IPython magic). >>> dg = DependencyGraph( ... 'John N 2\n' ... 'loves V 0\n' ... 'Mary N 2' ... ) >>> dg._repr_svg_().split('\n')[0] '' )r7dot2img)r dot_stringrrr _repr_svg_s zDependencyGraph._repr_svg_cCs t|jS)N)rr)rrrr__str__szDependencyGraph.__str__cCsdt|jdS)Nz)lenr)rrrr__repr__szDependencyGraph.__repr__c s4t|"}fdd|jjdDSQRXdS)a :param filename: a name of a file in Malt-TAB format :param zero_based: nodes in the input file are numbered starting from 0 rather than 1 (as produced by, e.g., zpar) :param str cell_separator: the cell separator. If not provided, cells are split by whitespace. :param str top_relation_label: the label by which the top relation is identified, for examlple, `ROOT`, `null` or `TOP`. :return: a list of DependencyGraphs csg|]}t|dqS))rrr)r).0r)rrrrr sz(DependencyGraph.load..z N)openreadsplit)filenamerrrinfiler)rrrrloads zDependencyGraph.loadcs<tj|j|dj}|j|dtfdd|DS)zl Returns the number of left children under the node specified by the given address. rrc3s|]}|krdVqdS)Nr)r>c)indexrr sz0DependencyGraph.left_children..)r from_iterablerr"sum)r node_indexchildrenr)rHr left_childrenszDependencyGraph.left_childrencs<tj|j|dj}|j|dtfdd|DS)zm Returns the number of right children under the node specified by the given address. rrc3s|]}|krdVqdS)rFNr)r>rG)rHrrrIsz1DependencyGraph.right_children..)rrJrr"rK)rrLrMr)rHrright_childrenszDependencyGraph.right_childrencCs&|j|ds"|j|dj|dS)Nr)r0rr)rr%rrradd_nodeszDependencyGraph.add_nodec!Csdd}dd}dd}dd} |||| d } t|trLd d |jd D}d d |D} dd | D} d} xRt| ddD]@\} }|j|}| dkrt|} n| t|kst|dkry | | }Wn2tk r}ztdj| |WYdd}~XnXy||| \} }}}}}}}Wn0t tfk rH||\}}}}}}}YnX|dkrVq|t |}|rl|d7}|j | j | |||||||d| dkr|dkr|}|j |d|j | q|W|j dd|r|j dd|d}|j ||_||_n tjddS)aParse a sentence. :param extractor: a function that given a tuple of cells returns a 7-tuple, where the values are ``word, lemma, ctag, tag, feats, head, rel``. :param str cell_separator: the cell separator. If not provided, cells are split by whitespace. :param str top_relation_label: the label by which the top relation is identified, for examlple, `ROOT`, `null` or `TOP`. cSs|\}}}|||||d|dfS)Nr)cellsrHr r rrrrextract_3_cellss z/DependencyGraph._parse..extract_3_cellscSs |\}}}}|||||d||fS)NrQr)rRrHr r rrrrrextract_4_cellss z/DependencyGraph._parse..extract_4_cellsc SsH|\}}}}}}}y t|}Wntk r2YnX|||||d||fS)NrQ)int ValueError) rRrH line_indexr r r _rrrrrextract_7_cellss  z/DependencyGraph._parse..extract_7_cellsc SsN|\ }}}}}}}} } } y t|}Wntk r8YnX|||||||| fS)N)rUrV) rRrHrWr r r r r rrrXrrrextract_10_cells's  z0DependencyGraph._parse..extract_10_cells) css|] }|VqdS)Nr)r>linerrrrI8sz)DependencyGraph._parse.. css|]}|jVqdS)N)rstrip)r>lrrrrI:scss|]}|r|VqdS)Nr)r>rbrrrrI;sNrF)startzTNumber of tab-delimited fields ({}) not supported by CoNLL(10) or Malt-Tab(4) formatrX)rr r r r r rrr[rrzBThe graph doesn't contain a node that depends on the root element.) isinstancestrrB enumerater<AssertionErrorKeyErrorrVr4 TypeErrorrUrrr#rrwarningswarn)rZinput_rrrrrSrTrYrZZ extractorslinesZ cell_numberrHr_rRer r r r r rrZ root_addressrrrrsl          zDependencyGraph._parseTcCs|d}|r|dkr|S|S)Nr ,r)rr%filterwrrr_wordzs zDependencyGraph._wordcsLj|}|d}ttj|dj}|rDt|fdd|DS|SdS)zTurn dependency graphs into NLTK trees. :param int i: index of a node :return: either a word (if the indexed node is a leaf) or a ``Tree``. r rcsg|]}j|qSr)_tree)r>r&)rrrr?sz)DependencyGraph._tree..N)r/r3rrJr"r)rir%r rr)rrrrs  zDependencyGraph._treecs<j}|d}ttj|dj}t|fdd|DS)z Starting with the ``root`` node, build a dependency tree using the NLTK ``Tree`` constructor. Dependency labels are omitted. r rcsg|]}j|qSr)rr)r>r&)rrrr?sz(DependencyGraph.tree..)rr3rrJr"r)rr%r rr)rrtreeszDependencyGraph.treeccsv|s |j}|d|df}xVttj|djD]<}|j|}||d|d|dffV|j|dEdHq2WdS)zs Extract dependency triples of the form: ((head word, head tag), rel, (dep word, dep tag)) r r rr)r%N)rr3rrJr"r/triples)rr%rrsr&rrrrus zDependencyGraph.triplesc Cs(y|j|dStk r"dSXdS)Nr)r IndexError)rrsrrr_hdszDependencyGraph._hdc Cs(y|j|dStk r"dSXdS)Nr)rrv)rrsrrr_relszDependencyGraph._relc Csi}x<|jjD].}x(|dD]}t|d|g}d||<qWqWx|jD]}i}xR|D]J}xD|D]<}|d|dkrbt|d|dg}||||||<qbWqXWxF|D]>} || || <| d| dkr|j|j| d| d} | SqWqJWdS)aECheck whether there are cycles. >>> dg = DependencyGraph(treebank_data) >>> dg.contains_cycle() False >>> cyclic_dg = DependencyGraph() >>> top = {'word': None, 'deps': [1], 'rel': 'TOP', 'address': 0} >>> child1 = {'word': None, 'deps': [2], 'rel': 'NTOP', 'address': 1} >>> child2 = {'word': None, 'deps': [4], 'rel': 'NTOP', 'address': 2} >>> child3 = {'word': None, 'deps': [1], 'rel': 'NTOP', 'address': 3} >>> child4 = {'word': None, 'deps': [3], 'rel': 'NTOP', 'address': 4} >>> cyclic_dg.nodes = { ... 0: top, ... 1: child1, ... 2: child2, ... 3: child3, ... 4: child4, ... } >>> cyclic_dg.root = top >>> cyclic_dg.contains_cycle() [3, 1, 2, 4] rrrFrF)rr"tupleget_cycle_pathr/) rZ distancesr%r&r2rXZ new_entriesZpair1Zpair2pairpathrrrcontains_cycles$      zDependencyGraph.contains_cyclecCslx"|dD]}||kr |dgSq WxB|dD]6}|j|j||}t|dkr.|jd|d|Sq.WgS)Nrrr)rzr/r<insert)rZ curr_nodeZgoal_node_indexr&r|rrrrzs zDependencyGraph.get_cycle_pathcsZ|dkrdn*|dkrdn|dkr*dntdj|djfd d t|jjDS) z The dependency graph in CoNLL format. :param style: the style to use for the format (3, 4, 10 columns) :type style: int :rtype: str r[z{word} {tag} {head} r\z{word} {tag} {head} {rel} r^z9{i} {word} {lemma} {ctag} {tag} {feats} {head} {rel} _ _ zTNumber of tab-delimited fields ({}) not supported by CoNLL(10) or Malt-Tab(4) formatrQc3s2|]*\}}|ddkrjfd|i|VqdS)r rrsN)r4)r>rsr%)templaterrrI sz+DependencyGraph.to_conll..)rVr4joinr3rr5)rstyler)rrto_conlls  zDependencyGraph.to_conllcsvddl}ttdtj}fdd|D}i_x |D]}j|dj|<q:W|j}|j||j||S)zJConvert the data in a ``nodelist`` into a networkx labeled directed graph.rNrFcs,g|]$}j|r|j|j|fqSr)rwrx)r>n)rrrr?sz,DependencyGraph.nx_graph..r ) networkxrranger<r nx_labelsZ MultiDiGraphZadd_nodes_fromZadd_edges_from)rrZ nx_nodelistZ nx_edgelistrgr)rrnx_graphs   zDependencyGraph.nx_graph)NNFNr)FNr)NFNr)T)N)__name__ __module__ __qualname____doc__r r!r'r*r-r/r0r7r:r;r= staticmethodrErNrOrPrrqrrrtrurwrxr}rzrrrrrrrsF -  -    t  2 rsvgcCsddlm}m}y tjdd|g|||dd}Wn,tk r\}ztd|WYdd}~XnX|j|j}}|r~td j||S) aB Create image representation fom dot_string, using the 'dot' program from the Graphviz package. Use the 't' argument to specify the image file format, for ex. 'png' or 'jpeg' (Running 'dot -T:' lists all available formats). sys.stdout is used instead of subprocess.PIPE, to avoid decoding errors r)stderrstdoutdotz-T%sT)inputrrtextz0Cannot find the dot binary from Graphviz packageNzACannot create image representation by running dot from string: {})sysrr subprocessrunOSError Exceptionr4)r9trrprocrmouterrrrrr8$s   r8c@seZdZdZdS)DependencyGraphErrorzDependency graph exception.N)rrrrrrrrrCsrcCsttttdS)N) malt_demo conll_democonll_file_democycle_finding_demorrrrdemoGsrFcCstd}|j}|j|rddl}ddlm}|j}|j|j|dd}|j ||dd|j |||j |j g|j g|jd |jdS) zw A demonstration of the result of reading a dependency version of the first sentence of the Penn Treebank. aPierre NNP 2 NMOD Vinken NNP 8 SUB , , 2 P 61 CD 5 NMOD years NNS 6 AMOD old JJ 2 NMOD , , 2 P will MD 0 ROOT join VB 8 VC the DT 11 NMOD board NN 9 OBJ as IN 9 VMOD a DT 15 NMOD nonexecutive JJ 15 NMOD director NN 12 PMOD Nov. NNP 9 VMOD 29 CD 16 NMOD . . 9 VMOD rN)pylabrF)Zdim2)Z node_sizeztree.png)rrtpprintrZ matplotlibrrinfoZ spring_layoutZdraw_networkx_nodesZdraw_networkx_labelsrZxticksZyticksZsavefigshow)ZnxdgrtrrrposrrrrNs     rcCs2tt}|j}|jt|t|jddS)zg A demonstration of how to read a string representation of a CoNLL format dependency tree. r\N)r conll_data1rtrprintr)rrtrrrr{s rcCsFtdddtjdD}x$|D]}|j}td|jq"WdS)NzMass conll_read demo...cSsg|]}|rt|qSr)r)r>entryrrrr?sz#conll_file_demo..z r`)r conll_data2rBrtr)Zgraphsgraphrtrrrrs  rcCstt}t|jt}|jddgddd|jddgddd|jddgddd|jddgddd|jddgdddt|jdS) NrFrr)r rrrZNTOPr\r[)r treebank_datarr}rP)rZ cyclic_dgrrrrs raPierre NNP 2 NMOD Vinken NNP 8 SUB , , 2 P 61 CD 5 NMOD years NNS 6 AMOD old JJ 2 NMOD , , 2 P will MD 0 ROOT join VB 8 VC the DT 11 NMOD board NN 9 OBJ as IN 9 VMOD a DT 15 NMOD nonexecutive JJ 15 NMOD director NN 12 PMOD Nov. NNP 9 VMOD 29 CD 16 NMOD . . 9 VMOD a/ 1 Ze ze Pron Pron per|3|evofmv|nom 2 su _ _ 2 had heb V V trans|ovt|1of2of3|ev 0 ROOT _ _ 3 met met Prep Prep voor 8 mod _ _ 4 haar haar Pron Pron bez|3|ev|neut|attr 5 det _ _ 5 moeder moeder N N soort|ev|neut 3 obj1 _ _ 6 kunnen kan V V hulp|ott|1of2of3|mv 2 vc _ _ 7 gaan ga V V hulp|inf 6 vc _ _ 8 winkelen winkel V V intrans|inf 11 cnj _ _ 9 , , Punc Punc komma 8 punct _ _ 10 zwemmen zwem V V intrans|inf 11 cnj _ _ 11 of of Conj Conj neven 7 vc _ _ 12 terrassen terras N N soort|mv|neut 11 cnj _ _ 13 . . Punc Punc punt 12 punct _ _ a1 Cathy Cathy N N eigen|ev|neut 2 su _ _ 2 zag zie V V trans|ovt|1of2of3|ev 0 ROOT _ _ 3 hen hen Pron Pron per|3|mv|datofacc 2 obj1 _ _ 4 wild wild Adj Adj attr|stell|onverv 5 mod _ _ 5 zwaaien zwaai N N soort|mv|neut 2 vc _ _ 6 . . Punc Punc punt 5 punct _ _ 1 Ze ze Pron Pron per|3|evofmv|nom 2 su _ _ 2 had heb V V trans|ovt|1of2of3|ev 0 ROOT _ _ 3 met met Prep Prep voor 8 mod _ _ 4 haar haar Pron Pron bez|3|ev|neut|attr 5 det _ _ 5 moeder moeder N N soort|ev|neut 3 obj1 _ _ 6 kunnen kan V V hulp|ott|1of2of3|mv 2 vc _ _ 7 gaan ga V V hulp|inf 6 vc _ _ 8 winkelen winkel V V intrans|inf 11 cnj _ _ 9 , , Punc Punc komma 8 punct _ _ 10 zwemmen zwem V V intrans|inf 11 cnj _ _ 11 of of Conj Conj neven 7 vc _ _ 12 terrassen terras N N soort|mv|neut 11 cnj _ _ 13 . . Punc Punc punt 12 punct _ _ 1 Dat dat Pron Pron aanw|neut|attr 2 det _ _ 2 werkwoord werkwoord N N soort|ev|neut 6 obj1 _ _ 3 had heb V V hulp|ovt|1of2of3|ev 0 ROOT _ _ 4 ze ze Pron Pron per|3|evofmv|nom 6 su _ _ 5 zelf zelf Pron Pron aanw|neut|attr|wzelf 3 predm _ _ 6 uitgevonden vind V V trans|verldw|onverv 3 vc _ _ 7 . . Punc Punc punt 6 punct _ _ 1 Het het Pron Pron onbep|neut|zelfst 2 su _ _ 2 hoorde hoor V V trans|ovt|1of2of3|ev 0 ROOT _ _ 3 bij bij Prep Prep voor 2 ld _ _ 4 de de Art Art bep|zijdofmv|neut 6 det _ _ 5 warme warm Adj Adj attr|stell|vervneut 6 mod _ _ 6 zomerdag zomerdag N N soort|ev|neut 3 obj1 _ _ 7 die die Pron Pron betr|neut|zelfst 6 mod _ _ 8 ze ze Pron Pron per|3|evofmv|nom 12 su _ _ 9 ginds ginds Adv Adv gew|aanw 12 mod _ _ 10 achter achter Adv Adv gew|geenfunc|stell|onverv 12 svp _ _ 11 had heb V V hulp|ovt|1of2of3|ev 7 body _ _ 12 gelaten laat V V trans|verldw|onverv 11 vc _ _ 13 . . Punc Punc punt 12 punct _ _ 1 Ze ze Pron Pron per|3|evofmv|nom 2 su _ _ 2 hadden heb V V trans|ovt|1of2of3|mv 0 ROOT _ _ 3 languit languit Adv Adv gew|geenfunc|stell|onverv 11 mod _ _ 4 naast naast Prep Prep voor 11 mod _ _ 5 elkaar elkaar Pron Pron rec|neut 4 obj1 _ _ 6 op op Prep Prep voor 11 ld _ _ 7 de de Art Art bep|zijdofmv|neut 8 det _ _ 8 strandstoelen strandstoel N N soort|mv|neut 6 obj1 _ _ 9 kunnen kan V V hulp|inf 2 vc _ _ 10 gaan ga V V hulp|inf 9 vc _ _ 11 liggen lig V V intrans|inf 10 vc _ _ 12 . . Punc Punc punt 11 punct _ _ 1 Zij zij Pron Pron per|3|evofmv|nom 2 su _ _ 2 zou zal V V hulp|ovt|1of2of3|ev 7 cnj _ _ 3 mams mams N N soort|ev|neut 4 det _ _ 4 rug rug N N soort|ev|neut 5 obj1 _ _ 5 ingewreven wrijf V V trans|verldw|onverv 6 vc _ _ 6 hebben heb V V hulp|inf 2 vc _ _ 7 en en Conj Conj neven 0 ROOT _ _ 8 mam mam V V trans|ovt|1of2of3|ev 7 cnj _ _ 9 de de Art Art bep|zijdofmv|neut 10 det _ _ 10 hare hare Pron Pron bez|3|ev|neut|attr 8 obj1 _ _ 11 . . Punc Punc punt 10 punct _ _ 1 Of of Conj Conj onder|metfin 0 ROOT _ _ 2 ze ze Pron Pron per|3|evofmv|nom 3 su _ _ 3 had heb V V hulp|ovt|1of2of3|ev 0 ROOT _ _ 4 gewoon gewoon Adj Adj adv|stell|onverv 10 mod _ _ 5 met met Prep Prep voor 10 mod _ _ 6 haar haar Pron Pron bez|3|ev|neut|attr 7 det _ _ 7 vriendinnen vriendin N N soort|mv|neut 5 obj1 _ _ 8 rond rond Adv Adv deelv 10 svp _ _ 9 kunnen kan V V hulp|inf 3 vc _ _ 10 slenteren slenter V V intrans|inf 9 vc _ _ 11 in in Prep Prep voor 10 mod _ _ 12 de de Art Art bep|zijdofmv|neut 13 det _ _ 13 buurt buurt N N soort|ev|neut 11 obj1 _ _ 14 van van Prep Prep voor 13 mod _ _ 15 Trafalgar_Square Trafalgar_Square MWU N_N eigen|ev|neut_eigen|ev|neut 14 obj1 _ _ 16 . . Punc Punc punt 15 punct _ _ __main__)r)F)rrrj collectionsr itertoolsrrrZ nltk.treerrr8rrrrrrrrrrrrrrrs.      -  V