3 d@@sddlZddlZddlZddlZddlZddlmZddlmZm Z m Z ddl m Z ddl mZddlmZddZd d Zd d ZGd dde ZedkrddlZejdS)N)ZipFilePathPointer)find_dir find_filefind_jars_within_path)ParserI)DependencyGraph)taggedsents_to_conllcCsRddlm}|d1d2d3d4d5d6d7d8d9d:d;dd?d@dAdBdCdDdEdFdGdHdIdJdKdLg}|jS)MNr) RegexpTagger\.$.\,$,\?$?\($(\)$)\[$[\]$]^-?[0-9]+(\.[0-9]+)?$CD(The|the|A|a|An|an)$DT&(He|he|She|she|It|it|I|me|Me|You|you)$PRP(His|his|Her|her|Its|its)$PRP$(my|Your|your|Yours|yours)$ (on|On|in|In|at|At|since|Since)$IN (for|For|ago|Ago|before|Before)$(till|Till|until|Until)$(by|By|beside|Beside)$(under|Under|below|Below)$(over|Over|above|Above)$ (across|Across|through|Through)$(into|Into|towards|Towards)$(onto|Onto|from|From)$.*able$JJ.*ness$NN.*ly$RB.*s$NNS.*ing$VBG.*ed$VBD.*)r r )r r )rr)rr)rr)rr)rr)rr)rr)rr)rr)r r)r!r")r#r")r$r")r%r")r&r")r'r")r(r")r)r")r*r")r+r,)r-r.)r/r0)r1r2)r3r4)r5r6)r7r.)Znltk.tagr tag)r Z_taggerr9//tmp/pip-build-v9q4h5k9/nltk/nltk/parse/malt.pymalt_regex_taggers> r;cCsxtjj|r|}n t|d d}dddg}tt|}dd|D}dddh}|j|sZttt d d |sptt |S) zE A module to find MaltParser .jar file and its dependencies. MALT_PARSER)env_varscSsh|]}tjj|dqS))ospathsplit).0jarr9r9r: Hsz"find_maltparser..z log4j.jarz libsvm.jarzliblinear-1.8.jarcSs|jdo|jdS)Nz maltparser-z.jar) startswithendswith)ir9r9r:Msz!find_maltparser..)r<) r@rAexistsrsetrissubsetAssertionErroranyfilterlist)parser_dirnameZ _malt_dirZmalt_dependenciesZ _malt_jarsZ_jarsr9r9r:find_maltparser=s     rRcCs.|dkr dStjj|r|St|dddSdS)z8 A module to find pre-trained MaltParser model. Nz malt_temp.mco MALT_MODELF)r=verbose)rS)r@rArJr)model_filenamer9r9r:find_malt_modelRs  rVc@sZeZdZdZdddZddd Zdd d Zdd d ZedddZ dddZ dddZ dS) MaltParsera A class for dependency parsing with MaltParser. The input is the paths to: - (optionally) a maltparser directory - (optionally) the path to a pre-trained MaltParser .mco model file - (optionally) the tagger to use for POS tagging before parsing - (optionally) additional Java arguments Example: >>> from nltk.parse import malt >>> # With MALT_PARSER and MALT_MODEL environment set. >>> mp = malt.MaltParser(model_filename='engmalt.linear-1.7.mco') # doctest: +SKIP >>> mp.parse_one('I shot an elephant in my pajamas .'.split()).tree() # doctest: +SKIP (shot I (elephant an) (in (pajamas my)) .) >>> # Without MALT_PARSER and MALT_MODEL environment. >>> mp = malt.MaltParser('/home/user/maltparser-1.9.2/', '/home/user/engmalt.linear-1.7.mco') # doctest: +SKIP >>> mp.parse_one('I shot an elephant in my pajamas .'.split()).tree() # doctest: +SKIP (shot I (elephant an) (in (pajamas my)) .) r>NcCsTt||_|dk r|ng|_t||_|jdk|_tj|_|dk rH|nt |_ dS)a An interface for parsing with the Malt Parser. :param parser_dirname: The path to the maltparser directory that contains the maltparser-1.x.jar :type parser_dirname: str :param model_filename: The name of the pre-trained model with .mco file extension. If provided, training will not be required. (see http://www.maltparser.org/mco/mco.html and see http://www.patful.com/chalk/node/185) :type model_filename: str :param tagger: The tagger used to POS tag the raw string before formatting to CONLL format. It should behave like `nltk.pos_tag` :type tagger: function :param additional_java_args: This is the additional Java arguments that one can use when calling Maltparser, usually this is the heapsize limits, e.g. `additional_java_args=['-Xmx1024m']` (see https://goo.gl/mpDBvQ) :type additional_java_args: list Nz malt_temp.mco) rR malt_jarsadditional_java_argsrVmodel_trainedtempfile gettempdir working_dirr;tagger)selfrQrUr_rYr9r9r:__init__rs     zMaltParser.__init__Fnullc !csP|jstdtjd|jddd }tjd|jddd}xt|D]}|jt|qFW|j|j |j |j dd}t j }yt j t jj|jd Wn YnX|j||} t j || d krtd d j|| ft|j 2} x*| jjd D]} tt| |d gVqWWdQRXWdQRXWdQRXt j|j t j|j dS)a Use MaltParser to parse multiple POS tagged sentences. Takes multiple sentences where each sentence is a list of (word, tag) tuples. The sentences must have already been tokenized and tagged. :param sentences: Input sentences to parse :type sentence: list(list(tuple(str, str))) :return: iter(iter(``DependencyGraph``)) the dependency graph representation of each sentence z0Parser has not been trained. Call train() first.zmalt_input.conll.wF)prefixdirmodedeletezmalt_output.conll.parse)rfrz0MaltParser parsing (%s) failed with exit code %d z )top_relation_labelN)r[ Exceptionr\NamedTemporaryFiler^rwritestrclosegenerate_malt_commandnamer@getcwdchdirrArBrZ_executejoinopenreaditerrremove) r` sentencesrTrj input_fileZ output_filelinecmdZ _current_pathretinfileZtree_strr9r9r:parse_tagged_sentssD    2 zMaltParser.parse_tagged_sentscs"fdd|D}j|||dS)an Use MaltParser to parse multiple sentences. Takes a list of sentences, where each sentence is a list of words. Each sentence will be automatically tagged with this MaltParser instance's tagger. :param sentences: Input sentences to parse :type sentence: list(list(str)) :return: iter(DependencyGraph) c3s|]}j|VqdS)N)r_)rCZsentence)r`r9r: sz)MaltParser.parse_sents..)rj)r)r`rzrTrjZtagged_sentencesr9)r`r: parse_sentss zMaltParser.parse_sentscCsdg}||j7}tjjdr dnd}|d|j|jg7}|dg7}tjj|j rl|dtjj |j d g7}n|d|j g7}|d |g7}|d kr|d |g7}|d |g7}|S)a This function generates the maltparser command use at the terminal. :param inputfilename: path to the input file :type inputfilename: str :param outputfilename: path to the output file :type outputfilename: str javawin;:z-cpzorg.maltparser.Maltz-cr?z-irhz-oz-m) rYsysplatformrFrurXr@rArJrZrB)r`Z inputfilenameZoutputfilenamerfr}Zclasspaths_separatorr9r9r:rps      z MaltParser.generate_malt_commandcCs&|rdntj}tj|||d}|jS)N)stdoutstderr) subprocessPIPEPopenwait)r}rToutputpr9r9r:rtszMaltParser._executec Csbtjd|jddd(}djdd|D}|jt|WdQRX|j|j|d tj |jdS) z Train MaltParser from a list of ``DependencyGraph`` objects :param depgraphs: list of ``DependencyGraph`` objects for training input data :type depgraphs: DependencyGraph zmalt_train.conll.rcF)rdrerfrg css|]}|jdVqdS) N)Zto_conll)rCZdgr9r9r:r sz#MaltParser.train..N)rT) r\rlr^rurmrntrain_from_filerqr@ry)r`Z depgraphsrTr{Z input_strr9r9r:trains zMaltParser.traincCst|tr`tjd|jddd<}|j}|j}|jt|WdQRX|j |j |dSQRX|j |dd}|j ||}|d krt d d j||fd |_dS) z Train MaltParser from a file :param conll_file: str for the filename of the training input data :type conll_file: str zmalt_train.conll.rcF)rdrerfrgN)rTZlearn)rfrz1MaltParser training (%s) failed with exit code %driT) isinstancerr\rlr^rvrwrmrnrrqrprtrkrur[)r`Z conll_filerTr{Zconll_input_fileZ conll_strr}r~r9r9r:r's   zMaltParser.train_from_file)r>NNN)Frb)Frb)NN)F)F)F) __name__ __module__ __qualname____doc__rarrrp staticmethodrtrrr9r9r9r:rW^s % B    rW__main__)inspectr@rrr\Z nltk.datarZnltk.internalsrrrZnltk.parse.apirZnltk.parse.dependencygraphrZnltk.parse.utilrr;rRrVrWrdoctesttestmodr9r9r9r: s"    & gC