3 dS@sjdZddlZddlZddlZddlmZddlZddlm Z GdddZ GdddZ Gd d d Z dS) a If you use the VADER sentiment analysis tools, please cite: Hutto, C.J. & Gilbert, E.E. (2014). VADER: A Parsimonious Rule-based Model for Sentiment Analysis of Social Media Text. Eighth International Conference on Weblogs and Social Media (ICWSM-14). Ann Arbor, MI, June 2014. N)product)pairwisecC@seZdZdZdZdcZdZddZddddd d d d d ddddddddddddddddddd d!d"d#d$d%d&d'd(d)d*d+d,d-d.d/d0d1d2d3d4d5d6d7d8d9d:d;dd?h;Zeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeed@BZ dAdAdBdedCdfdgdDZ e j dEe j ejdFZdGdHdIdJdKdLdMdNdOdPdQdRdSdTdUdVdWgZdXdYZdhd[d\Zdid^d_Zd`daZdbS)jVaderConstantsz8 A class to keep the Vader lists and constants. gn?g~jt?gGz?ZaintZarentZcannotZcantZcouldntZdarentZdidntZdoesntzain'tzaren'tzcan'tzcouldn'tzdaren'tzdidn'tzdoesn'tZdontZhadntZhasntZhaventZisntZmightntZmustntZneitherzdon'tzhadn'tzhasn'tzhaven'tzisn'tzmightn'tzmustn'tZneedntzneedn'tnevernoneZnopeZnornotZnothingZnowhereZoughtntZshantZshouldntZuhuhZwasntZwerentzoughtn'tzshan'tz shouldn'tzuh-uhzwasn'tzweren'tZwithoutZwontZwouldntzwon'tzwouldn'tZrarelyZseldomZdespite)BZ absolutelyZ amazinglyZawfullyZ completelyZ considerablyZ decidedlyZdeeplyZeffingZ enormouslyZentirelyZ especiallyZ exceptionallyZ extremelyZ fabulouslyZflippingZflippinZfrickingZfrickinZfriggingZfrigginZfullyZfuckingZgreatlyZhellaZhighlyZhugelyZ incrediblyZ intenselyZmajorlyZmoreZmostZ particularlyZpurelyZquiteZreallyZ remarkablysoZ substantiallyZ thoroughlyZtotallyZ tremendouslyZuberZ unbelievablyZ unusuallyZutterlyveryZalmostZbarelyZhardlyz just enoughzkind ofZkindaZkindofzkind-ofZlesslittleZ marginallyZ occasionallyZpartlyZscarcelyZslightlyZsomewhatzsort ofZsortaZsortofzsort-ofg?)zthe shitzthe bombzbad assz yeah rightzcut the mustardz kiss of deathz hand to mouth[].!?,;:-'"z!!z!!!z??z???z?!?z!?!z?!?!z!?!?cCsdS)N)selfrr4/tmp/pip-build-v9q4h5k9/nltk/nltk/sentiment/vader.py__init__szVaderConstants.__init__Tcsp|jtfdd|Dr dS|r:tdd|Dr:dSx0t|D]$\}}|jdkrD|jdkrDdSqDWdS)z< Determine if input contains negation words c3s|]}|jkVqdS)N)lower).0word) neg_wordsrr sz)VaderConstants.negated..Tcss|]}d|jkVqdS)zn'tN)r)rrrrrr sleastatF)NEGATEanyrr)rZ input_wordsZ include_ntfirstsecondr)rrnegatedszVaderConstants.negatedcCs|tj|||}|S)z| Normalize the score to be between -1 and 1 using an alpha that approximates the max expected value )mathsqrt)rZscorealphaZ norm_scorerrr normalizeszVaderConstants.normalizecCs^d}|j}||jkrZ|j|}|dkr0|d9}|jrZ|rZ|dkrP||j7}n ||j8}|S)zh Check if the preceding words increase, decrease, or negate/nullify the valence gr)r BOOSTER_DICTisupperC_INCR)rrvalence is_cap_diffZscalarZ word_lowerrrrscalar_inc_decs     zVaderConstants.scalar_inc_decNgnҿgGzgr5)T)r()__name__ __module__ __qualname____doc__ZB_INCRB_DECRr1N_SCALARr#r/SPECIAL_CASE_IDIOMSrecompileescapestring punctuationREGEX_REMOVE_PUNCTUATION PUNC_LISTrr'r,r4rrrrr!s>  rc@s0eZdZdZddZddZddZdd Zd S) SentiTextzL Identify sentiment-relevant string-level properties of input text. cCsFt|tst|jd}||_||_||_|j|_|j|j|_ dS)Nzutf-8) isinstancestrencodetextrCrB_words_and_emoticonswords_and_emoticonsallcap_differentialr3)rrHZ punc_listZregex_remove_punctuationrrrr s  zSentiText.__init__cCsd|jjd|j}|j}dd|D}ddt|j|D}ddt||jD}|}|j||S)zt Returns mapping of form: { 'cat,': 'cat', ',cat': 'cat', } cSsh|]}t|dkr|qS)r-)len)rwrrr #sz-SentiText._words_plus_punc..cSsi|]}|ddj|qS)r-rL)join)rprrr %sz.SentiText._words_plus_punc..cSsi|]}|ddj|qS)rrL)rP)rrQrrrrR&s)rBsubrHsplitrrCupdate)rZ no_punc_textZ words_onlyZ punc_beforeZ punc_afterwords_punc_dictrrr_words_plus_puncs zSentiText._words_plus_punccCsN|jj}|j}dd|D}x(t|D]\}}||kr*||||<q*W|S)z Removes leading and trailing puncutation Leaves contractions and most emoticons Does not preserve punc-plus-letter emoticons (e.g. :D) cSsg|]}t|dkr|qS)r-)rM)rwerrr 3sz2SentiText._words_and_emoticons..)rHrTrW enumerate)rZwesrVirXrrrrI+s zSentiText._words_and_emoticonscCsVd}d}x|D]}|jr|d7}qWt||}d|koHt|knrRd}|S)z Check whether just some words in the input are ALL CAPS :param list words: The words to inspect :returns: `True` if some but not all items in `words` are ALL CAPS Frr-T)r0rM)rwordsZ is_differentZ allcap_wordsrZcap_differentialrrrrK9s   zSentiText.allcap_differentialN)r6r7r8r9rrWrIrKrrrrrDs  rDc@szeZdZdZdddZddZddZd d Zd d Zd dZ ddZ ddZ ddZ ddZ ddZddZddZdS)SentimentIntensityAnalyzerz8 Give a sentiment intensity score to sentences. ;sentiment/vader_lexicon.zip/vader_lexicon/vader_lexicon.txtcCs$tjj||_|j|_t|_dS)N)nltkdataload lexicon_file make_lex_dictlexiconr constants)rrbrrrrPs z#SentimentIntensityAnalyzer.__init__cCsDi}x:|jjdD]*}|jjddd\}}t|||<qW|S)z6 Convert lexicon file to a dictionary   rr )rbrTstripfloat)rZlex_dictlinerZmeasurerrrrcXs z(SentimentIntensityAnalyzer.make_lex_dictcCst||jj|jj}g}|j}xx|D]p}d}|j|}|t|dkrf|jdkrf||djdksv|j|jjkr|j |q$|j |||||}q$W|j ||}|j ||S)z Return a float for sentiment strength based on the input text. Positive values are positive valence, negative value are negative valence. rr-kindZof) rDrerCrBrJindexrMrr/appendsentiment_valence _but_check score_valence)rrH sentitext sentimentsrJitemr2r[rrrpolarity_scoresbs      z*SentimentIntensityAnalyzer.polarity_scoresc Cs$|j}|j}|j}||jkr|j|}|jrX|rX|dkrL||jj7}n ||jj8}xtddD]} || ko||| dj|jkrd|jj||| d||} | dkr| dkr| d} | dkr| dkr| d} || }|j ||| |}| dkrd|j |||}qdW|j |||}|j ||S)Nrr r-gffffff?r g?) r3rJrrdr0rer1ranger4 _never_check _idioms_check _least_checkrm) rr2rqrsr[rrr3rJZitem_lowercasestart_isrrrrns6       z,SentimentIntensityAnalyzer.sentiment_valencecCs|dkrh||dj|jkrh||djdkrh||djdkr||djdkr||jj}n>|dkr||dj|jkr||djdkr||jj}|S)Nr-r!r r"r r)rrdrer;)rr2rJr[rrrrxs z'SentimentIntensityAnalyzer._least_checkcCsvdd|D}dht|@}|rr|jtt|}x>t|D]2\}}||krZ|d||<q<||kr<|d||<q.butg?g?)setrlnextiterrZ)rrJrrr{ZbiZsidxZ sentimentrrrrosz%SentimentIntensityAnalyzer._but_checkc Cs||dd||}dj||d||d||}||dd||d}dj||d||d||d}dj||d||d}|||||g} x&| D]} | |jjkr|jj| }PqWt|d|kr||d||d} | |jjkr|jj| }t|d|dkrjdj||||d||d} | |jjkrj|jj| }||jjks||jjkr||jj}|S)Nr- z{} {} {}r r z{} {})formatrer<rMr/r:) rr2rJr[ZonezeroZ twoonezeroZtwooneZ threetwooneZthreetwo sequencesseqZzerooneZ zeroonetworrrrws@            z(SentimentIntensityAnalyzer._idioms_checkcCs"|dkr*|jj||dgr*||jj}|dkr||ddkrl||ddksb||ddkrl|d}n&|jj|||dgr||jj}|dkr||ddkr||ddks||ddks||ddks||ddkr|d }n(|jj|||dgr||jj}|S) Nrr-r rrthisg?r g?)rer'r;)rr2rJryr[rrrrvs(      z'SentimentIntensityAnalyzer._never_checkcCs |j|}|j|}||}|S)N) _amplify_ep _amplify_qm)rsum_srH ep_amplifier qm_amplifierpunct_emph_amplifierrrr_punctuation_emphasiss  z0SentimentIntensityAnalyzer._punctuation_emphasiscCs"|jd}|dkrd}|d}|S)Nrg㥛 ?)count)rrHZep_countrrrrr&s  z&SentimentIntensityAnalyzer._amplify_epcCs0|jd}d}|dkr,|dkr(|d}nd}|S)Nrrr-r g ףp= ?gQ?)r)rrHZqm_countrrrrr0s  z&SentimentIntensityAnalyzer._amplify_qmcCsdd}d}d}xL|D]D}|dkr.|t|d7}|dkrF|t|d7}|dkr|d7}qW|||fS)Ngrr-)ri)rrrpos_sumneg_sum neu_countZsentiment_scorerrr_sift_sentiment_scores=s  z1SentimentIntensityAnalyzer._sift_sentiment_scorescCs|rtt|}|j||}|dkr.||7}n|dkr>||8}|jj|}|j|\}}}|tj|krr||7}n|tj|kr||8}|tj||} tj|| } tj|| } tj|| } nd}d} d} d} t| dt| dt| dt|dd} | S)Nrgr r)negneuposcompound) risumrrer,rr)fabsround)rrrrHrrrrrrtotalrrrZsentiment_dictrrrrpOs4     z(SentimentIntensityAnalyzer.score_valenceN)r^)r6r7r8r9rrcrtrnrxrorwrvrrrrrprrrrr]Ks  2 1  r]) r9r)r=r@ itertoolsrZ nltk.datar_Z nltk.utilrrrDr]rrrrs  gD