3 d7' @sddlZddlmZddlmZddlmZy ddlZWnek rLYnXddl m Z Gddde Z dd Z d d Z Gd d d ZGdddZdS)N)abstractmethod)sqrt)stdout)ClusterIc@sdeZdZdZdddZdddZedd Zd d Zed d Z ddZ ddZ ddZ ddZ dS)VectorSpaceClustererz Abstract clusterer which takes tokens and maps them into a vector space. Optionally performs singular value decomposition to reduce the dimensionality. FNcCsd|_||_||_dS)a) :param normalise: should vectors be normalised to length 1 :type normalise: boolean :param svd_dimensions: number of dimensions to use in reducing vector dimensionsionality with SVD :type svd_dimensions: int N)_Tt_should_normalise_svd_dimensions)selfZ normaliseZsvd_dimensionsr 1/tmp/pip-build-v9q4h5k9/nltk/nltk/cluster/util.py__init__szVectorSpaceClusterer.__init__c st|dkstjr&ttj|}jrĈjt|dkrtjj tj tj |\}}}|djtj jtj }|dddjf}|djddf} tj tj|| }tj |_j|||rfdd|DSdS)Nrcsg|]}j|qSr )classify).0vector)r r r ?sz0VectorSpaceClusterer.cluster..)lenAssertionErrorrlistmap _normaliser numpyZlinalgZsvdZ transposearrayidentityZfloat64dotrcluster_vectorspace) r vectorsZassign_clusterstraceudZvtSTZDtr )r r cluster(s  zVectorSpaceClusterer.clustercCsdS)zD Finds the clusters using the given set of vectors. Nr )r rrr r r rAsz(VectorSpaceClusterer.cluster_vectorspacecCs<|jr|j|}|jdk r(tj|j|}|j|}|j|S)N)rrrrrclassify_vectorspaceZ cluster_name)r rr"r r r rGs    zVectorSpaceClusterer.classifycCsdS)zN Returns the index of the appropriate cluster for the vector. Nr )r rr r r r#Osz)VectorSpaceClusterer.classify_vectorspacecCs4|jr|j|}|jdk r(tj|j|}|j||S)N)rrrrrlikelihood_vectorspace)r rlabelr r r likelihoodUs   zVectorSpaceClusterer.likelihoodcCs|j|}||krdSdS)zP Returns the likelihood of the vector belonging to the cluster. g?g)r#)r rr"Z predictedr r r r$\s z+VectorSpaceClusterer.likelihood_vectorspacecCs,|jr|j|}|jdk r(tj|j|}|S)zU Returns the vector after normalisation and dimensionality reduction N)rrrrr)r rr r r rcs   zVectorSpaceClusterer.vectorcCs|ttj||S)z7 Normalises the vector to unit length. )rrr)r rr r r rmszVectorSpaceClusterer._normalise)FN)FF)__name__ __module__ __qualname____doc__r r"rrrr#r&r$rrr r r r rs    rcCs||}ttj||S)z} Returns the euclidean distance between vectors u and v. This is equivalent to the length of the vector (u - v). )rrr)rvZdiffr r r euclidean_distancetsr,cCs0dtj||ttj||ttj||S)zw Returns 1 minus the cosine of the angle between vectors v and u. This is equal to ``1 - (u.v / |u||v|)``. )rrr)rr+r r r cosine_distance}sr.c@s2eZdZdZddZd ddZddZd d Zd S) _DendrogramNodezTree node of a dendrogram.cGs||_||_dS)N)_value _children)r valuechildrenr r r r sz_DendrogramNode.__init__TcCsD|jr.g}x|jD]}|j|j|qW|S|r:|jgS|gSdS)N)r1extendleavesr0)r valuesr5childr r r r5s z_DendrogramNode.leavescCs|j|fg}xpt||kr||j\}}|js<|j||fPx4|jD]*}|jr`|j|j|fqD|jd|fqDW|jqWg}x|D]\}}|j|jqW|S)Nr)r0rpopr1pushappendsortr5)r nqueueprioritynoder7groupsr r r r@s    z_DendrogramNode.groupscCst|j|jdkS)Nr)r.r0)r Z comparatorr r r __lt__sz_DendrogramNode.__lt__N)T)r'r(r)r*r r5r@rAr r r r r/s  r/c@s@eZdZdZgfddZddZddZgfdd Zd d Zd S) Dendrograma Represents a dendrogram, a tree with a specified branching order. This must be initialised with the leaf items, then iteratively call merge for each branch. This class constructs a tree representing the order of calls to the merge function. cCs(dd|D|_tj|j|_d|_dS)zs :param items: the items at the leaves of the dendrogram :type items: sequence of (any) cSsg|] }t|qSr )r/)ritemr r r rsz'Dendrogram.__init__..r-N)_itemscopy_original_items_merge)r itemsr r r r szDendrogram.__init__cslt|dksttjffdd|D}jd7_|j|d<x|ddD] }j|=qXWdS)a= Merges nodes at given indices in the dendrogram. The nodes will be combined which then replaces the first node specified. All other nodes involved in the merge will be removed. :param indices: indices of the items to merge (at least two) :type indices: seq of int c3s|]}j|VqdS)N)rD)ri)r r r sz#Dendrogram.merge..r-rN)rrr/rGrD)r indicesr?rJr )r r merges zDendrogram.mergecCs6t|jdkr"t|jf|j}n |jd}|j|S)z Finds the n-groups of items (leaves) reachable from a cut at depth n. :param n: number of groups :type n: int r-r)rrDr/rGr@)r r<rootr r r r@s zDendrogram.groupscs@d\}}}t|jdkr,t|jf|j}n |jd}|j}|rF|}ndd|D}ttt|ddtddfd d d d }|j|fg} fdd|D} x`| r| j \} } t tdd| j } t t|j | }| rt |}t|}xtt|D]}||| krz||kr@||d |n,||kr\|||d n|||||| |<n:||ko|knr||||n || |qW|dx(| j D]}|j r| j|j|fqW| jx| D]}||qW|dqW|djfdd|D|ddS)z Print the dendrogram in ASCII art to standard out. :param leaf_labels: an optional list of strings to use for labeling the leaves :type leaf_labels: list +-|r-rcSsg|]}d|jqS)z%s)r0)rleafr r r rsz#Dendrogram.show..rI cs|||S)Nr )Zcentreleftright)lhalfrhalfr r formatszDendrogram.show..formatcSstj|dS)N)rwrite)strr r r displaysz Dendrogram.show..displaycsg|] }dqS)rSr )rrR)rXr r rscSs|jddS)NFr)r5)cr r r sz!Dendrogram.show.. c3s|]}|jVqdS)N)center)rrC)widthr r rK#sz"Dendrogram.show..N)rOrPrQ)rSrS)rrDr/rGrFmaxrintr0r8rr1indexminranger:r;join)r Z leaf_labelsZJOINZHLINKZVLINKrNr5Zlast_rowr[r=Z verticalsr>r?Zchild_left_leafrLmin_idxZmax_idxrJr7Zverticalr )rXrVrWrar showsV          zDendrogram.showcCsBt|jdkr"t|jf|j}n |jd}|jd}dt|S)Nr-rFz)rrDr/rGr5)r rNr5r r r __repr__&s   zDendrogram.__repr__N) r'r(r)r*r rMr@rirjr r r r rBs   IrB)rEabcrmathrsysrr ImportErrorZnltk.cluster.apirrr,r.r/rBr r r r s     _ +