Frequent term-based text clustering.
8497365
Frequent term based text clustering
Text clustering methods can be used to structure large sets of text or hypertext documents. The well known methods of text clustering, however, do not really address the special problems of text clustering: very high dimensionality of the data, very large size of the databases and understandability of the cluster description. In this paper, we introduce a novel approach which uses frequent item (term) sets for text clustering. Such frequent sets can be efficiently discovered using algorithms for association rule mining. To cluster based on frequent term sets, we measure the mutual overlap of frequent sets with respect to the sets of supporting documents. We present two algorithms for frequent term based text clustering, FTC which creates flat clusterings and HFTC for hierarchical clustering. An experimental evaluation on classical text documents as well as on web documents demonstrates that the proposed algorithms obtain clusterings of comparable quality significantly more efficiently than state of the art text clustering algorithms. Furthermore, our methods provide an understandable description of the discovered clusters by their frequent term sets.
KDD
Florian Beil Martin Ester Xiaowei Xu
18719641
A frequent term based text clustering approach using novel similarity measure
Text clustering is an unsupervised process forming its basis solely on finding the similarity relationship between documents with the output as a set of clusters [14] In this research, a commonality measure is defined to find commonality between two text files which is used as a similarity measure. The main idea is to apply any existing frequent item finding algorithm such as apriori or fp tree to the initial set of text files to reduce the dimension of the input text files. A document feature vector is formed for all the documents. Then a vector is formed for all the static text input files. The algorithm outputs a set of clusters from the initial input of text files considered.
2014 IEEE International Advance Computing Conference (IACC)
G Suresh Reddy T V Rajinikanth A Ananda Rao
67278900
Frequent Term Based Text Clustering Using Hidden Support
As the use of Internet increases in all directions, generation of digital documents grow vigorously. Handling of these digital documents challenges the technology to get appropriate response to any query. Many researchers took this challenge and depict their interest to mine these digitalized treasure of knowledge and find hidden information. High dimensionality of text document is always a big challenge for researchers. Handling of high dimensional text documents for classifying them into clusters with accuracy is another stone of challenge. In this paper, we proposed a method hidden term based document clustering (HTBDC) which utilized frequent itemset based mining method. Here in our approach, we try to trade off between high dimensionality with high accuracy of clustering and we got good results. We evaluate our method on the bases of F score on standard datasets, and the results show that our method performs comparatively better.
Harsha Patil Ramjeevan Singh Thakur
61324262
Improving Frequent Term Based Text Clustering with Word Belief Network
The algorithm of frequent term based text clustering (FTC) can be applied to news topic clustering system, in order to help users locate interested topics and articles quickly. But it is difficult to set support threshold for mining association rules. This paper tries to build a word belief network, which satisfies basic rules of small worlds. So we can improve FTC algorithm with characteristics of small worlds and implement text clustering quickly. The paper puts forward an idea that adopts inverted index into this algorithm, which simplifies programming and improves operation efficiency. The experimental results verified that the system could find current hot news topics efficiently and users could locate their interested document collection.
Yong Zhang Rui Fang Liu Ruiyang Luo
61495799
A Novel Similarity Measure for frequent Term Based Text Clustering on high dimensional data
Text clustering is one of the main themes in text mining .It refers to the process of grouping document with similar contents or topics into clusters to improve both availability reliability of the mining, In this research, a frequent item set is a set of words which occur together frequently and are good canDi Dates for clusters. By considering only the items which occur frequently in the Data, we can also address problems like outlier removal, Dimensionality reduction, etc. The main idea is to apply any existing frequent item finding algorithm such as a priori or Dp tree to the initial set of text files to reduce the Dimension of the input text files. A Document Feature vector is formed for all the Documents. Then a vector is formed for all the static text input files. The algorithm outputs a set of clusters from the initial input of text files considered.
B Prajna
7687737
Frequent term based text document clustering: A new approach
Document clustering is used to organize the documents into groups. VSM (Vector Space Model) is a technique used to represent the document as a vector. Working with VSM to cluster the documents is easier. The main problem with text documents clustering is very high dimensionality of data. A term in the document represents a dimension. To reduce the dimensions of the document vector space, it is preprocessed. The main techniques involved are stemming and term filtering for dimensions reduction of document vectors. After dimensions reduction, term frequency vectors corresponding to each document are generated, where each cell in the term frequency vector represents frequencies of a term. Using proposed method in the paper, each pair of term frequency vectors are compared to find out the similarity value between every two corresponding documents. In this way, three similarity matrices minimum_match, maximum_match and average_match are generated which are further used in various clustering techniques to produce clusters. Clusters produced using proposed approach are compared with that of clusters produced based on cosine similarity in terms of F measure. Higher values of F measure for clusters produced using proposed method shows that proposed algorithm is better.
2015 International Conference on Soft Computing Techniques and Implementations (ICSCTI)
M Kumar Dhruv Kumar Yadav Vijay Kumar Gupta
3878769
Frequent term based text document clustering using similarity measures: A novel approach
Clustering is one of the epic and traditional ways to make sure that the documents are retrieved at the right pace and according to the requirement. Clustering leads to keeping the similar kind of documents all together and so that they can be retrieved easily. The measure through which the relation between two documents is measured is called similarity index. There are several kind of similarity index already in the process. The proposed algorithm uses two kind of similarity index and combines them to produce a new similarity index. Similarity index plays a vital role in the clustering and classification procedure. The proposed algorithm also uses Fuzzy logic for the clustering rules and furthermore it is classified by the Support Vector Machine to justify the accuracy of the proposed solution.
2017 Fourth International Conference on Image Information Processing (ICIIP)
Vijay Kumar Gupta Maitreyee Dutta M Kumar
14958814
Performance Improvement for Frequent Term based Text Clustering Algorithm
Frequent term based text clustering [2] is a recently introduced text clustering technique, which uses frequent term sets and dramatically decreases the dimensionality of the document vector space, thus especially addressing itself to the problems of text clustering: very high dimensionality of the date and very large size of the databases [2] Moreover, frequent term sets provide understandable meanings for clusters. Frequent term based text clustering algorithm (FTC) has shown significant efficiency comparing to some well known text clustering methods [2] but the quality of clustering still needs further enhancement. This report points out the problems of the overlap calculations in FTC and introduces polished algorithms that aim at improvements on both the running time and the cluster quality. The performances of FTC before and after improvements are compared on the basis of the experiments on classical text documents as well as on web documents. At last, an evaluation on the clustering procedure may provide a clue for further work. General Terms Algorithms, Performance, Experimentation.
Zhongmin Shi zshi
133671047
Klasterisasi pada Dokumen Berita Berbahasa Indonesia Berdasarkan Frequent Term Based Text Clustering (HFTC dan FTC)
Mega Rulliana
63227357
[ACM Press the eighth ACM SIGKDD international conference Edmonton, Alberta, Canada (2002.07.23 2002.07.26) Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining KDD '02 Frequent term based text clustering