CUI Donghu,ZHAO Yahui,CUI Rongyi*.Research on text classification of K-nearest neighbor algorithm based on relative entropy[J].Journal of Yanbian University,2021,47(02):175-179.
基于相对熵的KNN文本分类方法的研究
- Title:
- Research on text classification of K-nearest neighbor algorithm based on relative entropy
- 文章编号:
- 1004-4353(2021)02-0175-05
- 分类号:
- TP391.1
- 文献标志码:
- A
- 摘要:
- 摘要:为提高处理文本相似度的效果,提出了一种基于相对熵度量文本差异的KNN算法.该算法首先对文本进行预处理(分字与删去停用字)和构建特征字字典; 然后计算训练集中所有文本特征字的概率,并组成训练集(特征字概率矩阵); 最后计算预测文本的特征字概率向量,并通过计算和统计K个预测文本与训练集文本间相对熵最小的文本类别个数后将数目最多的类别作为测试样本的类别.实验结果表明,该算法的分类效果不仅显著优于传统KNN、SVM、Decision Tree、朴素Bayes算法的分类效果,且在小样本数据情况下
- Abstract:
- To improve the effectiveness of processing text similarity, a KNN algorithm based on the relative entropy measure of text feature differences was proposed in this paper. Firstly, the algorithm preprocessed the text, including character separation and deletion of stop characters, and constructed a feature character dictionary. Then the probabilities of all the text feature characters in the training set were calculated, and the training set(probability matrix of feature character)was formed. Finally, we calculated the probability vector of feature characters of the predicted text, and counted the number of text categories with the lowest relative entropy between the K predicted texts and the training set, and used the category with the highest number as the category of the test sample. The experimental results show that the classification effect of this algorithm is not only significantly better than that of the traditional KNN, SVM, Decision Tree, and Naive Bayes, but also significantly better than that of RNN algorithm in the case of small sample data.
参考文献/References:
[1] 刘娇,崔荣一,赵亚慧,等.跨语言文献相似度的分析方法[J].延边大学学报(自然科学版),2016,42(2):151-155.
[2] 张雷,崔荣一.基于编辑距离的词序敏感相似度度量方法[J].延边大学学报(自然科学版),2020,46(2):140-144.
[3] 邵珊珊.基于KNN的分类方法及其应用研究[D].秦皇岛:燕山大学,2019.
[4] 张冲.基于Attention -Based LSTM模型的文本分类技术的研究[D].南京:南京大学,2016.
[5] 王亚林,陈忍忍.不同机器学习算法在分类问题中的应用比较[J].黑龙江科学,2021,12(4):16-18.
[6] ABDULATEEF S, KHAN N A. Machine learning based sentiment text classification for evaluating treatment quality of discharge summary[J]. Information(Switzerland), 2020,11(5):17.
[7] KIBANOV M, BECKER M, MUELLER J, et al. Adaptive kNN using expected accuracy for classification of Geo - spatial data[J]. ACM Press, 2018,18(33):857-865.
[8] HUANG X, XIONG L, LIU Y, et al. An improved KNN short text classification algorithm based on category features[J]. Computer Engineering and Science, 2018,40(3):148-154.
[9] NOUSHAHRH G, AHMADI S. Multitask learning for text classification with deep neural networks[C]//International Conference on Innovative Techniques and Applications of Artificial Intelligence. Springer -Verlag: Springer International Publishing, 2016:119-133.
[10] 万家山,吴云志.基于深度学习的文本分类方法研究综述[J].天津理工大学学报,2021,37(2):41-47.
[11] 马燕.基于相对熵的作品作者判定方法[J].文教资料,2014(31):131-133.
[12] HUIBIN L, WEI C,AGUS S. Relative entropy based method for probabilistic sensitivity analysis in engineering design[J]. Journal of Mechanical Design, 2006,128(2):326-336.
[13] 胡春月.基于KNN算法的佚名诗词作者概率研究[J].技术与市场,2020,27(11):69-70.
[14] 丁义,杨建.欧式距离与标准化欧式距离在k近邻算法中的比较[J].软件,2020,41(10):135-136.
备注/Memo
收稿日期: 2021-02-17 *通信作者: 崔荣一(1962—),男,博士,教授,研究方向为模式识别、智能计算.
基金项目: 国家语委科研项目(YB135-76); 延边大学外国语言文学一流学科建设项目(18YLPY13)