• gensim的word2vec的简单使用


    from gensim.models import Word2Vec as wtv
    import jieba
    
    s1 = "刘新宇是一个自然语言处理算法工程师"
    s2 = "他从事算法方面的工作很多年"
    s3 = "刘新宇是一名非常资深的算法工程师"
    
    sentences = [jieba.lcut(s1),jieba.lcut(s2),jieba.lcut(s3)]
    print(sentences)
    
    model = wtv(sentences,min_count=1)
    
    print("this is the summary of the model:")
    print(model)
    
    words = list(model.wv.vocab)
    print('this is the vocabulary for our corpus:')
    print(words)
    
    print("the vector for the word singer:")
    print(model['资深'])
    
    w1 = ['资深']
    print(model.wv.most_similar(positive=w1,topn=5))
    

      结果:

    [['刘新宇', '是', '一个', '自然语言', '处理', '算法', '工程师'], ['他', '从事', '算法', '方面', '的', '工作', '很多年'], ['刘新宇', '是', '一名', '非常', '资深', '的', '算法', '工程师']]
    this is the summary of the model:
    Word2Vec(vocab=16, size=100, alpha=0.025)
    this is the vocabulary for our corpus:
    ['刘新宇', '是', '一个', '自然语言', '处理', '算法', '工程师', '他', '从事', '方面', '的', '工作', '很多年', '一名', '非常', '资深']
    the vector for the word singer:
    [-3.7734744e-03  2.6095781e-04 -3.1862229e-03  4.2619775e-03
     -4.5439765e-07 -1.8814839e-03 -1.9608627e-03 -4.3091276e-03
      2.0104370e-03  2.4313631e-03  3.6958823e-04 -4.6373094e-03
     -3.8697310e-03 -1.3030261e-03  3.4550454e-03  3.6497047e-04
     -3.0894275e-03  2.7997990e-04  2.4094102e-03  3.7222090e-03
     -3.3406885e-03  3.8989806e-03 -1.9044009e-03 -1.2546520e-03
     -2.7420574e-03  1.9589183e-03  4.6422374e-03  2.2412005e-03
     -4.5687910e-03 -3.0045302e-03 -7.5545040e-04 -1.9913551e-03
     -5.4244912e-04  3.4368648e-03  3.1319596e-03  3.5465839e-03
      4.0631965e-04 -8.0981071e-04 -4.5625023e-03  1.8198669e-03
     -2.3173515e-03  1.3505023e-03  4.2371401e-03  4.9272538e-03
     -2.1169472e-03  4.2408700e-03  3.7939013e-03 -1.9469961e-03
     -1.0268842e-03 -3.4248799e-03 -4.0382403e-03 -8.0300641e-04
     -4.3166843e-03  4.0071514e-03 -5.1711878e-04 -1.0944011e-03
     -9.2390249e-04 -2.0183630e-03 -1.1692114e-04  3.7988871e-03
      3.0223508e-03 -2.7847637e-03 -2.0709957e-03 -3.2283876e-03
     -7.3188142e-04  6.4730411e-06  2.4504904e-03 -1.8213416e-03
     -7.2910590e-04 -3.1336993e-03 -2.1612353e-03  3.4241637e-03
      9.4859622e-04 -1.1737887e-03 -4.3117562e-03 -2.7182067e-03
     -6.3206785e-04  3.5696046e-03 -2.9301932e-03  9.1675809e-04
      1.7115782e-03  1.6887123e-03  4.1562999e-03  3.1984923e-03
      2.4283223e-04  4.2053428e-03  2.4675422e-03  2.9653152e-03
      4.8725074e-03  3.6773803e-03  3.8778691e-03  7.9092768e-04
      3.6476396e-03 -1.1947335e-03 -2.0735445e-03  2.1955518e-03
     -1.3067436e-03 -2.5959394e-03  4.0679227e-04  7.9005008e-04]
    [('工作', 0.1197129413485527), ('非常', 0.08151963353157043), ('一个', 0.061945877969264984), ('的', 0.05767860636115074), ('他', 0.03252919763326645)]
    C:/Users/user/Desktop/pycharm_test/test_gensim.py:21: DeprecationWarning: Call to deprecated `__getitem__` (Method will be removed in 4.0.0, use self.wv.__getitem__() instead).
      print(model['资深'])
    

      Word2vec可调整的参数如下:

      def __init__(self, sentences=None, corpus_file=None, size=100, alpha=0.025, window=5, min_count=5,
                     max_vocab_size=None, sample=1e-3, seed=1, workers=3, min_alpha=0.0001,
                     sg=0, hs=0, negative=5, ns_exponent=0.75, cbow_mean=1, hashfxn=hash, iter=5, null_word=0,
                     trim_rule=None, sorted_vocab=1, batch_words=MAX_WORDS_IN_BATCH, compute_loss=False, callbacks=(),
                     max_final_vocab=None):
    

      

  • 相关阅读:
    RVM Ruby 版本管理器的删除 Gatling
    JWT 构建Rails API 授权登录
    Linux grep根据关键字匹配前后几行
    bootstrap-table 常用总结-树形结构
    linux 下jq的使用
    SHELL脚本获取域名对应的IP地址
    golang将切片或数组进行分组
    linux的统计实现
    Linux:“awk”命令的妙用
    rails 上传文件
  • 原文地址:https://www.cnblogs.com/LiuXinyu12378/p/13322139.html
Copyright © 2020-2023  润新知