python 自然语言处理（五）____WordNet

WordNet是面向语义的英语词典，与传统辞典类似，但结构更丰富。nltk中包括英语WordNet，共有155287个单词和117659个同义词。

1.寻找同义词

这里以motorcar为例，寻找它的同义词集。

>>> from nltk.corpus import wordnet as wn
>>> wn.synsets('motorcar')                                //找到同义词集
[Synset('car.n.01')]
>>> wn.synset('car.n.01').lemma_names
<bound method Synset.lemma_names of Synset('car.n.01')>
>>> wn.synset('car.n.01').lemma_names()                   //访问同义词集
['car', 'auto', 'automobile', 'machine', 'motorcar']
>>>

>>> wn.synset('car.n.01').definition()              //获取该词在该词集的定义
'a motor vehicle with four wheels; usually propelled by an internal combustion engine'
>>> wn.synset('car.n.01').examples()            //获取该词在该词集下的例句
['he needs a car to get to work']
>>> wn.synset('car.n.01').lemmas()
[Lemma('car.n.01.car'), Lemma('car.n.01.auto'), Lemma('car.n.01.automobile'), Lemma('car.n.01.machine'), Lemma('car.n.01.motorcar')]
>>> wn.lemma('car.n.01.automobile')
Lemma('car.n.01.automobile')
>>> wn.lemma('car.n.01.automobile').synset()
Synset('car.n.01')
>>> wn.lemma('car.n.01.automobile').name()
'automobile'
>>> wn.synsets('car')
[Synset('car.n.01'), Synset('car.n.02'), Synset('car.n.03'), Synset('car.n.04'), Synset('cable_car.n.01')]
>>> for synset in wn.synsets('car'):
...     print (synset.lemma_names())
...
['car', 'auto', 'automobile', 'machine', 'motorcar']
['car', 'railcar', 'railway_car', 'railroad_car']
['car', 'gondola']
['car', 'elevator_car']
['cable_car', 'car']
>>> wn.lemmas('car')                          //访问所有包含词car的词条
[Lemma('car.n.01.car'), Lemma('car.n.02.car'), Lemma('car.n.03.car'), Lemma('car.n.04.car'), Lemma('cable_car.n.01.car')]
>>>

2.WordNet的层次结构

WordNet的同义词集相当于抽象的概念，它们并不总是有对应的英语词汇。这些概念在层次结构中相互联系在一起。

如上图，是WordNet概念的层次片段。每个节点对应一个同义词集，边表示上位词/下位词关系，即上级概念与从属概念的关系。

>>> motorcar=wn.synset('car.n.01')
>>> types_of_motorcar=motorcar.hyponyms()
>>> types_of_motorcar[26]
Synset('stanley_steamer.n.01')
>>> sorted(
... [lemma.name()
... for synset in types_of_motorcar
... for lemma in synset.lemmas()])
['Model_T', 'S.U.V.', 'SUV', 'Stanley_Steamer', 'ambulance', 'beach_waggon', 'beach_wagon', 'bus', 'cab', 'compact', 'compact_car', 'convert
ible', 'coupe', 'cruiser', 'electric', 'electric_automobile', 'electric_car', 'estate_car', 'gas_guzzler', 'hack', 'hardtop', 'hatchback', '
heap', 'horseless_carriage', 'hot-rod', 'hot_rod', 'jalopy', 'jeep', 'landrover', 'limo', 'limousine', 'loaner', 'minicar', 'minivan', 'pace
_car', 'patrol_car', 'phaeton', 'police_car', 'police_cruiser', 'prowl_car', 'race_car', 'racer', 'racing_car', 'roadster', 'runabout', 'sal
oon', 'secondhand_car', 'sedan', 'sport_car', 'sport_utility', 'sport_utility_vehicle', 'sports_car', 'squad_car', 'station_waggon', 'statio
n_wagon', 'stock_car', 'subcompact', 'subcompact_car', 'taxi', 'taxicab', 'tourer', 'touring_car', 'two-seater', 'used-car', 'waggon', 'wago
n']
>>> motorcar.hypernyms()
[Synset('motor_vehicle.n.01')]
>>> paths=motorcar.hypernym_paths()
>>> len(paths)
2
>>> [synset.name for synset in paths[0]]
[<bound method Synset.name of Synset('entity.n.01')>, <bound method Synset.name of Synset('physical_entity.n.01')>, <bound method Synset.nam
e of Synset('object.n.01')>, <bound method Synset.name of Synset('whole.n.02')>, <bound method Synset.name of Synset('artifact.n.01')>, <bou
nd method Synset.name of Synset('instrumentality.n.03')>, <bound method Synset.name of Synset('container.n.01')>, <bound method Synset.name
of Synset('wheeled_vehicle.n.01')>, <bound method Synset.name of Synset('self-propelled_vehicle.n.01')>, <bound method Synset.name of Synset
('motor_vehicle.n.01')>, <bound method Synset.name of Synset('car.n.01')>]
>>> [synset.name() for synset in paths[0]]
['entity.n.01', 'physical_entity.n.01', 'object.n.01', 'whole.n.02', 'artifact.n.01', 'instrumentality.n.03', 'container.n.01', 'wheeled_veh
icle.n.01', 'self-propelled_vehicle.n.01', 'motor_vehicle.n.01', 'car.n.01']
>>> [synset.name() for synset in paths[1]]
['entity.n.01', 'physical_entity.n.01', 'object.n.01', 'whole.n.02', 'artifact.n.01', 'instrumentality.n.03', 'conveyance.n.03', 'vehicle.n.
01', 'wheeled_vehicle.n.01', 'self-propelled_vehicle.n.01', 'motor_vehicle.n.01', 'car.n.01']
>>> motorcar.root_hypernyms()
[Synset('entity.n.01')]
>>>

3.更多的词汇关系

上位词和下位词被称为词汇关系，因为它们是同义集之间的关系。这两者的关系为上下定位“is-a”层次。WordNet网络另一个重要的定位方式是从条目到它们的部件（部分）或到包含它们的东西（整体）。

1）部分-整体关系

>>> wn.synset('tree.n.01').part_meronyms()
[Synset('burl.n.02'), Synset('crown.n.07'), Synset('limb.n.02'), Synset('stump.n.01'), Synset('trunk.n.01')]
>>> wn.synset('tree.n.01').substance_meronyms()
[Synset('heartwood.n.01'), Synset('sapwood.n.01')]
>>> wn.synset('tree.n.01').member_holonyms()
[Synset('forest.n.01')]
>>> for synset in wn.synsets('mint', wn.NOUN):
...     print("%s : %s" % (synset.name(), synset.definition())
...
...
... )
...
batch.n.02 : (often followed by `of') a large number or amount or extent
mint.n.02 : any north temperate plant of the genus Mentha with aromatic leaves and small mauve flowers
mint.n.03 : any member of the mint family of plants
mint.n.04 : the leaves of a mint plant used fresh or candied
mint.n.05 : a candy that is flavored with a mint oil
mint.n.06 : a plant where money is coined by authority of the government
>>> wn.synset('mint.n.04').part_holonyms()
[Synset('mint.n.02')]
>>> wn.synset('mint.n.04').substance_holonyms()
[Synset('mint.n.05')]

2）蕴涵关系

>>> wn.synset('walk.v.01').entailments()
[Synset('step.v.01')]
>>> wn.synset('eat.v.01').entailments()
[Synset('chew.v.01'), Synset('swallow.v.01')]
>>> wn.synset('tease.v.03').entailments()
[Synset('arouse.v.07'), Synset('disappoint.v.01')]

3）反义词

>>> wn.lemma('supply.n.02.supply').antonyms()
[Lemma('demand.n.02.demand')]
>>> wn.lemma('rush.v.01.rush').antonyms()
[Lemma('linger.v.04.linger')]
>>> wn.lemma('horizontal.a.01.horizontal').antonyms()
[Lemma('inclined.a.02.inclined'), Lemma('vertical.a.01.vertical')]
>>> wn.lemma('staccato.r.01.staccato').antonyms()
[Lemma('legato.r.01.legato')]
>>>

4. 语义相似度

同义词集是由复杂的词汇关系网络所连接起来的。给定一个同义词集，可以遍历WordNet网络来查找相关含义的同义词集。每个同义词集都有一个或多个上位词路径连接到一个根上位词。连接到同一个根的两个同义词集可能有一些共同的上位词。如果两个同义词集共用一个特定的上位词——在上位词层次结构中处于较底层——它们一定有密切的联系。