2017-05-23 21:16:31 +00:00
|
|
|
|
//- 💫 DOCS > USAGE > SPACY 101 > POS TAGGING AND DEPENDENCY PARSING
|
|
|
|
|
|
|
|
|
|
|
|
p
|
2017-10-03 12:26:20 +00:00
|
|
|
|
| After tokenization, spaCy can #[strong parse] and #[strong tag] a
|
2017-05-23 21:16:31 +00:00
|
|
|
|
| given #[code Doc]. This is where the statistical model comes in, which
|
|
|
|
|
|
| enables spaCy to #[strong make a prediction] of which tag or label most
|
|
|
|
|
|
| likely applies in this context. A model consists of binary data and is
|
|
|
|
|
|
| produced by showing a system enough examples for it to make predictions
|
|
|
|
|
|
| that generalise across the language – for example, a word following "the"
|
|
|
|
|
|
| in English is most likely a noun.
|
|
|
|
|
|
|
|
|
|
|
|
p
|
|
|
|
|
|
| Linguistic annotations are available as
|
|
|
|
|
|
| #[+api("token#attributes") #[code Token] attributes]. Like many NLP
|
2017-05-28 17:25:34 +00:00
|
|
|
|
| libraries, spaCy #[strong encodes all strings to hash values] to reduce
|
2017-05-23 21:16:31 +00:00
|
|
|
|
| memory usage and improve efficiency. So to get the readable string
|
|
|
|
|
|
| representation of an attribute, we need to add an underscore #[code _]
|
|
|
|
|
|
| to its name:
|
|
|
|
|
|
|
|
|
|
|
|
+code.
|
|
|
|
|
|
doc = nlp(u'Apple is looking at buying U.K. startup for $1 billion')
|
|
|
|
|
|
|
|
|
|
|
|
for token in doc:
|
|
|
|
|
|
print(token.text, token.lemma_, token.pos_, token.tag_, token.dep_,
|
|
|
|
|
|
token.shape_, token.is_alpha, token.is_stop)
|
|
|
|
|
|
|
|
|
|
|
|
+aside
|
|
|
|
|
|
| #[strong Text:] The original word text.#[br]
|
|
|
|
|
|
| #[strong Lemma:] The base form of the word.#[br]
|
|
|
|
|
|
| #[strong POS:] The simple part-of-speech tag.#[br]
|
2017-05-25 09:17:21 +00:00
|
|
|
|
| #[strong Tag:] The detailed part-of-speech tag.#[br]
|
2017-05-23 21:16:31 +00:00
|
|
|
|
| #[strong Dep:] Syntactic dependency, i.e. the relation between tokens.#[br]
|
|
|
|
|
|
| #[strong Shape:] The word shape – capitalisation, punctuation, digits.#[br]
|
|
|
|
|
|
| #[strong is alpha:] Is the token an alpha character?#[br]
|
|
|
|
|
|
| #[strong is stop:] Is the token part of a stop list, i.e. the most common
|
|
|
|
|
|
| words of the language?#[br]
|
|
|
|
|
|
|
|
|
|
|
|
+table(["Text", "Lemma", "POS", "Tag", "Dep", "Shape", "alpha", "stop"])
|
|
|
|
|
|
- var style = [0, 0, 1, 1, 1, 1, 1, 1]
|
|
|
|
|
|
+annotation-row(["Apple", "apple", "PROPN", "NNP", "nsubj", "Xxxxx", true, false], style)
|
|
|
|
|
|
+annotation-row(["is", "be", "VERB", "VBZ", "aux", "xx", true, true], style)
|
|
|
|
|
|
+annotation-row(["looking", "look", "VERB", "VBG", "ROOT", "xxxx", true, false], style)
|
|
|
|
|
|
+annotation-row(["at", "at", "ADP", "IN", "prep", "xx", true, true], style)
|
|
|
|
|
|
+annotation-row(["buying", "buy", "VERB", "VBG", "pcomp", "xxxx", true, false], style)
|
|
|
|
|
|
+annotation-row(["U.K.", "u.k.", "PROPN", "NNP", "compound", "X.X.", false, false], style)
|
|
|
|
|
|
+annotation-row(["startup", "startup", "NOUN", "NN", "dobj", "xxxx", true, false], style)
|
|
|
|
|
|
+annotation-row(["for", "for", "ADP", "IN", "prep", "xxx", true, true], style)
|
|
|
|
|
|
+annotation-row(["$", "$", "SYM", "$", "quantmod", "$", false, false], style)
|
|
|
|
|
|
+annotation-row(["1", "1", "NUM", "CD", "compound", "d", false, false], style)
|
|
|
|
|
|
+annotation-row(["billion", "billion", "NUM", "CD", "pobj", "xxxx", true, false], style)
|
|
|
|
|
|
|
|
|
|
|
|
+aside("Tip: Understanding tags and labels")
|
|
|
|
|
|
| Most of the tags and labels look pretty abstract, and they vary between
|
|
|
|
|
|
| languages. #[code spacy.explain()] will show you a short description –
|
|
|
|
|
|
| for example, #[code spacy.explain("VBZ")] returns "verb, 3rd person
|
|
|
|
|
|
| singular present".
|
|
|
|
|
|
|
|
|
|
|
|
p
|
2017-10-03 12:26:20 +00:00
|
|
|
|
| Using spaCy's built-in #[+a("/usage/visualizers") displaCy visualizer],
|
2017-05-23 21:16:31 +00:00
|
|
|
|
| here's what our example sentence and its dependencies look like:
|
|
|
|
|
|
|
|
|
|
|
|
+codepen("030d1e4dfa6256cad8fdd59e6aefecbe", 460)
|
