Rooba
/
spaCy
mirror of https://github.com/explosion/spaCy.git
1
1
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Update annotation docs

This commit is contained in:
ines 2017-05-23 23:16:17 +02:00
parent 786af87ffb
commit a38393e2f6
1 changed files with 27 additions and 11 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

38
website/docs/api/annotation.jade
View File

@ -14,11 +14,12 @@ p
| (#[code ' ']) is included as a token. | (#[code ' ']) is included as a token.
+aside-code("Example"). +aside-code("Example").
from spacy.en import English from spacy.lang.en import English
nlp = English(parser=False) nlp = English()
tokens = nlp('Some\nspaces and\ttab characters') tokens = nlp('Some\nspaces and\ttab characters')
print([t.orth_ for t in tokens]) tokens_text = [t.text for t in tokens]
# ['Some', '\n', 'spaces', ' ', 'and', '\t', 'tab', 'characters'] assert tokens_text == ['Some', '\n', 'spaces', ' ', 'and',
'\t', 'tab', 'characters']
p p
| The whitespace tokens are useful for much the same reason punctuation is | The whitespace tokens are useful for much the same reason punctuation is
@ -38,6 +39,11 @@ p
+h(2, "pos-tagging") Part-of-speech Tagging +h(2, "pos-tagging") Part-of-speech Tagging
+aside("Tip: Understanding tags")
| You can also use #[code spacy.explain()] to get the escription for the
| string representation of a tag. For example,
| #[code spacy.explain("RB")] will return "adverb".
include _annotation/_pos-tags include _annotation/_pos-tags
+h(2, "lemmatization") Lemmatization +h(2, "lemmatization") Lemmatization
@ -50,25 +56,35 @@ p A "lemma" is the uninflected form of a word. In English, this means:
+item #[strong Nouns]: The form like "dog", not "dogs"; like "child", not "children" +item #[strong Nouns]: The form like "dog", not "dogs"; like "child", not "children"
+item #[strong Verbs]: The form like "write", not "writes", "writing", "wrote" or "written" +item #[strong Verbs]: The form like "write", not "writes", "writing", "wrote" or "written"
+aside("About spaCy's custom pronoun lemma")
| Unlike verbs and common nouns, there's no clear base form of a personal
| pronoun. Should the lemma of "me" be "I", or should we normalize person
| as well, giving "it" — or maybe "he"? spaCy's solution is to introduce a
| novel symbol, #[code.u-nowrap -PRON-], which is used as the lemma for
| all personal pronouns.
p p
| The lemmatization data is taken from | The lemmatization data is taken from
| #[+a("https://wordnet.princeton.edu") WordNet]. However, we also add a | #[+a("https://wordnet.princeton.edu") WordNet]. However, we also add a
| special case for pronouns: all pronouns are lemmatized to the special | special case for pronouns: all pronouns are lemmatized to the special
| token #[code -PRON-]. | token #[code -PRON-].
+infobox("About spaCy's custom pronoun lemma")
| Unlike verbs and common nouns, there's no clear base form of a personal
| pronoun. Should the lemma of "me" be "I", or should we normalize person
| as well, giving "it" — or maybe "he"? spaCy's solution is to introduce a
| novel symbol, #[code -PRON-], which is used as the lemma for
| all personal pronouns.
+h(2, "dependency-parsing") Syntactic Dependency Parsing +h(2, "dependency-parsing") Syntactic Dependency Parsing
+aside("Tip: Understanding labels")
| You can also use #[code spacy.explain()] to get the description for the
| string representation of a label. For example,
| #[code spacy.explain("prt")] will return "particle".
include _annotation/_dep-labels include _annotation/_dep-labels
+h(2, "named-entities") Named Entity Recognition +h(2, "named-entities") Named Entity Recognition
+aside("Tip: Understanding entity types")
| You can also use #[code spacy.explain()] to get the description for the
| string representation of an entity label. For example,
| #[code spacy.explain("LANGUAGE")] will return "any named language".
include _annotation/_named-entities include _annotation/_named-entities
+h(3, "biluo") BILUO Scheme +h(3, "biluo") BILUO Scheme