Rooba
/
spaCy
mirror of https://github.com/explosion/spaCy.git
1
1
Fork 0
Code Issues Packages Projects Releases Wiki Activity

* Don't free clobbered lexemes, as they might be part of a tail

This commit is contained in:
Matthew Honnibal 2014-08-01 08:22:38 +01:00
parent c48214460e
commit c7bb6b329c
1 changed files with 4 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
spacy/spacy.pyx
View File

@ -142,7 +142,10 @@ cdef class Language:
word = self.init_lexeme(string, hashed, split, length) word = self.init_lexeme(string, hashed, split, length)
cdef Lexeme* clobbered = <Lexeme*>self.happax.insert(hashed, <size_t>word) cdef Lexeme* clobbered = <Lexeme*>self.happax.insert(hashed, <size_t>word)
if clobbered != NULL: if clobbered != NULL:
free(clobbered) # Can't do this --- we might be pointing to the Lexeme in .tail.
# Fix that to reduce memory, probably.
#free(clobbered)
pass
self.bacov[hashed] = string self.bacov[hashed] = string
return word return word