Almond data is always pretokenized (because we must preprocess
numbers/quoted strings/etc.).
Previously, we used a bad HACK of hardcoding almond in the generic
Field subclass. Now, we instead thread through the tokenizer/detokenizer
arguments from the right places.
In the future, we will probably want task classes, to clean up
the mess of hacks and hardcoded task-specific tweaks everywhere.
The branches are getting messy! This branch is a clean updated version of the codebase which is actually compatible with 1) new version of pytorch 2) Almond project
mehrad