6f777425ea
Remove --reverse_task argument
...
If you want to train on the reverse Almond task, use "reverse_almond"
as a task name, as you should.
2019-03-19 10:03:28 -07:00
5989846771
Make use of task classes
...
And clean up the metric handling code as well
2019-03-19 10:01:45 -07:00
92aade6a66
Introduce a registry mechanism and a class hierarchy for tasks
...
Most tasks can be framed as CQA naturally, with the task specific
code living in the dataset code, but some require extra task-specific
handling (IDs, metrics, tokenization, etc.)
In preparation for having even more task specific handling for Almond,
start cleaning up by creating a class for each task.
2019-03-19 09:15:19 -07:00
23f7d3e28d
fix grad_norm computation
2019-03-14 09:38:11 -07:00
d2802455a3
Die immediately with a NaN loss
2019-03-14 08:59:31 -07:00
c795e6522a
train: log learning rate and gradient norm
...
Which can be useful for hyperparameter tuning
2019-03-13 23:29:29 -07:00
292419b299
adding weight regularization option
2019-03-13 17:24:10 -07:00
f65d939f9a
fix
2019-03-13 16:34:12 -07:00
919768b42b
server/predict: unify code loading config.json
2019-03-13 15:49:21 -07:00
a7ce796d76
Add Almond type embeddings
...
A better representation for typed (delexicalized) constants in
the Almond task
2019-03-13 15:48:46 -07:00
ea5655c957
Unify word embedding loading code
...
Into a new module
2019-03-13 10:42:04 -07:00
60dbc94028
fixing checkpoint saving procedure
2019-03-08 18:52:46 -08:00
6e8d5798d9
update tests accordingly
2019-03-04 19:37:23 -08:00
1f51e63253
Actually use eval_dir to store prediction files
...
os.path.join() ignores eval_dir if the checkpoints are stored in a directory with absolute path.
just use the provided eval_dir and everything works out.
2019-03-04 18:56:26 -08:00
cc2c1895c4
update export_results
2019-03-04 17:13:32 -08:00
5ca6c25a96
updates
2019-03-04 16:00:21 -08:00
b3bbad24c5
update tests
2019-03-04 15:08:56 -08:00
ef94113a67
let user specifies total number of checkpoints to keep
2019-03-04 12:04:03 -08:00
96ce6dc26f
saver: add .pth to new file
2019-03-04 00:21:42 -08:00
d1e54e9734
saver: actually save checkpoint.json
...
Won't go far without it
2019-03-03 23:37:37 -08:00
c9d8e920a9
Fix logger.debug call
...
It does not take more than one argument
2019-03-03 22:50:30 -08:00
b950927a2b
Clean up old checkpoints as we go along
...
Introduce an utility Saver class, that does what tensorflow's Saver
does: keeps track of saved checkpoints in a separate file, and
deletes the old ones before saving a new one.
2019-03-03 22:35:29 -08:00
410c6cd8ec
Merge remote-tracking branch 'origin/master' into wip/more-cleanups
2019-03-03 22:16:09 -08:00
c2406e62e0
Fix "get_commit" when invoking "decanlp" installed with "pip install -e"
...
sys.argv will be a script living in ~/.bin that loads and executes
the module, so it will not live in the git repository
2019-03-02 18:28:04 +00:00
1aa6306702
server: automatically grow the embedding matrix when new words are encountered
...
Otherwise, we feed actual unks to the model, while the model
is trained with character embeddings and expects to see some
actual value for everything.
2019-03-02 00:44:51 -08:00
72175af080
torchtext.field: use <unk> as the unk tokens
...
" UNK " has spaces, and a token with spaces is asking for trouble
2019-03-02 00:17:34 -08:00
dac1e88a8e
server: close gracefully on Ctrl-c and EOF
2019-03-02 00:17:34 -08:00
3a1b969ad3
updates
2019-03-01 17:51:54 -08:00
3a1fb01286
server: flush stdout after a request
2019-03-01 17:40:19 -08:00
ce5c1aa8df
Use logger instead of print()
...
print() uses stdout by default, which has two problems:
- it is not flushed until later (so messages don't show, or don't show
up in order with other loggers)
- it conflicts with stdin/stdout usage by `decanlp server --stdin`
2019-03-01 17:35:04 -08:00
8d4136a79a
server: add a simpler interface that just works over stdin/stdout
2019-03-01 16:29:05 -08:00
1a2a4a9ea9
One more stanford copyright
2019-03-01 16:18:10 -08:00
03ce9501ad
server: remove --data argument
...
The server does not load any data file
2019-03-01 16:14:08 -08:00
bafabac483
Fix argument handling
2019-03-01 16:13:10 -08:00
ad1a15637a
Fix missing import
2019-03-01 16:09:58 -08:00
4bddbada45
Fix typo
2019-03-01 16:08:52 -08:00
ac3ac8c680
Add Stanford copyright to all files that we touched
2019-03-01 15:54:54 -08:00
8e2b519ac3
Add copyright notices to all files
...
Makes the license clear and explicit
2019-03-01 15:51:45 -08:00
99e39f9528
Move generic dataset outside of torchtext
...
There is no reason for it to live inside torchtext, and having it
outside will help with using torchtext as a library
2019-03-01 15:47:11 -08:00
5447d0c37c
Add a "decanlp" script that calls out to the different subcommands
...
Usage:
- decanlp train ...
- decanlp predict ...
- decanlp convert-to-logical-forms ...
2019-03-01 15:43:02 -08:00
41b80bb4f4
Move all python files to a decanlp/ package
...
As per python conventions
2019-03-01 15:43:02 -08:00
Giovanni Campagna