README_equation_decoding

Under Construction

Conda environment:

Environment file for an opennmt conda environment to be able to run the opennmt model for the equation extraction: https://github.com/ml4ai/automates/blob/master/equation_extraction/opennmt_environment_20191025.yml

Preprocessing:

python preprocess.py \
  -data_type img \
  -src_dir /data/images/ \
  -train_src /data/src_train.txt \ 
  -train_tgt /data/tgt_train.txt \
  -valid_src /data/src_val.txt \
  -valid_tgt /data/tgt_val.txt \
  -save_data /data/saved_shards \
  -tgt_seq_length 150 \
  -tgt_words_min_frequency 2 \
  -shard_size 500 \
  -image_channel_size 1

Normalize formulas In order to normalize the formulas, you need to clone the im2markup repo and make one edit to the code:

diff --git a/scripts/preprocessing/preprocess_latex.js b/scripts/preprocessing/preprocess_latex.js
index 54b8604..86a016b 100644
--- a/scripts/preprocessing/preprocess_latex.js
+++ b/scripts/preprocessing/preprocess_latex.js
@@ -59,7 +59,7 @@ rl.on('line', function(line){
         console.error(line);
         console.error(norm_str);
         console.error(e);
-        console.log("");
+        console.log("XXXXXXXXXX");
     }
     global_str = ""
     norm_str = ""

You will then point to this modified repo in the script here:

python normalize_formulas.py --indir=/work/bsharp/test_norm/ --logfile=norm.log \
--im2markupdir=/work/github/im2markup

Training:

We recommend that you train with a GPU.

CUDA_VISIBLE_DEVICES=1 python train.py -model_type img -data /data/saved_shards \
-save_model /data/models/ -gpu_ranks 0 -batch_size 20 -max_grad_norm 20 \
-learning_rate 0.1 -word_vec_size 80 -encoder_type brnn -image_channel_size 1 --world_size 1

Predict:

Prediction can be done on a GPU or CPU. To run with a CPU, omit CUDA_VISIBLE_DEVICES argument in the following command.

CUDA_VISIBLE_DEVICES=0 python translate.py -data_type img -model /data/models/model_step_XXX.pt \
-src_dir /data/images -src /data/src_test.txt -output predictions.txt -max_length 150 \
-beam_size 5 -gpu 0 -verbose -image_channel_size 1