1。 第一个bug
运行
echo "Please tokenize this text." | java edu.stanford.nlp.process.PTBTokenizer 后显示。提示：
- -bash: java: command not found。
那我就觉得可能是java没安装。然后，我就去官网
下载的是放到了/data 目录下，然后解压，
解压完成后，vim ./bashrc,打开，然后输入如下的内容。
保存退出，source ~/.bashrc 一下。
这时候再 echo "Please tokenize this text." | java edu.stanford.nlp.process.PTBTokenizer就可以了。
2. 第二个bug UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 858: ordinal not in range(128) 这个bug
(jjenv_pytorch) root@032ba38f2b6e:/data/rl_abs_other/cnn-dailymail# ls README.md make_datafiles.py url_lists (jjenv_pytorch) root@032ba38f2b6e:/data/rl_abs_other/cnn-dailymail# (jjenv_pytorch) root@032ba38f2b6e:/data/rl_abs_other/cnn-dailymail# python make_datafiles.py /data/rl_abs_other/data/cnn/stories /data/rl_abs_other/data/dailymail/stories Preparing to tokenize /data/rl_abs_other/data/cnn/stories to cnn_stories_tokenized... Making list of files to tokenize... Tokenizing 92579 files in /data/rl_abs_other/data/cnn/stories and saving in cnn_stories_tokenized... Untokenizable: ? (U+20A9, decimal: 8361) Untokenizable: ? (U+202A, decimal: 8234) Untokenizable: ? (U+202A, decimal: 8234) Untokenizable: ? (U+20A9, decimal: 8361) Untokenizable: ? (U+202A, decimal: 8234) Untokenizable: ? (U+202F, decimal: 8239) Untokenizable: ? (U+202F, decimal: 8239) Untokenizable: ? (U+202F, decimal: 8239) Untokenizable: ? (U+20A9, decimal: 8361) Untokenizable: ? (U+202F, decimal: 8239) Untokenizable: ? (U+20A9, decimal: 8361) Untokenizable: ? (U+20A9, decimal: 8361) Untokenizable: ? (U+202A, decimal: 8234) Untokenizable: ? (U+202F, decimal: 8239) Untokenizable: ? (U+202C, decimal: 8236) Untokenizable: ? (U+202F, decimal: 8239) Untokenizable: ? (U+20A9, decimal: 8361) Untokenizable: ? (U+20A9, decimal: 8361) Untokenizable: ? (U+202A, decimal: 8234) Untokenizable: ? (U+202C, decimal: 8236) Untokenizable: ? (U+202A, decimal: 8234) Untokenizable: ? (U+20A9, decimal: 8361) Untokenizable: ? (U+20A9, decimal: 8361) Untokenizable: ? (U+202A, decimal: 8234) Untokenizable: ? (U+20A9, decimal: 8361) Untokenizable: ? (U+20A9, decimal: 8361) Untokenizable: ? (U+20A9, decimal: 8361) Untokenizable: ? (U+20A9, decimal: 8361) Untokenizable: ? (U+F06E, decimal: 61550) Untokenizable: ? (U+202F, decimal: 8239) Untokenizable: ? (U+202C, decimal: 8236) Untokenizable: ? (U+F022, decimal: 61474) Untokenizable: ? (U+202C, decimal: 8236) Untokenizable: ? (U+202A, decimal: 8234) Untokenizable: ? (U+202A, decimal: 8234) PTBTokenizer tokenized 80043350 tokens at 42671.94 tokens per second. Stanford CoreNLP Tokenizer has finished. Successfully finished tokenizing /data/rl_abs_other/data/cnn/stories to cnn_stories_tokenized. Preparing to tokenize /data/rl_abs_other/data/dailymail/stories to dm_stories_tokenized... Making list of files to tokenize... Tokenizing 219506 files in /data/rl_abs_other/data/dailymail/stories and saving in dm_stories_tokenized... Untokenizable: ? (U+FFFC, decimal: 65532) Untokenizable: ? (U+2010, decimal: 8208) Untokenizable: ? (U+202A, decimal: 8234) Untokenizable: ? (U+FFFD, decimal: 65533) Untokenizable: ? (U+202A, decimal: 8234) Untokenizable: ? (U+FFFD, decimal: 65533) Untokenizable: ? (U+202D, decimal: 8237) Untokenizable: ? (U+202A, decimal: 8234) Untokenizable: ? (U+FFFD, decimal: 65533) Untokenizable: ? (U+202C, decimal: 8236) Untokenizable: ? (U+202C, decimal: 8236) Untokenizable: ? (U+202A, decimal: 8234) Untokenizable: ? (U+2012, decimal: 8210) Untokenizable: ? (U+202C, decimal: 8236) Untokenizable: ? (U+202A, decimal: 8234) Untokenizable: ? (U+202A, decimal: 8234) Untokenizable: ? (U+2010, decimal: 8208) Untokenizable: ? (U+202C, decimal: 8236) Untokenizable: ? (U+202A, decimal: 8234) Untokenizable: ? (U+202D, decimal: 8237) Untokenizable: ? (U+202A, decimal: 8234) Untokenizable: ? (U+FFFD, decimal: 65533) Untokenizable: ? (U+202C, decimal: 8236) Untokenizable: ? (U+FFFD, decimal: 65533) Untokenizable: ? (U+202A, decimal: 8234) Untokenizable: ? (U+202C, decimal: 8236) Untokenizable: ? (U+202A, decimal: 8234) Untokenizable: ? (U+202A, decimal: 8234) Untokenizable: ? (U+FFFD, decimal: 65533) Untokenizable: ? (U+2010, decimal: 8208) Untokenizable: ? (U+202A, decimal: 8234) Untokenizable: ? (U+202A, decimal: 8234) Untokenizable: ? (U+2010, decimal: 8208) Untokenizable: ? (U+202A, decimal: 8234) Untokenizable: ? (U+202A, decimal: 8234) Untokenizable: ? (U+202A, decimal: 8234) Untokenizable: ? (U+2010, decimal: 8208) Untokenizable: ? (U+202F, decimal: 8239) Untokenizable: ? (U+202A, decimal: 8234) Untokenizable: ? (U+202A, decimal: 8234) Untokenizable: ? (U+FFFD, decimal: 65533) Untokenizable: ? (U+202B, decimal: 8235) Untokenizable: ? (U+202C, decimal: 8236) Untokenizable: ? (U+202A, decimal: 8234) Untokenizable: ? (U+202C, decimal: 8236) Untokenizable: ? (U+202D, decimal: 8237) Untokenizable: ? (U+202C, decimal: 8236) Untokenizable: ? (U+202C, decimal: 8236) Untokenizable: ? (U+FFFD, decimal: 65533) Untokenizable: ? (U+202A, decimal: 8234) Untokenizable: ? (U+202A, decimal: 8234) Untokenizable: ? (U+202A, decimal: 8234) Untokenizable: ? (U+FFFD, decimal: 65533) Untokenizable: ? (U+202C, decimal: 8236) Untokenizable: ? (U+202A, decimal: 8234) Untokenizable: ? (U+FFFD, decimal: 65533) Untokenizable: ? (U+202A, decimal: 8234) Untokenizable: ? (U+FFFD, decimal: 65533) Untokenizable: ? (U+202A, decimal: 8234) Untokenizable: ? (U+2010, decimal: 8208) Untokenizable: ? (U+FFFD, decimal: 65533) Untokenizable: ? (U+FFFD, decimal: 65533) Untokenizable: ? (U+202A, decimal: 8234) Untokenizable: ? (U+F001, decimal: 61441) Untokenizable: ? (U+202C, decimal: 8236) Untokenizable: ? (U+202A, decimal: 8234) Untokenizable: ? (U+202A, decimal: 8234) Untokenizable: ? (U+202C, decimal: 8236) Untokenizable: ? (U+202A, decimal: 8234) Untokenizable: ? (U+202A, decimal: 8234) Untokenizable: ? (U+202A, decimal: 8234) Untokenizable: ? (U+202A, decimal: 8234) Untokenizable: ? (U+F001, decimal: 61441) Untokenizable: ? (U+FFFD, decimal: 65533) Untokenizable: ? (U+202A, decimal: 8234) Untokenizable: ? (U+202A, decimal: 8234) Untokenizable: ? (U+202A, decimal: 8234) Untokenizable: ? (U+70E, decimal: 1806) Untokenizable: ? (U+202A, decimal: 8234) Untokenizable: ? (U+202A, decimal: 8234) Untokenizable: ? (U+202A, decimal: 8234) Untokenizable: ? (U+FFFD, decimal: 65533) Untokenizable: ? (U+202A, decimal: 8234) Untokenizable: ? (U+FFFD, decimal: 65533) Untokenizable: ? (U+202F, decimal: 8239) Untokenizable: ? (U+2010, decimal: 8208) Untokenizable: ? (U+202A, decimal: 8234) Untokenizable: ? (U+202C, decimal: 8236) Untokenizable: ? (U+202A, decimal: 8234) Untokenizable: ? (U+202A, decimal: 8234) Untokenizable: ? (U+FFFD, decimal: 65533) Untokenizable: ? (U+202C, decimal: 8236) Untokenizable: ? (U+FFFD, decimal: 65533) Untokenizable: ? (U+202C, decimal: 8236) Untokenizable: ? (U+202A, decimal: 8234) Untokenizable: ? (U+202A, decimal: 8234) Untokenizable: ? (U+FFFD, decimal: 65533) Untokenizable: ? (U+206E, decimal: 8302) Untokenizable: ? (U+202A, decimal: 8234) Untokenizable: ? (U+202A, decimal: 8234) Untokenizable: ? (U+202A, decimal: 8234) Untokenizable: ? (U+202A, decimal: 8234) Untokenizable: ? (U+202A, decimal: 8234) Untokenizable: ? (U+202A, decimal: 8234) Untokenizable: ? (U+202A, decimal: 8234) Untokenizable: ? (U+FFFD, decimal: 65533) Untokenizable: ? (U+202A, decimal: 8234) Untokenizable: ? (U+200D, decimal: 8205) Untokenizable: ? (U+202A, decimal: 8234) Untokenizable: ? (U+FFFD, decimal: 65533) Untokenizable: ? (U+202C, decimal: 8236) PTBTokenizer tokenized 203118231 tokens at 32507.27 tokens per second. Stanford CoreNLP Tokenizer has finished. Successfully finished tokenizing /data/rl_abs_other/data/dailymail/stories to dm_stories_tokenized. Making bin file for URLs listed in url_lists/all_test.txt... Writing story 0 of 11490; 0.00 percent done Traceback (most recent call last): File "make_datafiles.py", line 253, in <module> write_to_tar(all_test_urls, os.path.join(finished_files_dir, "test.tar")) File "make_datafiles.py", line 182, in write_to_tar article_sents, abstract_sents = get_art_abs(story_file) File "make_datafiles.py", line 106, in get_art_abs lines = read_story_file(story_file) File "make_datafiles.py", line 78, in read_story_file lines = f.read().split(' ') File "/root/anaconda3/envs/jjenv_pytorch/lib/python3.6/encodings/ascii.py", line 26, in decode return codecs.ascii_decode(input, self.errors)[0] UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 858: ordinal not in range(128) (jjenv_pytorch) root@032ba38f2b6e:/data/rl_abs_other/cnn-dailymail#
然后我以为是编码问题，就去 make_datafiles.py 的文件开头加上 # coding: utf-8 ,但是没有解决问题，后来参考了一篇帖子https://blog.csdn.net/qq_36847641/article/details/78414718
所以就把我自己的代码，做如下更改，就可以了。
但是，
然后我就继续运行make_datafiles.py文件，然后一路都顺利直到完成。
(jjenv_pytorch) root@032ba38f2b6e:/data/rl_abs_other/cnn-dailymail# python make_datafiles.py /data/rl/rl_abs_other/data/dailymail/stories Making bin file for URLs listed in url_lists/all_test.txt... Writing story 0 of 11490; 0.00 percent done Writing story 1000 of 11490; 8.70 percent done Writing story 2000 of 11490; 17.41 percent done Writing story 3000 of 11490; 26.11 percent done Writing story 4000 of 11490; 34.81 percent done Writing story 5000 of 11490; 43.52 percent done Writing story 6000 of 11490; 52.22 percent done Writing story 7000 of 11490; 60.92 percent done Writing story 8000 of 11490; 69.63 percent done Writing story 9000 of 11490; 78.33 percent done Writing story 10000 of 11490; 87.03 percent done Writing story 11000 of 11490; 95.74 percent done Finished writing file finished_files/test.tar Making bin file for URLs listed in url_lists/all_val.txt... Writing story 0 of 13368; 0.00 percent done Writing story 1000 of 13368; 7.48 percent done Writing story 2000 of 13368; 14.96 percent done Writing story 3000 of 13368; 22.44 percent done Writing story 4000 of 13368; 29.92 percent done Writing story 5000 of 13368; 37.40 percent done Writing story 6000 of 13368; 44.88 percent done Writing story 7000 of 13368; 52.36 percent done Writing story 8000 of 13368; 59.84 percent done Writing story 9000 of 13368; 67.32 percent done Writing story 10000 of 13368; 74.81 percent done Writing story 11000 of 13368; 82.29 percent done Writing story 12000 of 13368; 89.77 percent done Writing story 13000 of 13368; 97.25 percent done Finished writing file finished_files/val.tar Making bin file for URLs listed in url_lists/all_train.txt... Writing story 0 of 287227; 0.00 percent done Writing story 1000 of 287227; 0.35 percent done Writing story 2000 of 287227; 0.70 percent done Writing story 3000 of 287227; 1.04 percent done Writing story 4000 of 287227; 1.39 percent done Writing story 5000 of 287227; 1.74 percent done Writing story 6000 of 287227; 2.09 percent done Writing story 7000 of 287227; 2.44 percent done Writing story 8000 of 287227; 2.79 percent done Writing story 9000 of 287227; 3.13 percent done Writing story 10000 of 287227; 3.48 percent done Writing story 11000 of 287227; 3.83 percent done Writing story 12000 of 287227; 4.18 percent done Writing story 13000 of 287227; 4.53 percent done Writing story 14000 of 287227; 4.87 percent done Writing story 15000 of 287227; 5.22 percent done Writing story 16000 of 287227; 5.57 percent done Writing story 17000 of 287227; 5.92 percent done Writing story 18000 of 287227; 6.27 percent done Writing story 19000 of 287227; 6.61 percent done Writing story 20000 of 287227; 6.96 percent done Writing story 21000 of 287227; 7.31 percent done Writing story 22000 of 287227; 7.66 percent done Writing story 23000 of 287227; 8.01 percent done Writing story 24000 of 287227; 8.36 percent done Writing story 25000 of 287227; 8.70 percent done Writing story 26000 of 287227; 9.05 percent done Writing story 27000 of 287227; 9.40 percent done Writing story 28000 of 287227; 9.75 percent done Writing story 29000 of 287227; 10.10 percent done Writing story 30000 of 287227; 10.44 percent done Writing story 31000 of 287227; 10.79 percent done Writing story 32000 of 287227; 11.14 percent done Writing story 33000 of 287227; 11.49 percent done Writing story 34000 of 287227; 11.84 percent done Writing story 35000 of 287227; 12.19 percent done Writing story 36000 of 287227; 12.53 percent done Writing story 37000 of 287227; 12.88 percent done Writing story 38000 of 287227; 13.23 percent done Writing story 39000 of 287227; 13.58 percent done Writing story 40000 of 287227; 13.93 percent done Writing story 41000 of 287227; 14.27 percent done Writing story 42000 of 287227; 14.62 percent done Writing story 43000 of 287227; 14.97 percent done Writing story 44000 of 287227; 15.32 percent done Writing story 45000 of 287227; 15.67 percent done Writing story 46000 of 287227; 16.02 percent done Writing story 47000 of 287227; 16.36 percent done Writing story 48000 of 287227; 16.71 percent done Writing story 49000 of 287227; 17.06 percent done Writing story 50000 of 287227; 17.41 percent done Writing story 51000 of 287227; 17.76 percent done Writing story 52000 of 287227; 18.10 percent done Writing story 53000 of 287227; 18.45 percent done Writing story 54000 of 287227; 18.80 percent done Writing story 55000 of 287227; 19.15 percent done Writing story 56000 of 287227; 19.50 percent done Writing story 57000 of 287227; 19.84 percent done Writing story 58000 of 287227; 20.19 percent done Writing story 59000 of 287227; 20.54 percent done Writing story 60000 of 287227; 20.89 percent done Writing story 61000 of 287227; 21.24 percent done Writing story 62000 of 287227; 21.59 percent done Writing story 63000 of 287227; 21.93 percent done Writing story 64000 of 287227; 22.28 percent done Writing story 65000 of 287227; 22.63 percent done Writing story 66000 of 287227; 22.98 percent done Writing story 67000 of 287227; 23.33 percent done Writing story 68000 of 287227; 23.67 percent done Writing story 69000 of 287227; 24.02 percent done Writing story 70000 of 287227; 24.37 percent done Writing story 71000 of 287227; 24.72 percent done Writing story 72000 of 287227; 25.07 percent done Writing story 73000 of 287227; 25.42 percent done Writing story 74000 of 287227; 25.76 percent done Writing story 75000 of 287227; 26.11 percent done Writing story 76000 of 287227; 26.46 percent done Writing story 77000 of 287227; 26.81 percent done Writing story 78000 of 287227; 27.16 percent done Writing story 79000 of 287227; 27.50 percent done Writing story 80000 of 287227; 27.85 percent done Writing story 81000 of 287227; 28.20 percent done Writing story 82000 of 287227; 28.55 percent done Writing story 83000 of 287227; 28.90 percent done Writing story 84000 of 287227; 29.25 percent done Writing story 85000 of 287227; 29.59 percent done Writing story 86000 of 287227; 29.94 percent done Writing story 87000 of 287227; 30.29 percent done Writing story 88000 of 287227; 30.64 percent done Writing story 89000 of 287227; 30.99 percent done Writing story 90000 of 287227; 31.33 percent done Writing story 91000 of 287227; 31.68 percent done Writing story 92000 of 287227; 32.03 percent done Writing story 93000 of 287227; 32.38 percent done Writing story 94000 of 287227; 32.73 percent done Writing story 95000 of 287227; 33.07 percent done Writing story 96000 of 287227; 33.42 percent done Writing story 97000 of 287227; 33.77 percent done Writing story 98000 of 287227; 34.12 percent done Writing story 99000 of 287227; 34.47 percent done Writing story 100000 of 287227; 34.82 percent done Writing story 101000 of 287227; 35.16 percent done Writing story 102000 of 287227; 35.51 percent done Writing story 103000 of 287227; 35.86 percent done Writing story 104000 of 287227; 36.21 percent done Writing story 105000 of 287227; 36.56 percent done Writing story 106000 of 287227; 36.90 percent done Writing story 107000 of 287227; 37.25 percent done Writing story 108000 of 287227; 37.60 percent done Writing story 109000 of 287227; 37.95 percent done Writing story 110000 of 287227; 38.30 percent done Writing story 111000 of 287227; 38.65 percent done Writing story 112000 of 287227; 38.99 percent done Writing story 113000 of 287227; 39.34 percent done Writing story 114000 of 287227; 39.69 percent done Writing story 115000 of 287227; 40.04 percent done Writing story 116000 of 287227; 40.39 percent done Writing story 117000 of 287227; 40.73 percent done Writing story 118000 of 287227; 41.08 percent done Writing story 119000 of 287227; 41.43 percent done Writing story 120000 of 287227; 41.78 percent done Writing story 121000 of 287227; 42.13 percent done Writing story 122000 of 287227; 42.48 percent done Writing story 123000 of 287227; 42.82 percent done Writing story 124000 of 287227; 43.17 percent done Writing story 125000 of 287227; 43.52 percent done Writing story 126000 of 287227; 43.87 percent done Writing story 127000 of 287227; 44.22 percent done Writing story 128000 of 287227; 44.56 percent done Writing story 129000 of 287227; 44.91 percent done Writing story 130000 of 287227; 45.26 percent done Writing story 131000 of 287227; 45.61 percent done Writing story 132000 of 287227; 45.96 percent done Writing story 133000 of 287227; 46.30 percent done Writing story 134000 of 287227; 46.65 percent done Writing story 135000 of 287227; 47.00 percent done Writing story 136000 of 287227; 47.35 percent done Writing story 137000 of 287227; 47.70 percent done Writing story 138000 of 287227; 48.05 percent done Writing story 139000 of 287227; 48.39 percent done Writing story 140000 of 287227; 48.74 percent done Writing story 141000 of 287227; 49.09 percent done Writing story 142000 of 287227; 49.44 percent done Writing story 143000 of 287227; 49.79 percent done Writing story 144000 of 287227; 50.13 percent done Writing story 145000 of 287227; 50.48 percent done Writing story 146000 of 287227; 50.83 percent done Writing story 147000 of 287227; 51.18 percent done Writing story 148000 of 287227; 51.53 percent done Writing story 149000 of 287227; 51.88 percent done Writing story 150000 of 287227; 52.22 percent done Writing story 151000 of 287227; 52.57 percent done Writing story 152000 of 287227; 52.92 percent done Writing story 153000 of 287227; 53.27 percent done Writing story 154000 of 287227; 53.62 percent done Writing story 155000 of 287227; 53.96 percent done Writing story 156000 of 287227; 54.31 percent done Writing story 157000 of 287227; 54.66 percent done Writing story 158000 of 287227; 55.01 percent done Writing story 159000 of 287227; 55.36 percent done Writing story 160000 of 287227; 55.71 percent done Writing story 161000 of 287227; 56.05 percent done Writing story 162000 of 287227; 56.40 percent done Writing story 163000 of 287227; 56.75 percent done Writing story 164000 of 287227; 57.10 percent done Writing story 165000 of 287227; 57.45 percent done Writing story 166000 of 287227; 57.79 percent done Writing story 167000 of 287227; 58.14 percent done Writing story 168000 of 287227; 58.49 percent done Writing story 169000 of 287227; 58.84 percent done Writing story 170000 of 287227; 59.19 percent done Writing story 171000 of 287227; 59.53 percent done Writing story 172000 of 287227; 59.88 percent done Writing story 173000 of 287227; 60.23 percent done Writing story 174000 of 287227; 60.58 percent done Writing story 175000 of 287227; 60.93 percent done Writing story 176000 of 287227; 61.28 percent done Writing story 177000 of 287227; 61.62 percent done Writing story 178000 of 287227; 61.97 percent done Writing story 179000 of 287227; 62.32 percent done Writing story 180000 of 287227; 62.67 percent done Writing story 181000 of 287227; 63.02 percent done Writing story 182000 of 287227; 63.36 percent done Writing story 183000 of 287227; 63.71 percent done Writing story 184000 of 287227; 64.06 percent done Writing story 185000 of 287227; 64.41 percent done Writing story 186000 of 287227; 64.76 percent done Writing story 187000 of 287227; 65.11 percent done Writing story 188000 of 287227; 65.45 percent done Writing story 189000 of 287227; 65.80 percent done Writing story 190000 of 287227; 66.15 percent done Writing story 191000 of 287227; 66.50 percent done Writing story 192000 of 287227; 66.85 percent done Writing story 193000 of 287227; 67.19 percent done Writing story 194000 of 287227; 67.54 percent done Writing story 195000 of 287227; 67.89 percent done Writing story 196000 of 287227; 68.24 percent done Writing story 197000 of 287227; 68.59 percent done Writing story 198000 of 287227; 68.94 percent done Writing story 199000 of 287227; 69.28 percent done Writing story 200000 of 287227; 69.63 percent done Writing story 201000 of 287227; 69.98 percent done Writing story 202000 of 287227; 70.33 percent done Writing story 203000 of 287227; 70.68 percent done Writing story 204000 of 287227; 71.02 percent done Writing story 205000 of 287227; 71.37 percent done Writing story 206000 of 287227; 71.72 percent done Writing story 207000 of 287227; 72.07 percent done Writing story 208000 of 287227; 72.42 percent done Writing story 209000 of 287227; 72.76 percent done Writing story 210000 of 287227; 73.11 percent done Writing story 211000 of 287227; 73.46 percent done Writing story 212000 of 287227; 73.81 percent done Writing story 213000 of 287227; 74.16 percent done Writing story 214000 of 287227; 74.51 percent done Writing story 215000 of 287227; 74.85 percent done Writing story 216000 of 287227; 75.20 percent done Writing story 217000 of 287227; 75.55 percent done Writing story 218000 of 287227; 75.90 percent done Writing story 219000 of 287227; 76.25 percent done Writing story 220000 of 287227; 76.59 percent done Writing story 221000 of 287227; 76.94 percent done Writing story 222000 of 287227; 77.29 percent done Writing story 223000 of 287227; 77.64 percent done Writing story 224000 of 287227; 77.99 percent done Writing story 225000 of 287227; 78.34 percent done Writing story 226000 of 287227; 78.68 percent done Writing story 227000 of 287227; 79.03 percent done Writing story 228000 of 287227; 79.38 percent done Writing story 229000 of 287227; 79.73 percent done Writing story 230000 of 287227; 80.08 percent done Writing story 231000 of 287227; 80.42 percent done Writing story 232000 of 287227; 80.77 percent done Writing story 233000 of 287227; 81.12 percent done Writing story 234000 of 287227; 81.47 percent done Writing story 235000 of 287227; 81.82 percent done Writing story 236000 of 287227; 82.16 percent done Writing story 237000 of 287227; 82.51 percent done Writing story 238000 of 287227; 82.86 percent done Writing story 239000 of 287227; 83.21 percent done Writing story 240000 of 287227; 83.56 percent done Writing story 241000 of 287227; 83.91 percent done Writing story 242000 of 287227; 84.25 percent done Writing story 243000 of 287227; 84.60 percent done Writing story 244000 of 287227; 84.95 percent done Writing story 245000 of 287227; 85.30 percent done Writing story 246000 of 287227; 85.65 percent done Writing story 247000 of 287227; 85.99 percent done Writing story 248000 of 287227; 86.34 percent done Writing story 249000 of 287227; 86.69 percent done Writing story 250000 of 287227; 87.04 percent done Writing story 251000 of 287227; 87.39 percent done Writing story 252000 of 287227; 87.74 percent done Writing story 253000 of 287227; 88.08 percent done Writing story 254000 of 287227; 88.43 percent done Writing story 255000 of 287227; 88.78 percent done Writing story 256000 of 287227; 89.13 percent done Writing story 257000 of 287227; 89.48 percent done Writing story 258000 of 287227; 89.82 percent done Writing story 259000 of 287227; 90.17 percent done Writing story 260000 of 287227; 90.52 percent done Writing story 261000 of 287227; 90.87 percent done Writing story 262000 of 287227; 91.22 percent done Writing story 263000 of 287227; 91.57 percent done Writing story 264000 of 287227; 91.91 percent done Writing story 265000 of 287227; 92.26 percent done Writing story 266000 of 287227; 92.61 percent done Writing story 267000 of 287227; 92.96 percent done Writing story 268000 of 287227; 93.31 percent done Writing story 269000 of 287227; 93.65 percent done Writing story 270000 of 287227; 94.00 percent done Writing story 271000 of 287227; 94.35 percent done Writing story 272000 of 287227; 94.70 percent done Writing story 273000 of 287227; 95.05 percent done Writing story 274000 of 287227; 95.39 percent done Writing story 275000 of 287227; 95.74 percent done Writing story 276000 of 287227; 96.09 percent done Writing story 277000 of 287227; 96.44 percent done Writing story 278000 of 287227; 96.79 percent done Writing story 279000 of 287227; 97.14 percent done Writing story 280000 of 287227; 97.48 percent done Writing story 281000 of 287227; 97.83 percent done Writing story 282000 of 287227; 98.18 percent done Writing story 283000 of 287227; 98.53 percent done Writing story 284000 of 287227; 98.88 percent done Writing story 285000 of 287227; 99.22 percent done Writing story 286000 of 287227; 99.57 percent done Writing story 287000 of 287227; 99.92 percent done Finished writing file finished_files/train.tar Writing vocab file... Finished writing vocab file