Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion official/README-TPU.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
BERT, which stands for Bidirectional Encoder Representations from
Transformers.
[BERT FineTuning with Cloud TPU](https://cloud.google.com/ai-platform/training/docs/algorithms/bert-start) provides step by step instructions on Cloud TPU training. You can look [Bert MNLI Tensorboard.dev metrics](https://tensorboard.dev/experiment/LijZ1IrERxKALQfr76gndA) for MNLI fine tuning task.
* [transformer](nlp/transformer): A transformer model to translate the WMT
* [transformer](legacy/transformer): A transformer model to translate the WMT
English to German dataset.
[Training transformer on Cloud TPU](https://cloud.google.com/tpu/docs/tutorials/transformer-2.x) for step by step instructions on Cloud TPU training.

Expand Down
6 changes: 3 additions & 3 deletions official/legacy/bert/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ in order to keep consistent with BERT paper.
Pretrained checkpoints can be found in the following links:

**Note: We have switched BERT implementation
to use Keras functional-style networks in [nlp/modeling](../modeling).
to use Keras functional-style networks in [nlp/modeling](../../nlp/modeling).
The new checkpoints are:**

* **[`BERT-Large, Uncased (Whole Word Masking)`](https://storage.googleapis.com/cloud-tpu-checkpoints/bert/keras_bert/wwm_uncased_L-24_H-1024_A-16.tar.gz)**:
Expand Down Expand Up @@ -126,7 +126,7 @@ pip install tf-nightly
### Pre-training

There is no change to generate pre-training data. Please use the script
[`../data/create_pretraining_data.py`](../data/create_pretraining_data.py)
[`../../nlp/data/create_pretraining_data.py`](../../nlp/data/create_pretraining_data.py)
which is essentially branched from the [BERT research repo](https://github.com/google-research/bert)
to get processed pre-training data and it adapts to TF2 symbols and python3
compatibility.
Expand All @@ -152,7 +152,7 @@ python models/official/nlp/data/create_pretraining_data.py \
### Fine-tuning

To prepare the fine-tuning data for final model training, use the
[`../data/create_finetuning_data.py`](../data/create_finetuning_data.py) script.
[`../../nlp/data/create_finetuning_data.py`](../../nlp/data/create_finetuning_data.py) script.
Resulting datasets in `tf_record` format and training meta data should be later
passed to training or evaluation scripts. The task-specific arguments are
described in the following sections:
Expand Down
2 changes: 1 addition & 1 deletion official/legacy/transformer/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -207,7 +207,7 @@ A brief look at each component in the code:
* [ffn_layer.py](ffn_layer.py): Defines the feedforward network that is used in the encoder/decoder stacks. The network is composed of 2 fully connected layers.

Other files:
* [beam_search.py](beam_search.py) contains the beam search implementation, which is used during model inference to find high scoring translations.
* [beam_search.py](../../nlp/modeling/ops/beam_search.py) contains the beam search implementation, which is used during model inference to find high scoring translations.

### Model Trainer
[transformer_main.py](transformer_main.py) creates an `TransformerTask` to train and evaluate the model using tf.keras.
Expand Down
2 changes: 1 addition & 1 deletion official/nlp/docs/train.md
Original file line number Diff line number Diff line change
Expand Up @@ -189,7 +189,7 @@ python3 data/create_finetuning_data.py \
```

Resulting training and evaluation datasets in `tf_record` format will be later
passed to [train.py](train.py).
passed to [train.py](../train.py).

Then you can execute the following commands to start the training and evaluation
job.
Expand Down
Loading