Systems

There are two types of systems: QE and TLM. The first is the regular one used for Quality Estimation. The second, which stands for Translation Language Model, is used for pre-training the encoder component of a QE system.

All systems, regardless of them being QE or TLM systems are constructed on top of PytorchLightning’s (PTL) LightningModule class. This means they respect a certain design philosophy that can be consulted in PTL’s documentation.

Furthermore, all of our QE systems share a similar architecture. They are composed of

  • Encoder

  • Decoder

  • Output

  • (Optionally) TLM Output

TLM systems on the other hand are only composed of the Encoder + TLM Output and their goal is to pre-train the encoder so it can be plugged into a QE system.

The systems are divided into the 3 blocks that have the major responsabilities in both word and sentence level QE tasks:

Encoder: Embedding and creating features to be used for downstream tasks. i.e. Predictor, BERT, etc

Decoder: Responsible for learning feature transformations better suited for the downstream task. i.e. MLP, LSTM, etc

Output: Simple feedforwards that take decoder features and transform them into the prediction required by the downstream task. Something in the same line as the common “classification heads” being used with transformers.

TLM Output: A simple output layer that trains for the specific TLM objective. It can be useful to continue finetuning the predictor during training of the complete QE system.

TLM — kiwi.systems.tlm_system

All TLM systems inherit from kiwi.systems.tlm_system.TLMSystem.

Use kiwi pretrain to train these systems. These systems can then be used as the encoder part in QE systems by using the load_encoder flag.

Currently available are:

kiwi.systems.predictor.Predictor