kiwi.utils.tensors
¶
Module Contents¶
Classes¶
Records operation history and defines formulas for differentiating ops. 
Functions¶



Convolve a sequence and apply padding. 

Run a forward pass of padded_sequences through an rnn using packed sequence. 

Replace old tokens with new. 

Create a loss weight vector for nn.CrossEntropyLoss. 

Create a boolean mask from sequence lengths. 

Unmask a tensor and convert it back to a list of lists. 

Expand new dimensions based on a template tensor along dim axis. 

Expand a new dimension in t1 and t2 and expand them so that both 

Compute Mask of Tokens for side. 



Join together pieces of a token back into the original token dimension. 

kiwi.utils.tensors.
pad_zeros_around_timesteps
(batched_tensor: torch.Tensor) → torch.Tensor¶

kiwi.utils.tensors.
convolve_tensor
(sequences, window_size, pad_value=0)¶ Convolve a sequence and apply padding.
 Parameters
sequences – nD tensor
window_size – filter length
pad_value – int value used as padding
 Returns
(n+1)D tensor, where the last dimension has size window_size.

kiwi.utils.tensors.
apply_packed_sequence
(rnn, padded_sequences, lengths)¶ Run a forward pass of padded_sequences through an rnn using packed sequence.
 Parameters
rnn – The RNN that that we want to compute a forward pass with.
padded_sequences (FloatTensor b x seq x dim) – A batch of padded_sequences.
lengths (LongTensor batch) – The length of each sequence in the batch.
 Returns
the output of the RNN rnn with input padded_sequences
 Return type
output

kiwi.utils.tensors.
replace_token
(target: torch.LongTensor, old: int, new: int)¶ Replace old tokens with new.
 Parameters
target –
old – the token to be replaced by new.
new – the token used to replace old.

kiwi.utils.tensors.
make_classes_loss_weights
(vocab: Vocabulary, label_weights: Dict[str, float])¶ Create a loss weight vector for nn.CrossEntropyLoss.
 Parameters
vocab – vocabulary for classes.
label_weights – weight for specific classes (str); classes in vocab and not in this dict will get a weight of 1.
 Returns
weight Tensor of shape nb_classes.
 Return type
weights (FloatTensor)

kiwi.utils.tensors.
sequence_mask
(lengths: torch.LongTensor, max_len: Optional[int] = None)¶ Create a boolean mask from sequence lengths.
 Parameters
lengths – lengths with shape (bs,)
max_len – max sequence length; if None it will be set to lengths.max()

kiwi.utils.tensors.
unmask
(tensor, mask)¶ Unmask a tensor and convert it back to a list of lists.

kiwi.utils.tensors.
unsqueeze_as
(tensor, as_tensor, dim= 1)¶ Expand new dimensions based on a template tensor along dim axis.

kiwi.utils.tensors.
make_mergeable_tensors
(t1: torch.Tensor, t2: torch.Tensor)¶ Expand a new dimension in t1 and t2 and expand them so that both tensors will have the same number of timesteps.
 Parameters
t1 – tensor with shape (bs, …, m, d1)
t2 – tensor with shape (bs, …, n, d2)
 Returns
 tuple of
torch.Tensor: (bs, …, m, n, d1), torch.Tensor: (bs, …, m, n, d2)

class
kiwi.utils.tensors.
GradientMul
¶ Bases:
torch.autograd.Function
Records operation history and defines formulas for differentiating ops.
See the Note on extending the autograd engine for more details on how to use this class: https://pytorch.org/docs/stable/notes/extending.html#extendingtorchautograd
Every operation performed on
Tensor
s creates a new function object, that performs the computation, and records that it happened. The history is retained in the form of a DAG of functions, with edges denoting data dependencies (input < output
). Then, when backward is called, the graph is processed in the topological ordering, by callingbackward()
methods of eachFunction
object, and passing returned gradients on to nextFunction
s.Normally, the only way users interact with functions is by creating subclasses and defining new operations. This is a recommended way of extending torch.autograd.
Examples:
>>> class Exp(Function): >>> >>> @staticmethod >>> def forward(ctx, i): >>> result = i.exp() >>> ctx.save_for_backward(result) >>> return result >>> >>> @staticmethod >>> def backward(ctx, grad_output): >>> result, = ctx.saved_tensors >>> return grad_output * result >>> >>> #Use it by calling the apply method: >>> output = Exp.apply(input)

static
forward
(ctx, x, constant=0)¶ Performs the operation.
This function is to be overridden by all subclasses.
It must accept a context ctx as the first argument, followed by any number of arguments (tensors or other types).
The context can be used to store tensors that can be then retrieved during the backward pass.

static
backward
(ctx, grad)¶ Defines a formula for differentiating the operation.
This function is to be overridden by all subclasses.
It must accept a context
ctx
as the first argument, followed by as many outputs didforward()
return, and it should return as many tensors, as there were inputs toforward()
. Each argument is the gradient w.r.t the given output, and each returned value should be the gradient w.r.t. the corresponding input.The context can be used to retrieve tensors saved during the forward pass. It also has an attribute
ctx.needs_input_grad
as a tuple of booleans representing whether each input needs gradient. E.g.,backward()
will havectx.needs_input_grad[0] = True
if the first input toforward()
needs gradient computated w.r.t. the output.

static

kiwi.utils.tensors.
gradient_mul
¶

kiwi.utils.tensors.
retrieve_tokens_mask
(input_batch: BatchedSentence)¶ Compute Mask of Tokens for side.
Migrated from FieldEmbedder.get_mask()
 Parameters
input_batch (BatchedSentence) – batch of tensors
 Returns
mask tensor

kiwi.utils.tensors.
select_positions
(tensor, indices)¶

kiwi.utils.tensors.
pieces_to_tokens
(features_tensor, batch, strategy='first')¶ Join together pieces of a token back into the original token dimension.