kiwi.utils.tensors
GradientMul
Records operation history and defines formulas for differentiating ops.
pad_zeros_around_timesteps(batched_tensor: torch.Tensor) → torch.Tensor
pad_zeros_around_timesteps
convolve_tensor(sequences, window_size, pad_value=0)
convolve_tensor
Convolve a sequence and apply padding.
apply_packed_sequence(rnn, padded_sequences, lengths)
apply_packed_sequence
Run a forward pass of padded_sequences through an rnn using packed sequence.
replace_token(target: torch.LongTensor, old: int, new: int)
replace_token
Replace old tokens with new.
make_classes_loss_weights(vocab: Vocabulary, label_weights: Dict[str, float])
make_classes_loss_weights
Create a loss weight vector for nn.CrossEntropyLoss.
sequence_mask(lengths: torch.LongTensor, max_len: Optional[int] = None)
sequence_mask
Create a boolean mask from sequence lengths.
unmask(tensor, mask)
unmask
Unmask a tensor and convert it back to a list of lists.
unsqueeze_as(tensor, as_tensor, dim=-1)
unsqueeze_as
Expand new dimensions based on a template tensor along dim axis.
make_mergeable_tensors(t1: torch.Tensor, t2: torch.Tensor)
make_mergeable_tensors
Expand a new dimension in t1 and t2 and expand them so that both
retrieve_tokens_mask(input_batch: BatchedSentence)
retrieve_tokens_mask
Compute Mask of Tokens for side.
select_positions(tensor, indices)
select_positions
pieces_to_tokens(features_tensor, batch, strategy='first')
pieces_to_tokens
Join together pieces of a token back into the original token dimension.
kiwi.utils.tensors.
sequences – nD tensor
window_size – filter length
pad_value – int value used as padding
(n+1)D tensor, where the last dimension has size window_size.
rnn – The RNN that that we want to compute a forward pass with.
padded_sequences (FloatTensor b x seq x dim) – A batch of padded_sequences.
lengths (LongTensor batch) – The length of each sequence in the batch.
the output of the RNN rnn with input padded_sequences
output
target –
old – the token to be replaced by new.
new – the token used to replace old.
vocab – vocabulary for classes.
label_weights – weight for specific classes (str); classes in vocab and not in this dict will get a weight of 1.
weight Tensor of shape nb_classes.
weights (FloatTensor)
lengths – lengths with shape (bs,)
max_len – max sequence length; if None it will be set to lengths.max()
Expand a new dimension in t1 and t2 and expand them so that both tensors will have the same number of timesteps.
t1 – tensor with shape (bs, …, m, d1)
t2 – tensor with shape (bs, …, n, d2)
torch.Tensor: (bs, …, m, n, d1), torch.Tensor: (bs, …, m, n, d2)
Bases: torch.autograd.Function
torch.autograd.Function
See the Note on extending the autograd engine for more details on how to use this class: https://pytorch.org/docs/stable/notes/extending.html#extending-torch-autograd
Every operation performed on Tensor s creates a new function object, that performs the computation, and records that it happened. The history is retained in the form of a DAG of functions, with edges denoting data dependencies (input <- output). Then, when backward is called, the graph is processed in the topological ordering, by calling backward() methods of each Function object, and passing returned gradients on to next Function s.
Tensor
input <- output
backward()
Function
Normally, the only way users interact with functions is by creating subclasses and defining new operations. This is a recommended way of extending torch.autograd.
Examples:
>>> class Exp(Function): >>> >>> @staticmethod >>> def forward(ctx, i): >>> result = i.exp() >>> ctx.save_for_backward(result) >>> return result >>> >>> @staticmethod >>> def backward(ctx, grad_output): >>> result, = ctx.saved_tensors >>> return grad_output * result >>> >>> #Use it by calling the apply method: >>> output = Exp.apply(input)
forward
Performs the operation.
This function is to be overridden by all subclasses.
It must accept a context ctx as the first argument, followed by any number of arguments (tensors or other types).
The context can be used to store tensors that can be then retrieved during the backward pass.
backward
Defines a formula for differentiating the operation.
It must accept a context ctx as the first argument, followed by as many outputs did forward() return, and it should return as many tensors, as there were inputs to forward(). Each argument is the gradient w.r.t the given output, and each returned value should be the gradient w.r.t. the corresponding input.
ctx
forward()
The context can be used to retrieve tensors saved during the forward pass. It also has an attribute ctx.needs_input_grad as a tuple of booleans representing whether each input needs gradient. E.g., backward() will have ctx.needs_input_grad[0] = True if the first input to forward() needs gradient computated w.r.t. the output.
ctx.needs_input_grad
ctx.needs_input_grad[0] = True
gradient_mul
Migrated from FieldEmbedder.get_mask()
input_batch (BatchedSentence) – batch of tensors
mask tensor
kiwi.utils.io
kiwi.__main__