`kiwi.utils.io`¶

Module Contents¶

Base class for all pydantic configs. Used to configure base behaviour of configs.

`default_map_location`(storage, loc)
`load_torch_file`(file_path, map_location=None)
`save_file`(file_path, data, token_sep=' ', example_sep='\n')
`save_predicted_probabilities`(directory, predictions, prefix='')
`read_file`(path)	Read a file into a list of lists of words.
`target_gaps_to_target`(batch)	Extract target tags from wmt18 format file.
`target_gaps_to_gaps`(batch)	Extract gap tags from wmt18 format file.
`generate_slug`(text, delimiter='-')	Convert text to a normalized “slug” without whitespace.

class kiwi.utils.io.BaseConfig¶

Bases: pydantic.BaseModel

Base class for all pydantic configs. Used to configure base behaviour of configs.

class Config¶

kiwi.utils.io.save_file(file_path, data, token_sep=' ', example_sep='\n')¶

kiwi.utils.io.save_predicted_probabilities(directory, predictions, prefix='')¶

kiwi.utils.io.read_file(path)¶: Read a file into a list of lists of words.

kiwi.utils.io.target_gaps_to_target(batch)¶: Extract target tags from wmt18 format file.

kiwi.utils.io.target_gaps_to_gaps(batch)¶: Extract gap tags from wmt18 format file.

kiwi.utils.io.generate_slug(text, delimiter='-')¶

Convert text to a normalized “slug” without whitespace.

Borrowed from the nice https://humanfriendly.readthedocs.io, by Peter Odding.

Parameters

text – the original text, for example Some Random Text!.
delimiter – the delimiter to use for separating words (defaults to the - character).

Returns

the slug text, for example some-random-text.

Raises

ValueError – in an empty slug.

kiwi.utils.data_structures kiwi.utils.tensors