pine.qualitative_evaluation package

Submodules

pine.qualitative_evaluation.qualitative_evaluation module

class pine.qualitative_evaluation.qualitative_evaluation.ClusteredPositionalFeatures(language_model: pine.language_model.LanguageModel, data: Dict[str, List[int]])

Bases: object

The clusters of positional features in a log-bilinear language model.

Parameters

• language_model (LanguageModel) – A log-bilinear language model.

• data (dict of (str, list of int)) – Clusters of positional features in the log-bilinear language model.

Variables

• language_model (LanguageModel) – A log-bilinear language model.

• data (dict of (str, list of int)) – Clusters of positional features in the log-bilinear language model.

plot() → Figure

class pine.qualitative_evaluation.qualitative_evaluation.ExampleSentences(language_model: pine.language_model.LanguageModel, cluster_label: str, masked_word: str, first_sentence: Sequence[str], second_sentence: Sequence[str])

Bases: object

Two example sentences that characterize a cluster of positional features in a log-bilinear language model.

A context word from a cluster of positional features will be placed on two different positions of a sentence, where it produces the greatest difference in masked word predictions. This is a useful illustration of the behavior and the purpose of a cluster of positional features.

Parameters

• language_model (LanguageModel) – A log-bilinear language model.

• cluster_label (str) – A label of a cluster of positional features.

• masked_word (str) – A masked word.

• first_sentence (Sentence) – A sentence.

• second_sentence (Sentence) – Another sentence.

Variables

• language_model (LanguageModel) – A log-bilinear language model.

• cluster_label (str) – A label of a cluster of positional features.

• masked_word (str) – A masked word.

• first_sentence (Sentence) – A sentence.

• second_sentence (Sentence) – Another sentence.

class pine.qualitative_evaluation.qualitative_evaluation.PositionImportance(language_model: pine.language_model.LanguageModel, data: numpy.ndarray)

Bases: object

The importance of positions in a log-bilinear language model.

Parameters

• language_model (LanguageModel) – The log-bilinear language model.

• data (np.ndarray) – The importance of positions.

Variables

• language_model (LanguageModel) – The log-bilinear language model.

• data (np.ndarray) – The importance of positions.

plot() → Figure

class pine.qualitative_evaluation.qualitative_evaluation.SentenceProbability(sentence: Sequence[str], masked_word: str, score: float)

Bases: object

The probability of a sentence given a masked word.

Parameters

• sentence (Sentence) – A sentence.

• masked_word (str) – A masked word.

• score (float) – The probability of the sentence given the masked word.

Variables

• sentence (Sentence) – A sentence.

• masked_word (str) – A masked word.

• score (float) – The probability of the sentence given the masked word.

pine.qualitative_evaluation.qualitative_evaluation.classify_words(language_model: pine.language_model.LanguageModel, kind: str) → Dict[str, str]

pine.qualitative_evaluation.qualitative_evaluation.cluster_positional_features(language_model: pine.language_model.LanguageModel) → pine.qualitative_evaluation.qualitative_evaluation.ClusteredPositionalFeatures

pine.qualitative_evaluation.qualitative_evaluation.get_masked_word_probability(language_model: pine.language_model.LanguageModel, sentence: Sequence[str], masked_word: str, cluster_label: Optional[str] = None) → pine.qualitative_evaluation.qualitative_evaluation.SentenceProbability

pine.qualitative_evaluation.qualitative_evaluation.get_position_importance(language_model: pine.language_model.LanguageModel) → pine.qualitative_evaluation.qualitative_evaluation.PositionImportance

pine.qualitative_evaluation.qualitative_evaluation.predict_masked_words(language_model: pine.language_model.LanguageModel, sentence: Sequence[str], cluster_label: Optional[str] = None) → Iterable[str]

pine.qualitative_evaluation.qualitative_evaluation.produce_example_sentences(language_model: pine.language_model.LanguageModel, cluster_label: str) → pine.qualitative_evaluation.qualitative_evaluation.ExampleSentences

pine.qualitative_evaluation.util module

pine.qualitative_evaluation.util.index_to_position(language_model: pine.language_model.LanguageModel, index: int) → int

pine.qualitative_evaluation.util.position_to_index(language_model: pine.language_model.LanguageModel, position: int) → int

pine.qualitative_evaluation.view module

pine.qualitative_evaluation.view.gamma_forward(x)

pine.qualitative_evaluation.view.gamma_inverse(x)

pine.qualitative_evaluation.view.get_position_numbers(language_model: pine.language_model.LanguageModel) → numpy.ndarray

pine.qualitative_evaluation.view.plot_clustered_positional_features(language_model: pine.language_model.LanguageModel) → matplotlib.figure.Figure

pine.qualitative_evaluation.view.plot_position_importance(*language_models: pine.language_model.LanguageModel) → matplotlib.figure.Figure

Module contents

class pine.qualitative_evaluation.ClusteredPositionalFeatures(language_model: pine.language_model.LanguageModel, data: Dict[str, List[int]])

Bases: object

The clusters of positional features in a log-bilinear language model.

Parameters

• language_model (LanguageModel) – A log-bilinear language model.

• data (dict of (str, list of int)) – Clusters of positional features in the log-bilinear language model.

Variables

• language_model (LanguageModel) – A log-bilinear language model.

• data (dict of (str, list of int)) – Clusters of positional features in the log-bilinear language model.

plot() → Figure

class pine.qualitative_evaluation.ExampleSentences(language_model: pine.language_model.LanguageModel, cluster_label: str, masked_word: str, first_sentence: Sequence[str], second_sentence: Sequence[str])

Bases: object

Two example sentences that characterize a cluster of positional features in a log-bilinear language model.

A context word from a cluster of positional features will be placed on two different positions of a sentence, where it produces the greatest difference in masked word predictions. This is a useful illustration of the behavior and the purpose of a cluster of positional features.

Parameters

• language_model (LanguageModel) – A log-bilinear language model.

• cluster_label (str) – A label of a cluster of positional features.

• masked_word (str) – A masked word.

• first_sentence (Sentence) – A sentence.

• second_sentence (Sentence) – Another sentence.

Variables

• language_model (LanguageModel) – A log-bilinear language model.

• cluster_label (str) – A label of a cluster of positional features.

• masked_word (str) – A masked word.

• first_sentence (Sentence) – A sentence.

• second_sentence (Sentence) – Another sentence.

class pine.qualitative_evaluation.PositionImportance(language_model: pine.language_model.LanguageModel, data: numpy.ndarray)

Bases: object

The importance of positions in a log-bilinear language model.

Parameters

• language_model (LanguageModel) – The log-bilinear language model.

• data (np.ndarray) – The importance of positions.

Variables

• language_model (LanguageModel) – The log-bilinear language model.

• data (np.ndarray) – The importance of positions.

plot() → Figure

class pine.qualitative_evaluation.SentenceProbability(sentence: Sequence[str], masked_word: str, score: float)

Bases: object

The probability of a sentence given a masked word.

Parameters

• sentence (Sentence) – A sentence.

• masked_word (str) – A masked word.

• score (float) – The probability of the sentence given the masked word.

Variables

• sentence (Sentence) – A sentence.

• masked_word (str) – A masked word.

• score (float) – The probability of the sentence given the masked word.

pine.qualitative_evaluation.classify_words(language_model: pine.language_model.LanguageModel, kind: str) → Dict[str, str]

pine.qualitative_evaluation.cluster_positional_features(language_model: pine.language_model.LanguageModel) → pine.qualitative_evaluation.qualitative_evaluation.ClusteredPositionalFeatures

pine.qualitative_evaluation.get_masked_word_probability(language_model: pine.language_model.LanguageModel, sentence: Sequence[str], masked_word: str, cluster_label: Optional[str] = None) → pine.qualitative_evaluation.qualitative_evaluation.SentenceProbability

pine.qualitative_evaluation.get_position_importance(language_model: pine.language_model.LanguageModel) → pine.qualitative_evaluation.qualitative_evaluation.PositionImportance

pine.qualitative_evaluation.index_to_position(language_model: pine.language_model.LanguageModel, index: int) → int

pine.qualitative_evaluation.plot_clustered_positional_features(language_model: pine.language_model.LanguageModel) → matplotlib.figure.Figure

pine.qualitative_evaluation.plot_position_importance(*language_models: pine.language_model.LanguageModel) → matplotlib.figure.Figure

pine.qualitative_evaluation.position_to_index(language_model: pine.language_model.LanguageModel, position: int) → int

pine.qualitative_evaluation.predict_masked_words(language_model: pine.language_model.LanguageModel, sentence: Sequence[str], cluster_label: Optional[str] = None) → Iterable[str]

pine.qualitative_evaluation.produce_example_sentences(language_model: pine.language_model.LanguageModel, cluster_label: str) → pine.qualitative_evaluation.qualitative_evaluation.ExampleSentences