Dataset

deepke.relation_extraction.few_shot.dataset.base_data_module module

Base DataModule class.

class deepke.relation_extraction.few_shot.dataset.base_data_module.BaseDataModule(args)

Bases: torch.nn.modules.module.Module

Base DataModule.

static add_to_argparse(parser)

get_data_config()

Return important settings of the dataset, which will be passed to instantiate models.

prepare_data()

Use this method to do things that might write to disk or that need to be done only from a single GPU in distributed settings (so don’t set state self.x = y).

setup(stage=None)

Split into train, val, test, and set dims. Should assign torch Dataset objects to self.data_train, self.data_val, and optionally self.data_test.

train_dataloader()

val_dataloader()

test_dataloader()

training: bool

deepke.relation_extraction.few_shot.dataset.dialogue module

class deepke.relation_extraction.few_shot.dataset.dialogue.REDataset(args)

Bases: deepke.relation_extraction.few_shot.dataset.base_data_module.BaseDataModule

setup(stage=None)

Split into train, val, test, and set dims. Should assign torch Dataset objects to self.data_train, self.data_val, and optionally self.data_test.

prepare_data()

Use this method to do things that might write to disk or that need to be done only from a single GPU in distributed settings (so don’t set state self.x = y).

get_tokenizer()

training: bool

deepke.relation_extraction.few_shot.dataset.processor module

deepke.relation_extraction.few_shot.dataset.processor.tokenize(text, tokenizer)

class deepke.relation_extraction.few_shot.dataset.processor.InputExample(guid, text_a, text_b=None, label=None, text_c=None, entity=None)

Bases: object

A single training/test example for simple sequence classification.

class deepke.relation_extraction.few_shot.dataset.processor.InputExampleSST2(guid, text_a, text_b=None, label=None, text_c=None, entity=None)

Bases: object

A single training/test example for simple sequence classification.

class deepke.relation_extraction.few_shot.dataset.processor.InputFeaturesSST2(input_ids, attention_mask, token_type_ids, label_id)

Bases: object

A single set of features of data.

class deepke.relation_extraction.few_shot.dataset.processor.InputExampleWiki80(guid, sentence, span1, span2, ner1, ner2, label)

Bases: object

A single training/test example for span pair classification.

class deepke.relation_extraction.few_shot.dataset.processor.InputFeatures(input_ids, input_mask, segment_ids, label_id, entity=None)

Bases: object

A single set of features of data.

class deepke.relation_extraction.few_shot.dataset.processor.DataProcessor

Bases: object

Base class for data converters for sequence classification data sets.

get_train_examples(data_dir)

Gets a collection of `InputExample`s for the train set.

get_dev_examples(data_dir)

Gets a collection of `InputExample`s for the dev set.

get_labels()

Gets the list of labels for this data set.

class deepke.relation_extraction.few_shot.dataset.processor.Sst2Processor(data_dir, a)

Bases: deepke.relation_extraction.few_shot.dataset.processor.DataProcessor

Processor for the SST-2 data set (GLUE version).

get_example_from_tensor_dict(tensor_dict)

See base class.

get_train_examples(data_dir)

See base class.

get_dev_examples(data_dir)

See base class.

get_test_examples(data_dir)

See base class.

get_labels()

See base class.

class deepke.relation_extraction.few_shot.dataset.processor.relossProcessor(data_path='data', use_prompt=False)

Bases: deepke.relation_extraction.few_shot.dataset.processor.DataProcessor

get_train_examples(data_dir)

See base class.

get_test_examples(data_dir)

See base class.

get_dev_examples(data_dir)

See base class.

get_labels()

See base class.

class deepke.relation_extraction.few_shot.dataset.processor.bertProcessor(data_path='data', use_prompt=False)

Bases: deepke.relation_extraction.few_shot.dataset.processor.DataProcessor

get_train_examples(data_dir)

See base class.

get_test_examples(data_dir)

See base class.

get_dev_examples(data_dir)

See base class.

get_labels()

See base class.

class deepke.relation_extraction.few_shot.dataset.processor.ptuneProcessor(data_path='data', use_prompt=False, ptune_k=6)

Bases: deepke.relation_extraction.few_shot.dataset.processor.DataProcessor

D

TODO, add new samples, every sample if there is a trigger then mask trigger and replace the origin mask with right token, if no trigger in the sentence, random mask a word in the sentence and replace the origin mask with the right token.

get_train_examples(data_dir)

See base class.

get_test_examples(data_dir)

See base class.

get_dev_examples(data_dir)

See base class.

get_labels()

See base class.

class deepke.relation_extraction.few_shot.dataset.processor.wiki80Processor(data_path, use_prompt)

Bases: deepke.relation_extraction.few_shot.dataset.processor.DataProcessor

Processor for the TACRED data set.

get_train_examples(data_dir)

See base class.

get_dev_examples(data_dir)

See base class.

get_test_examples(data_dir)

See base class.

get_labels(negative_label='no_relation')

Gets the list of labels for this data set.

deepke.relation_extraction.few_shot.dataset.processor.convert_examples_to_features_for_loss(examples, max_seq_length, tokenizer)

deepke.relation_extraction.few_shot.dataset.processor.convert_examples_to_features_normal(examples, max_seq_length, tokenizer)

deepke.relation_extraction.few_shot.dataset.processor.convert_examples_to_features(examples, max_seq_length, tokenizer, args, rel2id)

Loads a data file into a list of `InputBatch`s.

deepke.relation_extraction.few_shot.dataset.processor.convert_examples_to_feature_sst2(examples, max_seq_length, tokenizer, args, rel2id)

Loads a data file into a list of `InputBatch`s.

deepke.relation_extraction.few_shot.dataset.processor.get_dataset(mode, args, tokenizer, processor)

deepke.relation_extraction.few_shot.dataset.processor.collate_fn(batch)