Module

deepke.attribution_extraction.standard.module.Attention module

class deepke.attribution_extraction.standard.module.Attention.DotAttention(dropout=0.0)

Bases: torch.nn.modules.module.Module

forward(Q, K, V, mask_out=None, head_mask=None)

一般输入信息 X 时，假设 K = V = X

att_weight = softmax( score_func(q, k) ) att = sum( att_weight * v )

Parameters

• Q – […, L, H]

• K – […, S, H]

• V – […, S, H]

• mask_out – […, 1, S]

Returns

training: bool

class deepke.attribution_extraction.standard.module.Attention.MultiHeadAttention(embed_dim, num_heads, dropout=0.0, output_attentions=True)

Bases: torch.nn.modules.module.Module

forward(Q, K, V, key_padding_mask=None, attention_mask=None, head_mask=None)

Parameters

• Q – [B, L, Hs]

• K – [B, S, Hs]

• V – [B, S, Hs]

• key_padding_mask – [B, S] 为 1/True 的地方需要 mask

• attention_mask – [S] / [L, S] 指定位置 mask 掉， 为 1/True 的地方需要 mask

• head_mask – [N] 指定 head mask 掉， 为 1/True 的地方需要 mask

training: bool

deepke.attribution_extraction.standard.module.CNN module

class deepke.attribution_extraction.standard.module.CNN.GELU

Bases: torch.nn.modules.module.Module

forward(x)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class deepke.attribution_extraction.standard.module.CNN.CNN(config)

Bases: torch.nn.modules.module.Module

nlp 里为了保证输出的句长 = 输入的句长，一般使用奇数 kernel_size，如 [3, 5, 7, 9] 当然也可以不等长输出，keep_length 设为 False 此时，padding = k // 2 stride 一般为 1

forward(x, mask=None)

Parameters

• x – torch.Tensor [batch_size, seq_max_length, input_size], [B, L, H] 一般是经过embedding后的值

• mask – [batch_size, max_len], 句长部分为0，padding部分为1。不影响卷积运算，max-pool一定不会pool到pad为0的位置

Returns

training: bool

deepke.attribution_extraction.standard.module.Capsule module

class deepke.attribution_extraction.standard.module.Capsule.Capsule(cfg)

Bases: torch.nn.modules.module.Module

forward(x)

x: [B, L, H] # 从 CNN / RNN 得到的结果

L 作为 input_num_capsules, H 作为 input_dim_capsule

static squash(x: torch.Tensor)

training: bool

deepke.attribution_extraction.standard.module.Embedding module

class deepke.attribution_extraction.standard.module.Embedding.Embedding(config)

Bases: torch.nn.modules.module.Module

forward(*x)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

deepke.attribution_extraction.standard.module.GCN module

class deepke.attribution_extraction.standard.module.GCN.GCN(cfg)

Bases: torch.nn.modules.module.Module

forward(x, adj)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class deepke.attribution_extraction.standard.module.GCN.Tree

Bases: object

add_child(child)

size()

depth()

deepke.attribution_extraction.standard.module.GCN.head_to_adj(head, directed=True, self_loop=False)

Convert a sequence of head indexes to an (numpy) adjacency matrix.

deepke.attribution_extraction.standard.module.GCN.pad_adj(adj, max_len)

deepke.attribution_extraction.standard.module.RNN module

class deepke.attribution_extraction.standard.module.RNN.RNN(config)

Bases: torch.nn.modules.module.Module

forward(x, x_len)

Parameters

• [batch_size (torch.Tensor) –

• seq_max_length –

• input_size] –

• [B –

• L –

• 一般是经过embedding后的值 (H_in]) –

• x_len – torch.Tensor [L] 已经排好序的句长值

Returns

torch.Tensor [B, L, H_out] 序列标注的使用结果 hn: torch.Tensor [B, N, H_out] / [B, H_out] 分类的结果，当 last_layer_hn 时只有最后一层结果

Return type

output

training: bool

deepke.attribution_extraction.standard.module.Transformer module

deepke.attribution_extraction.standard.module.Transformer.gelu(x)

Original Implementation of the gelu activation function in Google Bert repo when initially created. For information: OpenAI GPT’s gelu is slightly different (and gives slightly different results): 0.5 * x * (1 + torch.tanh(math.sqrt(2 / math.pi) * (x + 0.044715 * torch.pow(x, 3)))) Also see https://arxiv.org/abs/1606.08415

deepke.attribution_extraction.standard.module.Transformer.gelu_new(x)

Implementation of the gelu activation function currently in Google Bert repo (identical to OpenAI GPT). Also see https://arxiv.org/abs/1606.08415

deepke.attribution_extraction.standard.module.Transformer.swish(x)

class deepke.attribution_extraction.standard.module.Transformer.TransformerAttention(config)

Bases: torch.nn.modules.module.Module

forward(x, key_padding_mask=None, attention_mask=None, head_mask=None)

Parameters

• x – [B, L, Hs]

• attention_mask – [B, L] padding后的句子后面补0了，补0的位置为True，前面部分为False

• head_mask – [L] [N,L]

Returns

training: bool

class deepke.attribution_extraction.standard.module.Transformer.TransformerOutput(config)

Bases: torch.nn.modules.module.Module

forward(input_tensor)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class deepke.attribution_extraction.standard.module.Transformer.TransformerLayer(config)

Bases: torch.nn.modules.module.Module

forward(hidden_states, key_padding_mask=None, attention_mask=None, head_mask=None)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class deepke.attribution_extraction.standard.module.Transformer.Transformer(config)

Bases: torch.nn.modules.module.Module

training: bool

forward(hidden_states, key_padding_mask=None, attention_mask=None, head_mask=None)

Parameters

• hidden_states – [B, L, Hs]

• key_padding_mask – [B, S] 为 1/True 的地方需要 mask

• attn_mask – [S] / [L, S] 指定位置 mask 掉， 为 1/True 的地方需要 mask

• head_mask – [N] / [L, N] 指定 head mask 掉， 为 1/True 的地方需要 mask