enchanter.addons.layers¶

AutoEncoder¶

class enchanter.addons.layers.AutoEncoder(shapes: List[int], activation: Union[Callable[[torch.Tensor], torch.Tensor], torch.nn.modules.module.Module] = <built-in method relu of type object>)[source]¶

Bases: torch.nn.modules.module.Module

A class that creates an AutoEncoder.

Examples

>>> ae = AutoEncoder([32, 16, 8])

Initializes internal Module state, shared by both nn.Module and ScriptModule.

Methods

forward(x)

forward propagation processing.

forward(x: torch.Tensor) → torch.Tensor[source]¶

forward propagation processing.

Parameters: x – input data

Returns:

TemporalConvBlock¶

class enchanter.addons.layers.TemporalConvBlock(in_features: int, out_features: int, kernel_size: int, stride: int = 1, dilation: int = 1, dropout: float = 0.5, activation: torch.nn.modules.module.Module = ReLU(), final_activation: bool = False)[source]¶

Bases: torch.nn.modules.module.Module

Temporal Convolutional Block

Paper: `An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling: <https://arxiv.org/abs/1803.01271>`_

Parameters

in_features – the number of input channels
out_features – the number of output channels
kernel_size – kernel size
stride – stride
dilation – dilation rate
dropout – dropout rate
activation – activation function (default: ReLU)
final_activation – If true, apply the activation function after the residual connection

Methods

forward(x)

CausalConv1d¶

class enchanter.addons.layers.CausalConv1d(in_channels: int, out_channels: int, kernel_size: int, stride: int = 1, dilation: int = 1, groups: int = 1, bias: bool = True)[source]¶

Bases: torch.nn.modules.conv.Conv1d

Causal Conv1d

Paper: WaveNet: A Generative Model for Raw Audio

Parameters

in_channels – the number of input channels
out_channels – the number of output channels
kernel_size – kernel size
stride – stride
dilation – rate of dilation
groups – the number of groups
bias – if true use bias (default: True)

Methods

forward(x)

Forward propagation

forward(x: torch.Tensor) → torch.Tensor[source]¶

Forward propagation

Parameters: x – [N, in_channels, L]
Returns: [N, out_channels, L]

DenseInterpolation¶

class enchanter.addons.layers.DenseInterpolation(seq_len: int, factor: int)[source]¶

Bases: torch.nn.modules.module.Module

Parameters

seq_len – length of input sequence.
factor –

Initializes internal Module state, shared by both nn.Module and ScriptModule.

Methods

forward(x)

Apply Dense Interpolation to the input.

forward(x: torch.Tensor) → torch.Tensor[source]¶

Apply Dense Interpolation to the input.

Parameters: x (torch.Tensor) – The shape of the input array is assumed to be [N, seq_len, features].
Returns: (torch.Tensor)

MLP¶

class enchanter.addons.layers.MLP(shapes: List[int], activation: Union[Callable[[torch.Tensor], torch.Tensor], torch.nn.modules.module.Module] = <built-in method relu of type object>)[source]¶

Bases: torch.nn.modules.module.Module

Class to create MLP.

Parameters

shapes (List[int]) – The number of neurons in each layer of MLP. Assuming an array consisting of int type elements. The value of the 0th element of the given array is treated as the number of input dimensions to the model.
activation (Union[Callable[[torch.Tensor], torch.Tensor], nn.Module]) – Activation Function. Differentiable Callable objects such as torch.relu and enchanter.addons.Mish()

Examples

>>> import enchanter.addons as addons
>>> model = addons.layers.MLP([10, 512, 128, 5], addons.Mish())
>>> print(model)
>>> # ModuleList(
>>> #    (0): Linear(in_features=10, out_features=512, bias=True)
>>> #    (1): Linear(in_features=512, out_features=128, bias=True)
>>> #    (2): Linear(in_features=128, out_features=5, bias=True)
>>> #)

Initializes internal Module state, shared by both nn.Module and ScriptModule.

Methods

forward(x)

forward propagation processing.

forward(x: torch.Tensor) → torch.Tensor[source]¶

forward propagation processing.

Parameters: x – input data

Returns:

PositionalEncoding¶

class enchanter.addons.layers.PositionalEncoding(d_model: int, seq_len: int, dropout: float = 0.1)[source]¶

Bases: torch.nn.modules.module.Module

Positional Encoding used in the Transformer model proposed in Attention is all you need.

References

Sequence-to-Sequence Modeling with nn.Transformer and TorchText

Parameters

d_model – the number of expected features in the encoder/decoder inputs.
seq_len – length of input sequence.
dropout – dropout rate.

Methods

forward(x)

Forward propagation

forward(x: torch.Tensor) → torch.Tensor[source]¶

Forward propagation

Parameters: x – input data [N, F, L]
Returns: (N, E, L)

PositionWiseFeedForward¶

class enchanter.addons.layers.PositionWiseFeedForward(d_model: int, expansion: int = 2)[source]¶

Bases: torch.nn.modules.module.Module

PositionWise FeedForward proposed in Attention is all you need. This class uses 1x1 Conv1d internally.

Parameters

d_model – the number of expected features in the Position Wise Feed Forward inputs.
expansion – Magnification rate of the number of hidden dimensions. Default: 2

Examples

>>> import torch
>>> import enchanter.addons as addons
>>> x = torch.randn(1, 128, 512)    # [N, seq_len, features]
>>> ff = addons.layers.PositionWiseFeedForward(512)
>>> out = ff(x)

Initializes internal Module state, shared by both nn.Module and ScriptModule.

Methods

forward(x)

Apply PositionWiseFeedForward to the input.

forward(x: torch.Tensor) → torch.Tensor[source]¶

Apply PositionWiseFeedForward to the input.

Parameters: x (torch.Tensor) –

Returns:

ResidualSequential¶

class enchanter.addons.layers.ResidualSequential(*args: Any)[source]¶

Bases: torch.nn.modules.container.Sequential

Examples

>>> model = ResidualSequential(
>>>     Linear(10, 10),
>>>     ReLU(),
>>>     Linear(10, 10)
>>> )

Initializes internal Module state, shared by both nn.Module and ScriptModule.

Methods

forward(x)

Forward propagation

forward(x: torch.Tensor) → torch.Tensor[source]¶

Forward propagation

Parameters: x – input data

Returns:

SELayer1d¶

class enchanter.addons.layers.SELayer1d(in_features: int, reduction: int = 16)[source]¶

Bases: torch.nn.modules.module.Module

Squeeze and Excitation block for time series data.

Paper: Squeeze-and-Excitation Networks

Initializes internal Module state, shared by both nn.Module and ScriptModule.

Methods

forward(x)

Forward pass

forward(x: torch.Tensor) → torch.Tensor[source]¶

Forward pass

Parameters: x – input data [N, C, L]

Returns:

SELayer2d¶

class enchanter.addons.layers.SELayer2d(in_features: int, reduction: int = 16)[source]¶

Bases: torch.nn.modules.module.Module

Squeeze and Excitation block for image data.

Paper: Squeeze-and-Excitation Networks

Initializes internal Module state, shared by both nn.Module and ScriptModule.

Methods

forward(x)

Forward pass

forward(x: torch.Tensor) → torch.Tensor[source]¶

Forward pass

Parameters: x – input data [N, C, H, W]

Returns: