enchanter.addons.layers

AutoEncoder

class enchanter.addons.layers.AutoEncoder(shapes: List[int], activation: Union[Callable[[torch.Tensor], torch.Tensor], torch.nn.modules.module.Module] = <built-in method relu of type object>)[source]

Bases: torch.nn.modules.module.Module

A class that creates an AutoEncoder.

Examples

>>> ae = AutoEncoder([32, 16, 8])

Initializes internal Module state, shared by both nn.Module and ScriptModule.

Methods

forward(x)

forward propagation processing.

forward(x: torch.Tensor) → torch.Tensor[source]

forward propagation processing.

Parameters

x – input data

Returns:

TemporalConvBlock

class enchanter.addons.layers.TemporalConvBlock(in_features: int, out_features: int, kernel_size: int, stride: int = 1, dilation: int = 1, dropout: float = 0.5, activation: torch.nn.modules.module.Module = ReLU(), final_activation: bool = False)[source]

Bases: torch.nn.modules.module.Module

Temporal Convolutional Block

Paper: `An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling

<https://arxiv.org/abs/1803.01271>`_

Parameters

  • in_features – the number of input channels

  • out_features – the number of output channels

  • kernel_size – kernel size

  • stride – stride

  • dilation – dilation rate

  • dropout – dropout rate

  • activation – activation function (default: ReLU)

  • final_activation – If true, apply the activation function after the residual connection

Methods

forward(x)

Defines the computation performed at every call.

forward(x: torch.Tensor) → torch.Tensor[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

CausalConv1d

class enchanter.addons.layers.CausalConv1d(in_channels: int, out_channels: int, kernel_size: int, stride: int = 1, dilation: int = 1, groups: int = 1, bias: bool = True)[source]

Bases: torch.nn.modules.conv.Conv1d

Causal Conv1d

Paper: WaveNet: A Generative Model for Raw Audio

Parameters

  • in_channels – the number of input channels

  • out_channels – the number of output channels

  • kernel_size – kernel size

  • stride – stride

  • dilation – rate of dilation

  • groups – the number of groups

  • bias – if true use bias (default: True)

Methods

forward(x)

Forward propagation

forward(x: torch.Tensor) → torch.Tensor[source]

Forward propagation

Parameters

x – [N, in_channels, L]

Returns

[N, out_channels, L]

DenseInterpolation

class enchanter.addons.layers.DenseInterpolation(seq_len: int, factor: int)[source]

Bases: torch.nn.modules.module.Module

Parameters

  • seq_len – length of input sequence.

  • factor –

Initializes internal Module state, shared by both nn.Module and ScriptModule.

Methods

forward(x)

Apply Dense Interpolation to the input.

forward(x: torch.Tensor) → torch.Tensor[source]

Apply Dense Interpolation to the input.

Parameters

x (torch.Tensor) – The shape of the input array is assumed to be [N, seq_len, features].

Returns

(torch.Tensor)

MLP

class enchanter.addons.layers.MLP(shapes: List[int], activation: Union[Callable[[torch.Tensor], torch.Tensor], torch.nn.modules.module.Module] = <built-in method relu of type object>)[source]

Bases: torch.nn.modules.module.Module

Class to create MLP.

Parameters

  • shapes (List[int]) – The number of neurons in each layer of MLP. Assuming an array consisting of int type elements. The value of the 0th element of the given array is treated as the number of input dimensions to the model.

  • activation (Union[Callable[[torch.Tensor], torch.Tensor], nn.Module]) – Activation Function. Differentiable Callable objects such as torch.relu and enchanter.addons.Mish()

Examples

>>> import enchanter.addons as addons
>>> model = addons.layers.MLP([10, 512, 128, 5], addons.Mish())
>>> print(model)
>>> # ModuleList(
>>> #    (0): Linear(in_features=10, out_features=512, bias=True)
>>> #    (1): Linear(in_features=512, out_features=128, bias=True)
>>> #    (2): Linear(in_features=128, out_features=5, bias=True)
>>> #)

Initializes internal Module state, shared by both nn.Module and ScriptModule.

Methods

forward(x)

forward propagation processing.

forward(x: torch.Tensor) → torch.Tensor[source]

forward propagation processing.

Parameters

x – input data

Returns:

PositionalEncoding

class enchanter.addons.layers.PositionalEncoding(d_model: int, seq_len: int, dropout: float = 0.1)[source]

Bases: torch.nn.modules.module.Module

Positional Encoding used in the Transformer model proposed in Attention is all you need.

References

Sequence-to-Sequence Modeling with nn.Transformer and TorchText

Parameters

  • d_model – the number of expected features in the encoder/decoder inputs.

  • seq_len – length of input sequence.

  • dropout – dropout rate.

Methods

forward(x)

Forward propagation

forward(x: torch.Tensor) → torch.Tensor[source]

Forward propagation

Parameters

x – input data [N, F, L]

Returns

(N, E, L)

PositionWiseFeedForward

class enchanter.addons.layers.PositionWiseFeedForward(d_model: int, expansion: int = 2)[source]

Bases: torch.nn.modules.module.Module

PositionWise FeedForward proposed in Attention is all you need. This class uses 1x1 Conv1d internally.

Parameters

  • d_model – the number of expected features in the Position Wise Feed Forward inputs.

  • expansion – Magnification rate of the number of hidden dimensions. Default: 2

Examples

>>> import torch
>>> import enchanter.addons as addons
>>> x = torch.randn(1, 128, 512)    # [N, seq_len, features]
>>> ff = addons.layers.PositionWiseFeedForward(512)
>>> out = ff(x)

Initializes internal Module state, shared by both nn.Module and ScriptModule.

Methods

forward(x)

Apply PositionWiseFeedForward to the input.

forward(x: torch.Tensor) → torch.Tensor[source]

Apply PositionWiseFeedForward to the input.

Parameters

x (torch.Tensor) –

Returns:

ResidualSequential

class enchanter.addons.layers.ResidualSequential(*args: torch.nn.modules.module.Module)[source]
class enchanter.addons.layers.ResidualSequential(arg: OrderedDict[str, Module])

Bases: torch.nn.modules.container.Sequential

Examples

>>> model = ResidualSequential(
>>>     Linear(10, 10),
>>>     ReLU(),
>>>     Linear(10, 10)
>>> )

Initializes internal Module state, shared by both nn.Module and ScriptModule.

Methods

forward(x)

Forward propagation

forward(x: torch.Tensor) → torch.Tensor[source]

Forward propagation

Parameters

x – input data

Returns:

SELayer1d

class enchanter.addons.layers.SELayer1d(in_features: int, reduction: int = 16)[source]

Bases: torch.nn.modules.module.Module

Squeeze and Excitation block for time series data.

Paper: Squeeze-and-Excitation Networks

Initializes internal Module state, shared by both nn.Module and ScriptModule.

Methods

forward(x)

Forward pass

forward(x: torch.Tensor) → torch.Tensor[source]

Forward pass

Parameters

x – input data [N, C, L]

Returns:

SELayer2d

class enchanter.addons.layers.SELayer2d(in_features: int, reduction: int = 16)[source]

Bases: torch.nn.modules.module.Module

Squeeze and Excitation block for image data.

Paper: Squeeze-and-Excitation Networks

Initializes internal Module state, shared by both nn.Module and ScriptModule.

Methods

forward(x)

Forward pass

forward(x: torch.Tensor) → torch.Tensor[source]

Forward pass

Parameters

x – input data [N, C, H, W]

Returns: