# カスタムレイヤーとユーティリティ

このページには、ライブラリで使用されるすべてのカスタム レイヤーと、モデリングに提供されるユーティリティ関数がリストされます。

これらのほとんどは、ライブラリ内のモデルのコードを研究する場合にのみ役に立ちます。

## Pytorch custom modules[[transformers.Conv1D]]

#### transformers.Conv1D[[transformers.Conv1D]]

```python
transformers.Conv1D(nf, nx)
```

[Source](https://github.com/huggingface/transformers/blob/main/src/transformers/pytorch_utils.py#L95)

**Parameters:**

nf (`int`) : The number of output features.

nx (`int`) : The number of input features.

1D-convolutional layer as defined by Radford et al. for OpenAI GPT (and also used in GPT-2).

Basically works like a linear layer but the weights are transposed.

## PyTorch Helper Functions[[transformers.apply_chunking_to_forward]]

#### transformers.apply_chunking_to_forward[[transformers.apply_chunking_to_forward]]

```python
transformers.apply_chunking_to_forward(forward_fn: Callable[..., torch.Tensor], chunk_size: int, chunk_dim: int, *input_tensors)
```

[Source](https://github.com/huggingface/transformers/blob/main/src/transformers/pytorch_utils.py#L124)

**Parameters:**

forward_fn (`Callable[..., torch.Tensor]`) : The forward function of the model.

chunk_size (`int`) : The chunk size of a chunked tensor: `num_chunks = len(input_tensors[0]) / chunk_size`.

chunk_dim (`int`) : The dimension over which the `input_tensors` should be chunked.

input_tensors (`tuple[torch.Tensor]`) : The input tensors of `forward_fn` which will be chunked

**Returns:** `torch.Tensor`

A tensor with the same shape as the `forward_fn` would have given if applied`.

This function chunks the `input_tensors` into smaller input tensor parts of size `chunk_size` over the dimension
`chunk_dim`. It then applies a layer `forward_fn` to each chunk independently to save memory.

If the `forward_fn` is independent across the `chunk_dim` this function will yield the same result as directly
applying `forward_fn` to `input_tensors`.

Examples:

```python
# rename the usual forward() fn to forward_chunk()
def forward_chunk(self, hidden_states):
    hidden_states = self.decoder(hidden_states)
    return hidden_states

# implement a chunked forward function
def forward(self, hidden_states):
    return apply_chunking_to_forward(self.forward_chunk, self.chunk_size_lm_head, self.seq_len_dim, hidden_states)
```

#### transformers.pytorch_utils.prune_linear_layer[[transformers.pytorch_utils.prune_linear_layer]]

```python
transformers.pytorch_utils.prune_linear_layer(layer: nn.Linear, index: torch.LongTensor, dim: int = 0)
```

[Source](https://github.com/huggingface/transformers/blob/main/src/transformers/pytorch_utils.py#L61)

**Parameters:**

layer (`torch.nn.Linear`) : The layer to prune.

index (`torch.LongTensor`) : The indices to keep in the layer.

dim (`int`, *optional*, defaults to 0) : The dimension on which to keep the indices.

**Returns:** `torch.nn.Linear`

The pruned layer as a new layer with `requires_grad=True`.

Prune a linear layer to keep only entries in index.

Used to remove heads.