张量的操作

发表于 更新于 分类于

1. 张量的操作：拼接、切分、索引和变换

image-20200720225048285

1.1 张量拼接与切分

1.1.1 通过torch.cat不拓展维度拼接

1
2
3
torch.cat(tensors, 
          dim=0, 
          out=None) → Tensor

功能：将张量按维度dim进行拼接

  • tensors: 张量序列
  • dim : 要拼接的维度
1
2
3
4
5
6
7
8
import torch

t = torch.arange(6).reshape((2, 3))

t_0 = torch.cat([t, t], dim=0)
t_1 = torch.cat([t, t, t], dim=1)

print("t:\n{}\nt_0:\n{} shape:{}\nt_1:\n{} shape:{}".format(t, t_0, t_0.shape, t_1, t_1.shape))
1
2
3
4
5
6
7
8
9
10
11
t:
tensor([[0, 1, 2],
        [3, 4, 5]])
t_0:
tensor([[0, 1, 2],
        [3, 4, 5],
        [0, 1, 2],
        [3, 4, 5]]) shape:torch.Size([4, 3])
t_1:
tensor([[0, 1, 2, 0, 1, 2, 0, 1, 2],
        [3, 4, 5, 3, 4, 5, 3, 4, 5]]) shape:torch.Size([2, 9])

dim=0就纵向拼接，dim=1就横向拼接

1.1.2 通过torch.stack在新的维度上拼接

1
2
3
torch.stack(tensors, 
            dim=0, 
            out=None) → Tensor

功能：在新创建的维度dim上进行拼接

  • tensors:张量序列
  • dim :要拼接的维度
1
2
3
4
5
6
7
8
import torch
import numpy as np

t = torch.arange(6).reshape((2, 3))

t_stack = torch.stack([t, t, t], dim=0)

print("t_stack:\n{} shape:{}".format(t_stack, t_stack.shape))
1
2
3
4
5
6
7
8
t_stack:
tensor([[[0, 1, 2],
         [3, 4, 5]],
        [[0, 1, 2],
         [3, 4, 5]],
        [[0, 1, 2],
         [3, 4, 5]]]) 
shape:torch.Size([3, 2, 3])

选择在第0维度stack，则把原来的0、1维度往后推至1、2维度，空出0维度用于拼接。

维度2就是第3个括号，维度0就是第一个括号

1.1.3 通过torch.chunk按维度平均切分

1
2
3
torch.chunk(input, 
            chunks, 
            dim=0) → List of Tensors

功能：将张量按维度dim进行平均切分
返回值：张量列表
注意事项：若不能整除，最后一份张量小于其他张量

  • input: 要切分的张量
  • chunks : 要切分的份数
  • dim : 要切分的维度
1
2
3
4
5
6
7
8
import torch
import numpy as np

a = torch.arange(14).reshape((2, 7))  # 7
list_of_tensors = torch.chunk(a, dim=1, chunks=3)   # 3

for idx, t in enumerate(list_of_tensors):
    print("第{}个张量：\n{}, shape is {}".format(idx+1, t, t.shape)) 
1
2
3
4
5
6
7
8
9
第1个张量：
tensor([[0, 1, 2],
        [7, 8, 9]]), shape is torch.Size([2, 3])
第2个张量：
tensor([[ 3,  4,  5],
        [10, 11, 12]]), shape is torch.Size([2, 3])
第3个张量：
tensor([[ 6],
        [13]]), shape is torch.Size([2, 1])

在维度1上做均分，最后一份不能等分就会比其他的短，长度7做3均分，则7/3=2.333，向上取整则前两份为3，最后一份为7-3-3=1

1.1.4 通过torch.split按维度平均切分

1
2
3
torch.split(tensor, 
            split_size_or_sections, 
            dim=0)

功能：将张量按维度dim进行切分
返回值：张量列表

  • tensor: 要切分的张量
  • split_size_or_sections : 为int时，表示每一份的长度；为list时，按list元素切分
  • dim : 要切分的维度
1
2
3
4
5
6
7
8
import torch
import numpy as np

t = torch.arange(10).reshape((2, 5))
list_of_tensors = torch.split(t, [2, 1, 2], dim=1)

for idx, t in enumerate(list_of_tensors):
    print("第{}个张量：\n{}, shape is {}".format(idx+1, t, t.shape))
1
2
3
4
5
6
7
8
9
第1个张量：
tensor([[0, 1],
        [5, 6]]), shape is torch.Size([2, 2])
第2个张量：
tensor([[2],
        [7]]), shape is torch.Size([2, 1])
第3个张量：
tensor([[3, 4],
        [8, 9]]), shape is torch.Size([2, 2])

如果这里[2, 1, 2]的和不是恰好输入张量t在维度1的维数，就会报错

1.2 张量索引

1.2.1 通过torch.index_select索引数据

1
2
3
4
torch.index_select(input, 
                   dim, 
                   index, 
                   out=None) → Tensor

功能：在维度dim上，按index索引数据
返回值：依index索引数据拼接的张量

  • input: 要索引的张量
  • dim: 要索引的维度
  • index : 要索引数据的序号
1
2
3
4
5
6
7
8
import torch
import numpy as np
torch.manual_seed(1)

t = torch.randint(0, 9, size=(3, 3))
idx = torch.tensor([0, 2], dtype=torch.long)  # float
t_select = torch.index_select(t, dim=0, index=idx)
print("t:\n{}\nt_select:\n{}".format(t, t_select))
1
2
3
4
5
6
7
t:
tensor([[4, 5, 0],
        [5, 7, 1],
        [2, 5, 8]])
t_select:
tensor([[4, 5, 0],
        [2, 5, 8]])

idx的数据类型必须是torch.long

1.2.2 通过torch.masked_select索引数据

1
2
3
torch.masked_select(input, 
                    mask, 
                    out=None) → Tensor

功能：按mask中的True进行索引
返回值：一维张量

  • input: 要索引的张量
  • mask: 与input同形状的布尔类型张量
1
2
3
4
5
6
7
8
9
import torch
import numpy as np

torch.manual_seed(1)

t = torch.randint(0, 9, size=(3, 3))
mask = t.le(5)  # ge is mean greater than or equal/   gt: greater than  le  lt
t_select = torch.masked_select(t, mask)
print("t:\n{}\nmask:\n{}\nt_select:\n{} ".format(t, mask, t_select))
1
2
3
4
5
6
7
8
9
10
t:
tensor([[4, 5, 0],
        [5, 7, 1],
        [2, 5, 8]])
mask:
tensor([[ True,  True,  True],
        [ True, False,  True],
        [ True,  True, False]])
t_select:
tensor([4, 5, 0, 5, 1, 2, 5])

注意：返回的是True位置的元素的一维张量

1.3 张量变换

1.3.1 通过torch.reshape改变张量的形状

1
torch.reshape(input, shape) → Tensor

功能：变换张量形状
注意事项：当张量在内存中是连续时，新张量与input共享数据内存，但是不会改变原张量形状
• input: 要变换的张量
• shape: 新张量的形状

1
2
3
4
5
6
7
8
9
10
11
12
import torch
import numpy as np
torch.manual_seed(1)

t = torch.randperm(8)
t_reshape = torch.reshape(t, (2, 4))
print("t:{}\nt_reshape:\n{}".format(t, t_reshape))

t[0] = 1024
print("t:{}\nt_reshape:\n{}".format(t, t_reshape))
print("t.data 内存地址:{}".format(id(t.data)))
print("t_reshape.data 内存地址:{}".format(id(t_reshape.data)))
1
2
3
4
5
6
7
8
9
10
t:tensor([5, 4, 2, 6, 7, 3, 1, 0])
t_reshape:
tensor([[5, 4, 2, 6],
        [7, 3, 1, 0]])
t:tensor([1024,    4,    2,    6,    7,    3,    1,    0])
t_reshape:
tensor([[1024,    4,    2,    6],
        [   7,    3,    1,    0]])
t.data 内存地址:2794497053544
t_reshape.data 内存地址:2794497053544

对比t.reshape((2, 4))，其实是一模一样的。

1
2
3
4
5
6
7
8
9
10
11
12
import torch
import numpy as np
torch.manual_seed(1)

t = torch.randperm(8)
t_reshape = t.reshape((2, 4))
print("t:{}\nt_reshape:\n{}".format(t, t_reshape))

t[0] = 1024
print("t:{}\nt_reshape:\n{}".format(t, t_reshape))
print("t.data 内存地址:{}".format(id(t.data)))
print("t_reshape.data 内存地址:{}".format(id(t_reshape.data)))
1
2
3
4
5
6
7
8
9
10
t:tensor([5, 1, 2, 6, 0, 3, 7, 4])
t_reshape:
tensor([[5, 1, 2, 6],
        [0, 3, 7, 4]])
t:tensor([1024,    1,    2,    6,    0,    3,    7,    4])
t_reshape:
tensor([[1024,    1,    2,    6],
        [   0,    3,    7,    4]])
t.data 内存地址:140520946484736
t_reshape.data 内存地址:140520946484736
1
2
3
4
5
6
7
8
t = torch.randperm(8)
t_reshape = torch.reshape(t, (-1, 2, 2))  # -1
print("t:{}\nt_reshape:\n{}".format(t, t_reshape))

t[0] = 1024
print("t:{}\nt_reshape:\n{}".format(t, t_reshape))
print("t.data 内存地址:{}".format(id(t.data)))
print("t_reshape.data 内存地址:{}".format(id(t_reshape.data)))
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
t:tensor([5, 4, 2, 6, 7, 3, 1, 0])
t_reshape:
tensor([[[5, 4],
         [2, 6]],

        [[7, 3],
         [1, 0]]])
t:tensor([1024,    4,    2,    6,    7,    3,    1,    0])
t_reshape:
tensor([[[1024,    4],
         [   2,    6]],

        [[   7,    3],
         [   1,    0]]])
t.data 内存地址:1488245429096
t_reshape.data 内存地址:1488245429096

-1表示这个维度的维数不用关心，是由其余维度自动确定的。上面代码中，维度1和维度2都是2，则可以自动计算出维度0的维数是2

1.3.2 通过torch.transpose交换张量两个维度

1
2
3
torch.transpose(input, 
                dim0, 
                dim1) → Tensor

功能：交换张量的两个维度

  • input: 要变换的张量
  • dim0: 要交换的维度
  • dim1: 要交换的维度
1
2
3
4
5
6
7
import torch
import numpy as np
torch.manual_seed(1)

t = torch.rand((2, 3, 4))
t_transpose = torch.transpose(t, dim0=1, dim1=2)  # c*h*w     h*w*c
print("t:{}\nt shape:{}\nt_transpose:{}\nt_transpose shape: {}".format(t, t.shape, t_transpose, t_transpose.shape))
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
t:tensor([[[0.7576, 0.2793, 0.4031, 0.7347],
         [0.0293, 0.7999, 0.3971, 0.7544],
         [0.5695, 0.4388, 0.6387, 0.5247]],

        [[0.6826, 0.3051, 0.4635, 0.4550],
         [0.5725, 0.4980, 0.9371, 0.6556],
         [0.3138, 0.1980, 0.4162, 0.2843]]])
t shape:torch.Size([2, 3, 4])
t_transpose:tensor([[[0.7576, 0.0293, 0.5695],
         [0.2793, 0.7999, 0.4388],
         [0.4031, 0.3971, 0.6387],
         [0.7347, 0.7544, 0.5247]],

        [[0.6826, 0.5725, 0.3138],
         [0.3051, 0.4980, 0.1980],
         [0.4635, 0.9371, 0.4162],
         [0.4550, 0.6556, 0.2843]]])
t_transpose shape: torch.Size([2, 4, 3])

1.3.2 通过torch.t转置张量

1
torch.t(input) → Tensor

功能：2维张量转置，对矩阵而言，等价于

1
torch.transpose(input, 0, 1)
1
2
3
4
5
6
7
import torch
import numpy as np
torch.manual_seed(1)

t = torch.rand((2, 3))
t_transpose = torch.t(t)  # c*h*w     h*w*c
print("t:{}\nt shape:{}\nt_transpose:{}\nt_transpose shape: {}".format(t, t.shape, t_transpose, t_transpose.shape))
1
2
3
4
5
6
7
t:tensor([[0.7576, 0.2793, 0.4031],
        [0.7347, 0.0293, 0.7999]])
t shape:torch.Size([2, 3])
t_transpose:tensor([[0.7576, 0.7347],
        [0.2793, 0.0293],
        [0.4031, 0.7999]])
t_transpose shape: torch.Size([3, 2])

1.3.3 通过torch.squeeze压缩张量维度

1
torch.squeeze(input, dim=None, out=None) → Tensor

功能：压缩长度为1的维度（轴）

  • dim: 若为None，移除所有长度为1的轴；若指定维度，当且仅当该轴长度为1时，可以被移除；
1
2
3
4
5
6
7
8
9
10
11
12
import torch
import numpy as np
torch.manual_seed(1)

t = torch.rand((1, 2, 3, 1))
t_sq = torch.squeeze(t)
t_0 = torch.squeeze(t, dim=0)
t_1 = torch.squeeze(t, dim=1)
print('t = \n{}\nt.shape = {}\n'.format(t, t.shape))
print('t_sq = \n{}\nt_sq.shape = {}\n'.format(t_sq, t_sq.shape))
print('t_0 = \n{}\nt_0.shape = {}\n'.format(t_0, t_0.shape))
print('t_1 = \n{}\nt_1.shape = {}\n'.format(t_1, t_1.shape))
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
t = 
tensor([[[[0.7576],
          [0.2793],
          [0.4031]],

         [[0.7347],
          [0.0293],
          [0.7999]]]])
t.shape = torch.Size([1, 2, 3, 1])

t_sq = 
tensor([[0.7576, 0.2793, 0.4031],
        [0.7347, 0.0293, 0.7999]])
t_sq.shape = torch.Size([2, 3])

t_0 = 
tensor([[[0.7576],
         [0.2793],
         [0.4031]],

        [[0.7347],
         [0.0293],
         [0.7999]]])
t_0.shape = torch.Size([2, 3, 1])

t_1 = 
tensor([[[[0.7576],
          [0.2793],
          [0.4031]],

         [[0.7347],
          [0.0293],
          [0.7999]]]])
t_1.shape = torch.Size([1, 2, 3, 1])

就是去除多余的括号，因为dim=1时，括号是多余的。比如[[3， 2]]这个的维度是(1, 2)那么其实和[3, 2]是没有区别的

1.3.4 通过torch.unsqueeze扩展张量维度

1
torch.unsqueeze(input, dim) → Tensor

功能：依据dim扩展维度

  • dim: 扩展的维度
1
2
3
4
5
6
7
8
9
10
11
12
13
import torch
import numpy as np
torch.manual_seed(1)

t = torch.arange(3)
t = torch.reshape(t, (1, 3))
t_sq = torch.squeeze(t)
t_0 = torch.unsqueeze(t_sq, dim=0)
t_1 = torch.unsqueeze(t_sq, dim=1)
print('t = \n{}\nt.shape = {}\n'.format(t, t.shape))
print('t_sq = \n{}\nt_sq.shape = {}\n'.format(t_sq, t_sq.shape))
print('t_0 = \n{}\nt_0.shape = {}\n'.format(t_0, t_0.shape))
print('t_1 = \n{}\nt_1.shape = {}\n'.format(t_1, t_1.shape))
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
t = 
tensor([[0, 1, 2]])
t.shape = torch.Size([1, 3])

t_sq = 
tensor([0, 1, 2])
t_sq.shape = torch.Size([3])

t_0 = 
tensor([[0, 1, 2]])
t_0.shape = torch.Size([1, 3])

t_1 = 
tensor([[0],
        [1],
        [2]])
t_1.shape = torch.Size([3, 1])

就是加括号，加维度为1的没有用的括号。

2. 张量的数学运算

image-20200720223450998

2.1 通过torch.add对两个张量相加

1
torch.add(input, other, *, alpha=1, out=None)

功能：逐元素计算out=input+alpha×other

  • input: 第一个张量
  • alpha: 乘项因子
  • other: 第二个张量
1
2
3
4
5
6
7
8
9
10
import torch
import numpy as np

torch.manual_seed(1)

t_0 = torch.randn((3, 3))
t_1 = torch.ones_like(t_0)
t_add = torch.add(t_0, 10, t_1)

print("t_0:\n{}\nt_1:\n{}\nt_add_10:\n{}".format(t_0, t_1, t_add))
1
2
3
4
5
6
7
8
9
10
11
12
t_0:
tensor([[ 0.6614,  0.2669,  0.0617],
        [ 0.6213, -0.4519, -0.1661],
        [-1.5228,  0.3817, -1.0276]])
t_1:
tensor([[1., 1., 1.],
        [1., 1., 1.],
        [1., 1., 1.]])
t_add_10:
tensor([[10.6614, 10.2669, 10.0617],
        [10.6213,  9.5481,  9.8339],
        [ 8.4772, 10.3817,  8.9724]])

(input, other, *, alpha=1, out=None)的$ *表示表示 *$后面的参数只能用关键字传参

例外如果见到/，则表示/前面的参数只能用位置参数

2.2 通过torch.addcmul对张量加乘

1
torch.addcmul(input, tensor1, tensor2, *, value=1, out=None) → Tensor
  • input (Tensor) – the tensor to be added
  • tensor1 (Tensor) – the tensor to be multiplied
  • tensor2 (Tensor) – the tensor to be multiplied
  • value (Number*,* optional) – multiplier for tensor1. ∗* tensor 2
  • out (Tensor, optional) – the output tensor.

outi=inputi+value×tensor1i×tensor2iout_i=input_i+value×tensor1_i×tensor2_i

1
2
3
4
5
6
7
8
9
10
import torch
import numpy as np
torch.manual_seed(1)

t_0 = torch.reshape(torch.arange(6), (2, 3))
t_1 = torch.reshape(torch.arange(6, 12), (2, 3))
t_2 = torch.ones_like(t_0)
t_add = torch.addcmul(t_0, t_1, t_2, value=2)

print("t_0:\n{}\nt_1:\n{}\nt_2:\n{}\nt_add:\n{}".format(t_0, t_1, t_2, t_add))
1
2
3
4
5
6
7
8
9
10
11
12
t_0:
tensor([[0, 1, 2],
        [3, 4, 5]])
t_1:
tensor([[ 6,  7,  8],
        [ 9, 10, 11]])
t_2:
tensor([[1, 1, 1],
        [1, 1, 1]])
t_add:
tensor([[12, 15, 18],
        [21, 24, 27]])
1
2
3
4
5
6
7
8
9
10
import torch
import numpy as np
torch.manual_seed(1)

t_0 = torch.reshape(torch.arange(6, dtype=float), (2, 3))
t_1 = torch.reshape(torch.arange(6, 12, dtype=float), (2, 3))
t_2 = torch.ones_like(t_0, dtype=float)
t_add = torch.addcmul(t_0, t_1, t_2, value=0.2)

print("t_0:\n{}\nt_1:\n{}\nt_2:\n{}\nt_add:\n{}".format(t_0, t_1, t_2, t_add))
1
2
3
4
5
6
7
8
9
10
11
12
t_0:
tensor([[0., 1., 2.],
        [3., 4., 5.]], dtype=torch.float64)
t_1:
tensor([[ 6.,  7.,  8.],
        [ 9., 10., 11.]], dtype=torch.float64)
t_2:
tensor([[1., 1., 1.],
        [1., 1., 1.]], dtype=torch.float64)
t_add:
tensor([[1.2000, 2.4000, 3.6000],
        [4.8000, 6.0000, 7.2000]], dtype=torch.float64)

如果value为浮点数，则参与运算的tensor必须都是浮点类型

2.3 其他函数

我就不写了，学会查看官方文档，非常方便而且详细。

https://pytorch.org/docs/stable/index.html