mindspore_Ghostnet/GhostNet.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "ecf03d06",
   "metadata": {},
   "source": [
    "## **引言**\n",
    "\n",
    "卷积神经网络推动了计算机视觉诸多任务的进步，比如图像识别、目标检测等。但是，神经网络在移动设备上的应用还亟待解决，主要原因是现有模型**又大又慢**。因而，一些研究提出了模型的压缩方法，比如剪枝、量化、知识蒸馏等；还有一些则着重于高效的网络结构设计，比如MobileNet，ShuffleNet等。本文就设计了一种全新的神经网络基本单元Ghost模块，从而搭建出轻量级神经网络架构**GhostNet**。\n",
    "\n",
    "众所周知，在一个训练好的深度神经网络中，通常会包含丰富甚至**冗余的特征图**，以保证对输入数据有全面的理解。\n",
    "\n",
    "![image1](./images/image-20221010175743115.png)\n",
    "\n",
    "图1 ResNet50第一个残差块处理后的特征图可视化\n",
    "\n",
    "如上图所示，在**ResNet-50**中，将经过第一个残差块处理后的特征图拿出来，三个相似的（**红、蓝、绿**）特征图对示例用相同颜色的框注释。 该对中的一个特征图可以通过**廉价操作**将另一特征图变换而获得，可以认为其中一个特征图是另一个的“**ghost**”。\n",
    "\n",
    "在本文中，作者提出了一种新颖的**Ghost模块**，可以使用**更少的参数**来生成更多特征图。\n",
    "\n",
    "具体来说，深度神经网络中的**普通卷积层**将分为**两部分**。第一部分涉及**普通卷积**，但是将严格控制它们的总数。给定第一部分的**固有特征图**，然后将一系列**简单的线性运算**应用于**生成更多特征图**。\n",
    "\n",
    "与普通卷积神经网络相比，在不更改输出特征图大小的情况下，该Ghost模块中所需的**参数总数**和**计算复杂度**均已降低。基于Ghost模块，作者建立了一种有效的神经体系结构，即**GhostNet**。作者首先在基准神经体系结构中**替换原始的卷积层**，以证明Ghost模块的有效性，然后在几个基准视觉数据集上验证GhostNet的优越性。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3db24731",
   "metadata": {},
   "source": [
    "普通卷积操作：**卷积操作**就是**卷积核**（过滤器 / Filter）在原始图片中**进行滑动**得到**特征图**的过程。\n",
    "\n",
    "![image-20221010180117058](./images/image-20221010180117058.png)\n",
    "\n",
    "图2 图像与卷积核\n",
    "\n",
    "\n",
    "\n",
    "![image-20221010180259527](./images\\image-20221010180259527.png)\n",
    "![image-20221010180324315](./images\\image-20221010180324315.png)\n",
    "\n",
    "图3 卷积操作\n",
    "\n",
    "![image-20221010182805976](./images\\image-20221010182805976.png)\n",
    "\n",
    "图4 卷积操作"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2c62cac4",
   "metadata": {},
   "source": [
    " 我们可以通过在卷积运算之前**更改卷积核矩阵的数值**来执行诸如边缘检测，锐化和模糊之类的操作——这意味着不同的卷积核可以从图像中检测不同的特征，例如边缘， 曲线等。\n",
    "\n",
    "![image-20221010183348805](./images\\image-20221010183348805.png)\n",
    "\n",
    "图5 卷积结果\n",
    "\n",
    "上图输入通道为3channel,3x3\n",
    "\n",
    "核函数为3个卷积核（输入通道为3，2个channel（输出通道为2），kernel_size未知\n",
    "\n",
    "输出即为2个通道\n",
    "\n",
    "普通卷积\n",
    "![image-20221011152412155](./images\\image-20221011152412155.png)\n",
    "\n",
    "ghost module操作\n",
    "![image-20221011152542091](./images\\image-20221011152542091.png)\n",
    "\n",
    "\n",
    "图6 普通卷积与ghost module操作\n",
    "\n",
    "从m个通道的输入数据生成n个通道的输出，卷积核的个数是m*n，在一些比较大的通道数上，其效率会非常的低。而且普通卷积的方法在输出的不同特征图中，通常会包含丰富甚至冗余的**特征图**。通过研究发现这些特征图可以由通过一些**简单线性变换**得到。\n",
    "\n",
    "Ghost module的方法将其中**一部分特征图**通过以往的普通卷积方法得到，**另一部分**通过一些**线性变换**从这些**已经得到的特征图**中获取。这样就能大大**降低卷积操作的次数**，提高运行效率。\n",
    "\n",
    "例：从m个通道的输入数据生成n个通道的输出，卷积核的个数是**mn**\n",
    "\n",
    "若使用Ghost module，设中间特征通道数为m<p<n，且线性变换为**depthwise conv**，则卷积核的个数为**mp+(n-p)**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6094cde0",
   "metadata": {},
   "source": [
    "## GhostNet结构\n",
    "\n",
    "下图是Ghost bottleneck结构图，很类似resnet结构，不同的是channel是先升维再降维。\n",
    "\n",
    "![image-20221011152243605](./images\\image-20221011152243605.png)\n",
    "![image-20221011152249621](./images\\image-20221011152249621.png)\n",
    "\n",
    "图7 Ghost bottleneck结构图\n",
    "\n",
    "下面是GhostNet的网络结构图\n",
    "\n",
    "<img src=\"./images\\image-20221011152641626.png\" alt=\"image-20221011152641626\" style=\"zoom:50%;\" />\n",
    "\n",
    "​\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t图8 GhostNet的网络结构图"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "07000cd2",
   "metadata": {},
   "source": [
    "先导入各种前置包"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "3f52f327",
   "metadata": {},
   "outputs": [],
   "source": [
    "from functools import partial\n",
    "import math\n",
    "import numpy as np\n",
    "import mindspore.nn as nn\n",
    "from mindspore.ops import operations as P\n",
    "from mindspore import Tensor\n",
    "import mindspore as ms\n",
    "from mindspore import ops, Tensor, context, nn\n",
    "from mindspore import dtype as mstype\n",
    "\n",
    "#禁用warnings\n",
    "import warnings\n",
    "warnings.filterwarnings(\"ignore\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f314561f",
   "metadata": {},
   "source": [
    "数据集加载"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7c3d96cf-0249-4269-8be9-912f70653af4",
   "metadata": {},
   "outputs": [],
   "source": [
    "#如果有ASCEND则ASCEND 加速\n",
    "# ms.set_context(mode=ms.GRAPH_MODE, device_target=\"Ascend\")\n",
    "\n",
    "# 数据集读取\n",
    "from mindvision.classification.dataset import Cifar10\n",
    "\n",
    "# 数据集根目录\n",
    "data_dir = \"./CIFAR10\"\n",
    "\n",
    "# 下载解压并加载CIFAR-10训练数据集\n",
    "download_train = Cifar10(path=data_dir, split=\"train\", batch_size=4096, repeat_num=1, shuffle=True, resize=32, download=True)\n",
    "dataset_train = download_train.run()\n",
    "\n",
    "step_size = dataset_train.get_dataset_size()\n",
    "\n",
    "# 下载解压并加载CIFAR-10测试数据集\n",
    "download_eval = Cifar10(path=data_dir, split=\"test\", batch_size=1024, resize=32, download=True)\n",
    "dataset_eval = download_eval.run()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "19ecf6b8",
   "metadata": {},
   "source": [
    "查看数据集结构"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "85c40e03-49ed-4e54-a4b1-d46ea19ee4fe",
   "metadata": {},
   "outputs": [],
   "source": [
    "# 可视化\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "data = next(dataset_train.create_dict_iterator())\n",
    "\n",
    "images = data[\"image\"].asnumpy()\n",
    "labels = data[\"label\"].asnumpy()\n",
    "print(f\"Image shape: {images.shape}, Label: {labels}\")\n",
    "\n",
    "plt.figure()\n",
    "for i in range(1, 7):\n",
    "    plt.subplot(2, 3, i)\n",
    "    image_trans = np.transpose(images[i - 1], (1, 2, 0))\n",
    "    mean = np.array([0.4914, 0.4822, 0.4465])\n",
    "    std = np.array([0.2023, 0.1994, 0.2010])\n",
    "    image_trans = std * image_trans + mean\n",
    "    image_trans = np.clip(image_trans, 0, 1)\n",
    "    plt.title(f\"{download_train.index2label[labels[i - 1]]}\")\n",
    "    plt.imshow(image_trans)\n",
    "    plt.axis(\"off\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "15705941",
   "metadata": {},
   "source": [
    "深度可分离卷积的结构\n",
    "depthwise conv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "57d72ebd-1033-4c81-b959-663beebf5f61",
   "metadata": {},
   "outputs": [],
   "source": [
    "class ConvBnAct(nn.Cell):\n",
    "    def __init__(self, in_chs, out_chs, kernel_size,\n",
    "                 stride=1, act_layer=nn.ReLU):\n",
    "        super(ConvBnAct, self).__init__()\n",
    "        self.conv = nn.Conv2d(in_chs, out_chs, kernel_size, stride, pad_mode='pad', padding=kernel_size // 2,\n",
    "                              has_bias=False)\n",
    "        self.bn = nn.BatchNorm2d(out_chs)\n",
    "        self.act = act_layer()\n",
    "\n",
    "    def construct(self, x):\n",
    "        x = self.conv(x)\n",
    "        x = self.bn(x)\n",
    "        x = self.act(x)\n",
    "        return x"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "547e9870",
   "metadata": {},
   "source": [
    "构造GhostModule层\n",
    "最为重要的部分"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6e975bcb-8ca1-4a47-94bd-a78bebf467b5",
   "metadata": {},
   "outputs": [],
   "source": [
    "class GhostModule(nn.Cell):\n",
    "\n",
    "    def __init__(self, in_channels, out_channels, kernel_size=1, ratio=2, dw_size=3, stride=1, relu=True):\n",
    "        super(GhostModule, self).__init__()\n",
    "        self.out_channels = out_channels\n",
    "        init_channels = math.ceil(out_channels / ratio)  \n",
    "        new_channels = init_channels * (ratio - 1)  \n",
    "            \n",
    "        self.primary_conv = nn.SequentialCell(\n",
    "            nn.Conv2d(in_channels=in_channels, out_channels=init_channels, kernel_size=kernel_size, stride=stride,\n",
    "                      pad_mode='pad', padding=kernel_size // 2, has_bias=False),\n",
    "            nn.BatchNorm2d(init_channels),\n",
    "            nn.ReLU() if relu else nn.SequentialCell(),\n",
    "        )\n",
    "        self.cheap_operation = nn.SequentialCell(\n",
    "            nn.Conv2d(in_channels=init_channels, out_channels=new_channels, kernel_size=dw_size, stride=1,\n",
    "                      pad_mode='pad', padding=dw_size // 2, group=init_channels, has_bias=False),\n",
    "            nn.BatchNorm2d(new_channels),\n",
    "            nn.ReLU() if relu else nn.SequentialCell(),\n",
    "        )\n",
    "\n",
    "    def construct(self, x):\n",
    "        \"\"\" construct \"\"\"\n",
    "        x1 = self.primary_conv(x)\n",
    "        x2 = self.cheap_operation(x1)\n",
    "        concat = ms.ops.Concat(axis=1)\n",
    "        out = concat((x1, x2))\n",
    "        return out[:, :self.out_channels, :, :]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4a1e2678-883c-440c-9585-00902e1cb5f0",
   "metadata": {},
   "outputs": [],
   "source": [
    "def _make_divisible(x, divisor=4):\n",
    "    return int(np.ceil(x * 1. / divisor) * divisor)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f9b70fb9-fe37-4765-9e26-8a7aebe40148",
   "metadata": {},
   "outputs": [],
   "source": [
    "def hard_sigmoid(x, inplace: bool = False):\n",
    "    if inplace:\n",
    "        return x.add_(3.).clamp_(0., 6.).div_(6.)\n",
    "    else:\n",
    "        relu = ms.ops.ReLU()\n",
    "        return relu(x + 3.) / 6."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "528db059-a08c-4e43-942a-d1eddf090f6f",
   "metadata": {},
   "outputs": [],
   "source": [
    "class GlobalAvgPooling(nn.Cell):\n",
    "    def __init__(self, keep_dims=False):\n",
    "        super(GlobalAvgPooling, self).__init__()\n",
    "        self.mean = P.ReduceMean(keep_dims=keep_dims)\n",
    "\n",
    "    def construct(self, x):\n",
    "        \"\"\" construct \"\"\"\n",
    "        x = self.mean(x, (2, 3))\n",
    "        return x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ceb52498-ada3-44be-b3db-882bf2620de1",
   "metadata": {},
   "outputs": [],
   "source": [
    "class SE(nn.Cell):\n",
    "    def __init__(self, num_out, ratio=4):\n",
    "        super(SE, self).__init__()\n",
    "        num_mid = _make_divisible(num_out // ratio)\n",
    "        self.pool = GlobalAvgPooling(keep_dims=True)\n",
    "        self.conv_reduce = nn.Conv2d(in_channels=num_out, out_channels=num_mid,\n",
    "                                     kernel_size=1, has_bias=True, pad_mode='pad')\n",
    "        self.act1 = nn.ReLU()\n",
    "        self.conv_expand = nn.Conv2d(in_channels=num_mid, out_channels=num_out,\n",
    "                                     kernel_size=1, has_bias=True, pad_mode='pad')\n",
    "        self.act2 = hard_sigmoid\n",
    "        self.mul = P.Mul()\n",
    "\n",
    "    def construct(self, x):\n",
    "        \"\"\" construct of SE module \"\"\"\n",
    "        out = self.pool(x)\n",
    "        out = self.conv_reduce(out)\n",
    "        out = self.act1(out)\n",
    "        out = self.conv_expand(out)\n",
    "        out = self.act2(out)\n",
    "        out = self.mul(x, out)\n",
    "        return out"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b1e75041",
   "metadata": {},
   "source": [
    "Bottleneck结构"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5c294dd7-242d-4906-b331-bcf382c2fe25",
   "metadata": {},
   "outputs": [],
   "source": [
    "class GhostBottleneck(nn.Cell):\n",
    "    \"\"\" Ghost bottleneck w/ optional SE\"\"\"\n",
    "\n",
    "    def __init__(self, in_chs, mid_chs, out_chs, dw_kernel_size=3,\n",
    "                 stride=1, act_layer=nn.ReLU, se_ratio=0.):\n",
    "        super(GhostBottleneck, self).__init__()\n",
    "        has_se = se_ratio is not None and se_ratio > 0.\n",
    "        self.stride = stride\n",
    "\n",
    "        self.ghost1 = GhostModule(in_chs, mid_chs, relu=True)\n",
    "\n",
    "        if self.stride > 1:\n",
    "            self.conv_dw = nn.Conv2d(in_channels=mid_chs, out_channels=mid_chs, kernel_size=dw_kernel_size,\n",
    "                                     stride=stride, pad_mode='pad',\n",
    "                                     padding=(dw_kernel_size - 1) // 2,\n",
    "                                     group=mid_chs, has_bias=False)\n",
    "            self.bn_dw = nn.BatchNorm2d(mid_chs)\n",
    "\n",
    "        if has_se:\n",
    "            self.se = SE(mid_chs, ratio=se_ratio)\n",
    "        else:\n",
    "            self.se = None\n",
    "            \n",
    "        self.ghost2 = GhostModule(mid_chs, out_chs, relu=False)\n",
    "        \n",
    "        if (in_chs == out_chs and self.stride == 1):\n",
    "            self.shortcut = nn.SequentialCell()\n",
    "        else:\n",
    "            self.shortcut = nn.SequentialCell(\n",
    "                nn.Conv2d(in_channels=in_chs, out_channels=in_chs, kernel_size=dw_kernel_size, stride=stride,\n",
    "                          pad_mode='pad', padding=(dw_kernel_size - 1) // 2, group=in_chs, has_bias=False),\n",
    "                nn.BatchNorm2d(in_chs),\n",
    "                nn.Conv2d(in_channels=in_chs, out_channels=out_chs, kernel_size=1, stride=1,\n",
    "                          pad_mode='valid', padding=0, has_bias=False),\n",
    "                nn.BatchNorm2d(out_chs),\n",
    "            )\n",
    "\n",
    "    def construct(self, x):\n",
    "        residual = x\n",
    "        x = self.ghost1(x)\n",
    "        if self.stride > 1:\n",
    "            x = self.conv_dw(x)\n",
    "            x = self.bn_dw(x)\n",
    "        if self.se is not None:\n",
    "            x = self.se(x)\n",
    "        x = self.ghost2(x)\n",
    "        x += self.shortcut(residual)\n",
    "        return x"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c19ca091",
   "metadata": {},
   "source": [
    "主体部分\n",
    "\n",
    "值得注意的是：\n",
    "\n",
    "1、mindspore中在前向传播处使用的任何变量需要在init中定义为self变量才可使用\n",
    "\n",
    "2、最后的output层的liner采用dense"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2c484cac-5c11-4f52-80e3-581a9679705c",
   "metadata": {},
   "outputs": [],
   "source": [
    "class GhostNet(nn.Cell):\n",
    "    def __init__(self, cfgs,in_channel = 3, num_classes=10, width=1.0, dropout=0.2):\n",
    "        super(GhostNet, self).__init__()\n",
    "        # setting of inverted residual blocks\n",
    "        self.cfgs = cfgs\n",
    "\n",
    "        # ---- building first layer ---- #\n",
    "        output_channel = _make_divisible(16 * width, 4)  # setting divisible channels | output_channel = 16\n",
    "        self.conv_stem = nn.Conv2d(in_channels=in_channel,\n",
    "                                   out_channels=output_channel,\n",
    "                                   kernel_size=3, \n",
    "                                   padding=1, \n",
    "                                   stride=2,\n",
    "                                   has_bias=False, \n",
    "                                   pad_mode='pad')  # first conv\n",
    "        self.bn1 = nn.BatchNorm2d(output_channel)\n",
    "        self.act1 = nn.ReLU()\n",
    "        input_channel = output_channel\n",
    "\n",
    "        # ---- building inverted residual [blocks] ---- #\n",
    "        stages = nn.SequentialCell()\n",
    "        block = GhostBottleneck\n",
    "        for cfg in self.cfgs:\n",
    "            layers = nn.SequentialCell()\n",
    "            for k, exp_size, c, se_ratio, s in cfg:\n",
    "                output_channel = _make_divisible(c * width, 4)\n",
    "                hidden_channel = _make_divisible(exp_size * width, 4)\n",
    "                layers.append(block(input_channel, hidden_channel, output_channel, k, s,\n",
    "                                    se_ratio=se_ratio))\n",
    "                input_channel = output_channel\n",
    "            stages.append(layers)\n",
    "\n",
    "        output_channel = _make_divisible(exp_size * width, 4)\n",
    "        stages.append(ConvBnAct(input_channel, output_channel, 1))\n",
    "\n",
    "        input_channel = output_channel\n",
    "        self.blocks = stages\n",
    "\n",
    "        # ---- building last several layers ---- #\n",
    "        output_channel = 128\n",
    "        self.global_pool = GlobalAvgPooling(keep_dims=True)\n",
    "        self.conv_head = nn.Conv2d(in_channels=input_channel,\n",
    "                                   out_channels=output_channel,\n",
    "                                   kernel_size=1, padding=0, stride=1,\n",
    "                                   has_bias=True, pad_mode='pad')\n",
    "        self.act2 = nn.ReLU()\n",
    "        self.squeeze = P.Flatten()\n",
    "\n",
    "        # ---- cls head ---- #\n",
    "        self.final_dropout = dropout\n",
    "        if self.final_dropout > 0.:\n",
    "            self.Dropout = nn.Dropout(self.final_dropout)\n",
    "            \n",
    "        self.classifier = nn.Dense(output_channel, num_classes,has_bias=True)\n",
    "        \n",
    "        self._initialize_weights()\n",
    "\n",
    "    def construct(self, x):\n",
    "        # ---- Input Conv ---- #\n",
    "        x = self.conv_stem(x)  # (bs, 1, 224, 224) -> (bs, 16, 112, 112)\n",
    "        x = self.bn1(x)\n",
    "        x = self.act1(x)\n",
    "        # ---- Blocks Conv ---- #\n",
    "        x = self.blocks(x)\n",
    "\n",
    "        # ---- Last Conv ---- #\n",
    "        x = self.global_pool(x)\n",
    "        x = self.conv_head(x)\n",
    "        x = self.act2(x)\n",
    "\n",
    "        # ---- FC && Cls ---- #\n",
    "        x = self.squeeze(x)\n",
    "        if self.final_dropout > 0.:\n",
    "            x = self.Dropout(x)\n",
    "        x = self.classifier(x)\n",
    "\n",
    "        return x\n",
    "    \n",
    "    def _initialize_weights(self):\n",
    "        self.init_parameters_data()\n",
    "        for _, m in self.cells_and_names():\n",
    "            if isinstance(m, (nn.Conv2d)):\n",
    "                n = m.kernel_size[0] * m.kernel_size[1] * m.out_channels\n",
    "                m.weight.set_data(Tensor(np.random.normal(0, np.sqrt(2. / n),\n",
    "                                                          m.weight.data.shape).astype(\"float32\")))\n",
    "                if m.bias is not None:\n",
    "                    m.bias.set_data(\n",
    "                        Tensor(np.zeros(m.bias.data.shape, dtype=\"float32\")))\n",
    "            elif isinstance(m, nn.BatchNorm2d):\n",
    "                m.gamma.set_data(\n",
    "                    Tensor(np.ones(m.gamma.data.shape, dtype=\"float32\")))\n",
    "                m.beta.set_data(\n",
    "                    Tensor(np.zeros(m.beta.data.shape, dtype=\"float32\")))\n",
    "            elif isinstance(m, nn.Dense):\n",
    "                m.weight.set_data(Tensor(np.random.normal(\n",
    "                    0, 0.01, m.weight.data.shape).astype(\"float32\")))\n",
    "                if m.bias is not None:\n",
    "                    m.bias.set_data(\n",
    "                        Tensor(np.zeros(m.bias.data.shape, dtype=\"float32\")))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c80bb705",
   "metadata": {},
   "source": [
    "参数列表\n",
    "\n",
    "参考上文的GhostNet网络结构图"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ce096adb-eaf3-4846-898f-0dba949007fc",
   "metadata": {},
   "outputs": [],
   "source": [
    "def ghostnet(**kwargs):\n",
    "    \"\"\"\n",
    "    Constructs a GhostNet model\n",
    "    \"\"\"\n",
    "    cfgs = [\n",
    "        # kernel_size, exp_size(hidden_size), output_channels, SE_rate, stride\n",
    "        # stage 1\n",
    "        [[3, 16, 16, 0, 1]],   \n",
    "        [[3, 48, 24, 0, 2]],\n",
    "\n",
    "        # stage 2\n",
    "        [[3, 72, 24, 0, 1]],\n",
    "        [[5, 72, 40, 0.25, 2]],\n",
    "\n",
    "        # stage 3\n",
    "        [[5, 120, 40, 0.25, 1]],\n",
    "        [[3, 240, 80, 0, 2]],\n",
    "\n",
    "        # stage 4\n",
    "        [[3, 200, 80, 0, 1],\n",
    "         [3, 184, 80, 0, 1],\n",
    "         [3, 184, 80, 0, 1],\n",
    "         [3, 480, 112, 0.25, 1],\n",
    "         [3, 672, 112, 0.25, 1]],\n",
    "        [[5, 672, 160, 0.25, 2]],\n",
    "\n",
    "        # stage 5\n",
    "        [[5, 960, 160, 0, 1],\n",
    "         [5, 960, 160, 0.25, 1],\n",
    "         [5, 960, 160, 0, 1],\n",
    "         [5, 960, 160, 0.25, 1]]\n",
    "    ]\n",
    "    return GhostNet(cfgs, **kwargs)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0c26926d",
   "metadata": {},
   "source": [
    "测试一下网络能不能跑通"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "065ffbcd-f9b6-4d14-a59d-c99343a13887",
   "metadata": {},
   "outputs": [],
   "source": [
    "net = ghostnet()\n",
    "uniform = ms.ops.UniformReal()\n",
    "input = uniform((24, 3, 36, 36)) \n",
    "output = net(input)\n",
    "\n",
    "print(output.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bef782ad-7b62-48ff-a731-d698e53791ad",
   "metadata": {},
   "outputs": [],
   "source": [
    "epochs_num = 1000"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "800b43db",
   "metadata": {},
   "source": [
    "以下是mindspore的训练loss监视器，我们将其继承并添加了几个功能，让其可以直接将结果输出到文本中"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "42030d75-194d-403c-9e06-eb9afb81a555",
   "metadata": {},
   "outputs": [],
   "source": [
    "from mindvision.engine.callback import LossMonitor\n",
    "from mindspore.train.callback._callback import Callback, _handle_loss\n",
    "import time\n",
    "class Mylossmonitor(LossMonitor):\n",
    "    def __init__(self, fp, lr_init=None, per_print_times=1):\n",
    "        self.lr_init = lr_init\n",
    "        self.per_print_times = per_print_times\n",
    "        self.fp =fp\n",
    "        self.last_print_time = 0\n",
    "    \n",
    "    def epoch_end(self, run_context):\n",
    "        callback_params = run_context.original_args()\n",
    "        epoch_mseconds = (time.time() - self.epoch_time) * 1000\n",
    "        per_step_mseconds = epoch_mseconds / callback_params.batch_num\n",
    "        print(f\"Epoch time: {epoch_mseconds:5.3f} ms, \"\n",
    "              f\"per step time: {per_step_mseconds:5.3f} ms, \"\n",
    "              f\"avg loss: {np.mean(self.losses):5.3f}\", flush=True)\n",
    "        self.fp.write(f\"Epoch time: {epoch_mseconds:5.3f} ms, \" +\n",
    "              f\"per step time: {per_step_mseconds:5.3f} ms, \" + \n",
    "              f\"avg loss: {np.mean(self.losses):5.3f}\\n\")\n",
    "    \n",
    "    def step_end(self, run_context):\n",
    "        \"\"\"After step end print training info.\"\"\"\n",
    "        callback_params = run_context.original_args()\n",
    "        step_mseconds = (time.time() - self.step_time) * 1000\n",
    "        loss = callback_params.net_outputs\n",
    "\n",
    "        if isinstance(loss, (tuple, list)):\n",
    "            if isinstance(loss[0], ms.Tensor) and isinstance(loss[0].asnumpy(), np.ndarry):\n",
    "                loss = loss[0]\n",
    "\n",
    "        if isinstance(loss, ms.Tensor) and isinstance(loss.asnumpy(), np.ndarray):\n",
    "            loss = np.mean(loss.asnumpy())\n",
    "\n",
    "        self.losses.append(loss)\n",
    "        cur_step_in_epoch = (callback_params.cur_step_num - 1) % callback_params.batch_num + 1\n",
    "\n",
    "        # Boundary check.\n",
    "        if isinstance(loss, float) and (np.isnan(loss) or np.isinf(loss)):\n",
    "            raise ValueError(f\"Invalid loss, terminate training.\")\n",
    "\n",
    "        def print_info():\n",
    "            lr_output = self.lr_init[callback_params.cur_step_num - 1] if isinstance(self.lr_init,\n",
    "                                                                                     list) else self.lr_init\n",
    "            print(f\"Epoch:[{(callback_params.cur_epoch_num - 1):3d}/{callback_params.epoch_num:3d}], \"\n",
    "                  f\"step:[{cur_step_in_epoch:5d}/{callback_params.batch_num:5d}], \"\n",
    "                  f\"loss:[{loss:5.3f}/{np.mean(self.losses):5.3f}], \"\n",
    "                  f\"time:{step_mseconds:5.3f} ms, \"\n",
    "                  f\"lr:{lr_output:5.5f}\", flush=True)\n",
    "            self.fp.write(f\"Epoch:[{(callback_params.cur_epoch_num - 1):3d}/{callback_params.epoch_num:3d}], \" +\n",
    "                  f\"step:[{cur_step_in_epoch:5d}/{callback_params.batch_num:5d}], \" +\n",
    "                  f\"loss:[{loss:5.3f}/{np.mean(self.losses):5.3f}], \" +\n",
    "                  f\"time:{step_mseconds:5.3f} ms, \" +\n",
    "                  f\"lr:{lr_output:5.5f}\\n\")\n",
    "\n",
    "        if (callback_params.cur_step_num - self.last_print_time) >= self.per_print_times:\n",
    "            self.last_print_time = callback_params.cur_step_num\n",
    "            print_info()\n",
    "        "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ef942325",
   "metadata": {},
   "source": [
    "训练部分\n",
    "\n",
    "loss采用SoftmaxCrossEntropyWithLogits\n",
    "\n",
    "optimizer采用带动量的Momentum,lr为0.01"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8e873bd9-cca8-4cd9-abbf-618438fd4d09",
   "metadata": {},
   "outputs": [],
   "source": [
    "from mindspore.train.serialization import load_checkpoint, load_param_into_net\n",
    "\n",
    "net_loss = nn.SoftmaxCrossEntropyWithLogits(sparse=True, reduction='mean')\n",
    "\n",
    "net_opt = nn.Momentum(net.trainable_params(), learning_rate=0.01, momentum=0.9)\n",
    "\n",
    "config_ck = ms.CheckpointConfig(save_checkpoint_steps=1875, keep_checkpoint_max=10)\n",
    "\n",
    "#directory为网络保存地址\n",
    "ckpoint = ms.ModelCheckpoint(prefix=\"GhostNet\", directory=\"./GhostNet\", config=config_ck)\n",
    "\n",
    "##此处为读取已完成的网络参数\n",
    "# CKPT_1 = './GhostNet/GhostNet-600_24.ckpt'\n",
    "# param_dict = load_checkpoint(CKPT_1)\n",
    "# load_param_into_net(net, param_dict)\n",
    "# load_param_into_net(net_opt, param_dict)\n",
    "\n",
    "model = ms.Model(net, loss_fn=net_loss, optimizer=net_opt, metrics={'accuracy'})\n",
    "\n",
    "import datetime\n",
    "today = datetime.datetime.today().isoformat()\n",
    "t1 = time.clock()\n",
    "file_name = './log' + today +'.txt'\n",
    "\n",
    "with open(file_name , 'w+', buffering = 1, encoding = 'utf-8') as f:\n",
    "    model.train(epochs_num, dataset_train, callbacks=[ckpoint, Mylossmonitor(f,0.01, 1875)])\n",
    "\n",
    "    acc = model.eval(dataset_eval)\n",
    "\n",
    "    f.write(str(epochs_num)+\" : \" + str(acc) + '\\n')\n",
    "    t2 = time.clock()\n",
    "    f.write(f\"cost time: {(t2-t1)*1000} ms\\n\")\n",
    "    f.close()\n",
    "\n",
    "print(\"{}\".format(acc))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.9.6 64-bit",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.6"
  },
  "vscode": {
   "interpreter": {
    "hash": "865d8b2eb28e274047ba64063dfb6a2aabf0dfec4905d304d7a76618dae6fdd4"
   }
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "state": {},
    "version_major": 2,
    "version_minor": 0
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}