Skip to content

Commit

Permalink
Update 3_neural_networks_tutorial.ipynb
Browse files Browse the repository at this point in the history
  • Loading branch information
syliudf authored Jul 19, 2019
1 parent 39bca9f commit eeefed5
Showing 1 changed file with 35 additions and 27 deletions.
62 changes: 35 additions & 27 deletions chapter1/3_neural_networks_tutorial.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@
},
{
"cell_type": "code",
"execution_count": 2,
"execution_count": 1,
"metadata": {},
"outputs": [
{
Expand Down Expand Up @@ -124,7 +124,7 @@
},
{
"cell_type": "code",
"execution_count": 4,
"execution_count": 2,
"metadata": {},
"outputs": [
{
Expand Down Expand Up @@ -153,15 +153,15 @@
},
{
"cell_type": "code",
"execution_count": 5,
"execution_count": 6,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"tensor([[-0.0204, -0.0268, -0.0829, 0.1420, -0.0192, 0.1848, 0.0723, -0.0393,\n",
" -0.0275, 0.0867]], grad_fn=<ThAddmmBackward>)\n"
"tensor([[ 0.1120, 0.0713, 0.1014, -0.0696, -0.1210, 0.0084, -0.0206, 0.1366,\n",
" -0.0455, -0.0036]], grad_fn=<AddmmBackward>)\n"
]
}
],
Expand All @@ -181,7 +181,7 @@
},
{
"cell_type": "code",
"execution_count": 6,
"execution_count": 7,
"metadata": {},
"outputs": [],
"source": [
Expand Down Expand Up @@ -232,14 +232,14 @@
},
{
"cell_type": "code",
"execution_count": 7,
"execution_count": 8,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"tensor(1.3172, grad_fn=<MseLossBackward>)\n"
"tensor(0.8109, grad_fn=<MseLossBackward>)\n"
]
}
],
Expand All @@ -257,9 +257,8 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Now, if you follow ``loss`` in the backward direction, using its\n",
"``.grad_fn`` attribute, you will see a graph of computations that looks\n",
"like this:\n",
"现在,如果在反向过程中跟随``loss`` , 使用它的\n",
"``.grad_fn`` 属性,将看到如下所示的计算图。\n",
"\n",
"::\n",
"\n",
Expand All @@ -268,19 +267,29 @@
" -> MSELoss\n",
" -> loss\n",
"\n",
"So, when we call ``loss.backward()``, the whole graph is differentiated\n",
"w.r.t. the loss, and all Tensors in the graph that has ``requires_grad=True``\n",
"will have their ``.grad`` Tensor accumulated with the gradient.\n",
"所以,当我们调用 ``loss.backward()``时,整张计算图都会\n",
"根据loss进行微分,而且图中所有设置为``requires_grad=True``的张量\n",
"将会拥有一个随着梯度累积的``.grad`` 张量。\n",
"\n",
"For illustration, let us follow a few steps backward:\n",
"为了说明,让我们向后退几步:\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 9,
"metadata": {},
"outputs": [],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"<MseLossBackward object at 0x7f3b49fe2470>\n",
"<AddmmBackward object at 0x7f3bb05f17f0>\n",
"<AccumulateGrad object at 0x7f3b4a3c34e0>\n"
]
}
],
"source": [
"print(loss.grad_fn) # MSELoss\n",
"print(loss.grad_fn.next_functions[0][0]) # Linear\n",
Expand All @@ -304,7 +313,7 @@
},
{
"cell_type": "code",
"execution_count": 8,
"execution_count": 10,
"metadata": {},
"outputs": [
{
Expand All @@ -314,7 +323,7 @@
"conv1.bias.grad before backward\n",
"tensor([0., 0., 0., 0., 0., 0.])\n",
"conv1.bias.grad after backward\n",
"tensor([ 0.0074, -0.0249, -0.0107, 0.0326, -0.0017, -0.0059])\n"
"tensor([ 0.0051, 0.0042, 0.0026, 0.0152, -0.0040, -0.0036])\n"
]
}
],
Expand Down Expand Up @@ -364,7 +373,7 @@
},
{
"cell_type": "code",
"execution_count": 9,
"execution_count": 11,
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -385,11 +394,10 @@
"cell_type": "markdown",
"metadata": {},
"source": [
".. Note::\n",
".. 注意::\n",
" \n",
" Observe how gradient buffers had to be manually set to zero using\n",
" ``optimizer.zero_grad()``. This is because gradients are accumulated\n",
" as explained in `Backprop`_ section.\n",
" 观察如何使用``optimizer.zero_grad()``手动将梯度缓冲区设置为零。\n",
" 这是因为梯度是按Backprop部分中的说明累积的。\n",
"\n"
]
},
Expand All @@ -403,9 +411,9 @@
],
"metadata": {
"kernelspec": {
"display_name": "Pytorch for Deeplearning",
"display_name": "Python 3",
"language": "python",
"name": "pytorch"
"name": "python3"
},
"language_info": {
"codemirror_mode": {
Expand All @@ -417,7 +425,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.7"
"version": "3.7.3"
}
},
"nbformat": 4,
Expand Down

0 comments on commit eeefed5

Please sign in to comment.