Skip to content

Commit

Permalink
Minor fixes for 15-2
Browse files Browse the repository at this point in the history
  • Loading branch information
Atcold committed Nov 5, 2020
1 parent c243b9c commit 0865005
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions docs/en/week15/15-2.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ $$F_\beta(y)\dot{=}-\frac{1}{\beta} \log \frac{1}{|\mathcal{Z}|}{\int}_\mathcal{
where $\beta=(k_B T)^{-1}$ is the inverse temperature, consisting of the Boltzmann constant multiplied by the temperature. If the temperature is very high, $\beta$ is going to be extremely small and if the temperature is cold, then $\beta\rightarrow \infty$.
**Simple discrete approximation:**
$$\tilde{F}_\beta(y)=-\frac{1}{\beta} \log \frac{1}{|\mathcal{Z}|}\underset{z\in\mathcal{Z}}{\sum} \exp[{-\beta}E(y,z)]\Delta z$$
Here, we define $-\frac{1}{\beta} \log \frac{1}{|\mathcal{Z}|}\underset{z\in\mathcal{Z}}{\sum} \exp[{-\beta}E(y,z)]$ to be the $\underset{z}{\text{softmin}}_\beta[E(y,z)]$, such that the relaxation of the zero temperature limit free energy becomes the *actual*-softmin.
Here, we define $-\frac{1}{\beta} \log \frac{1}{|\mathcal{Z}|}\underset{z\in\mathcal{Z}}{\sum} \exp[{-\beta}E(y,z)]$ to be the $\smash{\underset{z}{\text{softmin}}}_\beta[E(y,z)]$, such that the relaxation of the zero temperature limit free energy becomes the *actual*-softmin.
**Examples:**
We will now revisit examples from the previous practicum and see the effects from applying the relaxed version.

Expand Down Expand Up @@ -115,7 +115,7 @@ Objective - Finding a well behaved energy function
A loss functional, minimized during learning, is used to measure the quality of the available energy functions. In simple terms, loss functional is a scalar function that tells us how good our energy function is. A distinction should be made between the energy function, which is minimized by the inference process, and the loss functional (introduced in Section 2), which is minimized by the learning process.


$$\mathcal{L}(F(.),Y) = \frac{1}{N} \sum_{n=1}{N} l(F(.),y^{(n)}) \in R$$
$$\mathcal{L}(F(.),Y) = \frac{1}{N} \sum_{n=1}^{N} l(F(.),y^{(n)}) \in \R$$


$\mathcal{L}$ is the loss function of the whole dataset that can be expressed as the average of these per sample loss function functionals.
Expand Down

0 comments on commit 0865005

Please sign in to comment.