MachineLearning/hands-on
1
0
Fork 0
Code Issues Pull Requests Releases Activity

4장 수식 업데이트

Browse Source
This commit is contained in:
Haesun Park
2018-04-24 22:40:27 +09:00
parent f90d5a85b7
commit b4852aff9e
1 changed files with 30 additions and 30 deletions
Download Patch File Download Diff File Expand all files Collapse all files

60
book_equations.ipynb
View File

@@ -178,7 +178,7 @@
"$\n",
"\n",
"\n",
"**Equation 4-6: Gradient vector of the cost function**\n",
"**식 4-6: 비용 함수의 그래디언트 벡터**\n",
"\n",
"$\n",
"\\nabla_{\\mathbf{\\theta}}\\, \\text{MSE}(\\mathbf{\\theta}) =\n",
@@ -192,20 +192,20 @@
"$\n",
"\n",
"\n",
"**Equation 4-7: Gradient Descent step**\n",
"**식 4-7: 경사 하강법의 스텝**\n",
"\n",
"$\n",
"\\mathbf{\\theta}^{(\\text{next step})} = \\mathbf{\\theta} - \\eta \\nabla_{\\mathbf{\\theta}}\\, \\text{MSE}(\\mathbf{\\theta})\n",
"\\mathbf{\\theta}^{(\\text{다음 스텝})}\\,\\,\\, = \\mathbf{\\theta} - \\eta \\nabla_{\\mathbf{\\theta}}\\, \\text{MSE}(\\mathbf{\\theta})\n",
"$\n",
"\n",
"\n",
"$ O(\\frac{1}{\\text{iterations}}) $\n",
"$ O(\\frac{1}{\\epsilon}) $\n",
"\n",
"\n",
"$ \\hat{y} = 0.56 x_1^2 + 0.93 x_1 + 1.78 $\n",
"\n",
"\n",
"$ y = 0.5 x_1^2 + 1.0 x_1 + 2.0 + \\text{Gaussian noise} $\n",
"$ y = 0.5 x_1^2 + 1.0 x_1 + 2.0 + \\text{가우시안 잡음} $\n",
"\n",
"\n",
"$ \\dfrac{(n+d)!}{d!\\,n!} $\n",
@@ -214,28 +214,28 @@
"$ \\alpha \\sum_{i=1}^{n}{\\theta_i^2}$\n",
"\n",
"\n",
"**Equation 4-8: Ridge Regression cost function**\n",
"**식 4-8: 릿지 회귀의 비용 함수**\n",
"\n",
"$\n",
"J(\\mathbf{\\theta}) = \\text{MSE}(\\mathbf{\\theta}) + \\alpha \\dfrac{1}{2}\\sum\\limits_{i=1}^{n}\\theta_i^2\n",
"$\n",
"\n",
"\n",
"**Equation 4-9: Ridge Regression closed-form solution**\n",
"**식 4-9: 릿지 회귀의 정규 방정식**\n",
"\n",
"$\n",
"\\hat{\\mathbf{\\theta}} = (\\mathbf{X}^T \\cdot \\mathbf{X} + \\alpha \\mathbf{A})^{-1} \\cdot \\mathbf{X}^T \\cdot \\mathbf{y}\n",
"$\n",
"\n",
"\n",
"**Equation 4-10: Lasso Regression cost function**\n",
"**식 4-10: 라쏘 회귀의 비용 함수**\n",
"\n",
"$\n",
"J(\\mathbf{\\theta}) = \\text{MSE}(\\mathbf{\\theta}) + \\alpha \\sum\\limits_{i=1}^{n}\\left| \\theta_i \\right|\n",
"$\n",
"\n",
"\n",
"**Equation 4-11: Lasso Regression subgradient vector**\n",
"**식 4-11: 라쏘 회귀의 서브그래디언트 벡터**\n",
"\n",
"$\n",
"g(\\mathbf{\\theta}, J) = \\nabla_{\\mathbf{\\theta}}\\, \\text{MSE}(\\mathbf{\\theta}) + \\alpha\n",
@@ -244,104 +244,104 @@
" \\operatorname{sign}(\\theta_2) \\\\\n",
" \\vdots \\\\\n",
" \\operatorname{sign}(\\theta_n) \\\\\n",
"\\end{pmatrix} \\quad \\text{where } \\operatorname{sign}(\\theta_i) =\n",
"\\end{pmatrix} \\quad \\text{여기서 } \\operatorname{sign}(\\theta_i) =\n",
"\\begin{cases}\n",
"-1 & \\text{if } \\theta_i < 0 \\\\\n",
"0 & \\text{if } \\theta_i = 0 \\\\\n",
"+1 & \\text{if } \\theta_i > 0\n",
"-1 & \\theta_i < 0 \\text{일 때 } \\\\\n",
"0 & \\theta_i = 0 \\text{일 때 } \\\\\n",
"+1 & \\theta_i > 0 \\text{일 때 }\n",
"\\end{cases}\n",
"$\n",
"\n",
"\n",
"**Equation 4-12: Elastic Net cost function**\n",
"**식 4-12: 엘라스틱넷 비용 함수**\n",
"\n",
"$\n",
"J(\\mathbf{\\theta}) = \\text{MSE}(\\mathbf{\\theta}) + r \\alpha \\sum\\limits_{i=1}^{n}\\left| \\theta_i \\right| + \\dfrac{1 - r}{2} \\alpha \\sum\\limits_{i=1}^{n}{\\theta_i^2}\n",
"$\n",
"\n",
"\n",
"**Equation 4-13: Logistic Regression model estimated probability (vectorized form)**\n",
"**식 4-13: 로지스틱 회귀 모델의 확률 추정(벡터 표현식)**\n",
"\n",
"$\n",
"\\hat{p} = h_{\\mathbf{\\theta}}(\\mathbf{x}) = \\sigma(\\mathbf{\\theta}^T \\cdot \\mathbf{x})\n",
"$\n",
"\n",
"\n",
"**Equation 4-14: Logistic function**\n",
"**식 4-14: 로지스틱 함수**\n",
"\n",
"$\n",
"\\sigma(t) = \\dfrac{1}{1 + \\exp(-t)}\n",
"$\n",
"\n",
"\n",
"**Equation 4-15: Logistic Regression model prediction**\n",
"**식 4-15: 로지스틱 회귀 모델 예측**\n",
"\n",
"$\n",
"\\hat{y} =\n",
"\\begin{cases}\n",
" 0 & \\text{if } \\hat{p} < 0.5, \\\\\n",
" 1 & \\text{if } \\hat{p} \\geq 0.5.\n",
" 0 & \\hat{p} < 0.5 \\text{일 때 } \\\\\n",
" 1 & \\hat{p} \\geq 0.5 \\text{일 때 } \n",
"\\end{cases}\n",
"$\n",
"\n",
"\n",
"**Equation 4-16: Cost function of a single training instance**\n",
"**식 4-16: 하나의 훈련 샘플에 대한 비용 함수**\n",
"\n",
"$\n",
"c(\\mathbf{\\theta}) =\n",
"\\begin{cases}\n",
" -\\log(\\hat{p}) & \\text{if } y = 1, \\\\\n",
" -\\log(1 - \\hat{p}) & \\text{if } y = 0.\n",
" -\\log(\\hat{p}) & y = 1 \\text{일 때 } \\\\\n",
" -\\log(1 - \\hat{p}) & y = 0 \\text{일 때 }\n",
"\\end{cases}\n",
"$\n",
"\n",
"\n",
"**Equation 4-17: Logistic Regression cost function (log loss)**\n",
"**식 4-17: 로지스틱 회귀의 비용 함수(로그 손실)**\n",
"\n",
"$\n",
"J(\\mathbf{\\theta}) = -\\dfrac{1}{m} \\sum\\limits_{i=1}^{m}{\\left[ y^{(i)} log\\left(\\hat{p}^{(i)}\\right) + (1 - y^{(i)}) log\\left(1 - \\hat{p}^{(i)}\\right)\\right]}\n",
"$\n",
"\n",
"\n",
"**Equation 4-18: Logistic cost function partial derivatives**\n",
"**식 4-18: 로지스틱 비용 함수의 편도함수**\n",
"\n",
"$\n",
"\\dfrac{\\partial}{\\partial \\theta_j} \\text{J}(\\mathbf{\\theta}) = \\dfrac{1}{m}\\sum\\limits_{i=1}^{m}\\left(\\mathbf{\\sigma(\\theta}^T \\cdot \\mathbf{x}^{(i)}) - y^{(i)}\\right)\\, x_j^{(i)}\n",
"$\n",
"\n",
"\n",
"**Equation 4-19: Softmax score for class k**\n",
"**식 4-19: 클래스 k에 대한 소프트맥스 점수**\n",
"\n",
"$\n",
"s_k(\\mathbf{x}) = ({\\mathbf{\\theta}^{(k)}})^T \\cdot \\mathbf{x}\n",
"$\n",
"\n",
"\n",
"**Equation 4-20: Softmax function**\n",
"**식 4-20: 소프트맥스 함수**\n",
"\n",
"$\n",
"\\hat{p}_k = \\sigma\\left(\\mathbf{s}(\\mathbf{x})\\right)_k = \\dfrac{\\exp\\left(s_k(\\mathbf{x})\\right)}{\\sum\\limits_{j=1}^{K}{\\exp\\left(s_j(\\mathbf{x})\\right)}}\n",
"$\n",
"\n",
"\n",
"**Equation 4-21: Softmax Regression classifier prediction**\n",
"**식 4-21: 소프트맥스 회귀 분류기의 예측**\n",
"\n",
"$\n",
"\\hat{y} = \\underset{k}{\\operatorname{argmax}} \\, \\sigma\\left(\\mathbf{s}(\\mathbf{x})\\right)_k = \\underset{k}{\\operatorname{argmax}} \\, s_k(\\mathbf{x}) = \\underset{k}{\\operatorname{argmax}} \\, \\left( ({\\mathbf{\\theta}^{(k)}})^T \\cdot \\mathbf{x} \\right)\n",
"$\n",
"\n",
"\n",
"**Equation 4-22: Cross entropy cost function**\n",
"**식 4-22: 크로스 엔트로피 비용 함수**\n",
"\n",
"$\n",
"J(\\mathbf{\\Theta}) = - \\dfrac{1}{m}\\sum\\limits_{i=1}^{m}\\sum\\limits_{k=1}^{K}{y_k^{(i)}\\log\\left(\\hat{p}_k^{(i)}\\right)}\n",
"$\n",
"\n",
"**Cross entropy between two discrete probability distributions $p$ and $q$ (page 141):**\n",
"**두 확률 분포 $p$ 와 $q$ 사이의 크로스 엔트로피 (196 페이지):**\n",
"$ H(p, q) = -\\sum\\limits_{x}p(x) \\log q(x) $\n",
"\n",
"\n",
"**Equation 4-23: Cross entropy gradient vector for class k**\n",
"**식 4-23: 클래스 k 에 대한 크로스 엔트로피의 그래디언트 벡터**\n",
"\n",
"$\n",
"\\nabla_{\\mathbf{\\theta}^{(k)}} \\, J(\\mathbf{\\Theta}) = \\dfrac{1}{m} \\sum\\limits_{i=1}^{m}{ \\left ( \\hat{p}^{(i)}_k - y_k^{(i)} \\right ) \\mathbf{x}^{(i)}}\n",