青青草原综合久久大伊人导航_色综合久久天天综合_日日噜噜夜夜狠狠久久丁香五月_热久久这里只有精品

O(1) 的小樂

Job Hunting

公告

記錄我的生活和工作。。。
<2010年10月>
262728293012
3456789
10111213141516
17181920212223
24252627282930
31123456

統(tǒng)計

  • 隨筆 - 182
  • 文章 - 1
  • 評論 - 41
  • 引用 - 0

留言簿(10)

隨筆分類(70)

隨筆檔案(182)

文章檔案(1)

如影隨形

搜索

  •  

最新隨筆

最新評論

閱讀排行榜

評論排行榜

Expectation-maximization algorithm EM算法

     In statistics, an expectation-maximization (EM) algorithm is a method for finding maximum likelihood ormaximum a posteriori (MAP) estimates of parameters in statistical models, where the model depends on unobserved latent variables. EM is an iterative method which alternates between performing an expectation (E) step, which computes the expectation of the log-likelihood evaluated using the current estimate for the latent variables, and a maximization (M) step, which computes parameters maximizing the expected log-likelihood found on the E step. These parameter-estimates are then used to determine the distribution of the latent variables in the next E step.

 

  EM算法可用于很多問題的框架,其中需要估計一組描述概率分布的參數(shù)\boldsymbol\theta,只給定了由此產(chǎn)生的全部數(shù)據(jù)中能觀察到的一部分!

  EM算法是一種迭代算法,它由基本的兩個步驟組成:

  E step:估計期望步驟

  使用對隱變量的現(xiàn)有估計來計算log極大似然

  M step: 最大化期望步驟

  計算一個對隱變量更好的估計,使其最大化log似然函數(shù)對隱變量Y的期望。用新計算的隱變量參數(shù)代替之前的對隱變量的估計,進行下一步的迭代!

 

 

觀測數(shù)據(jù):觀測到的隨機變量X的IID樣本:

image

缺失數(shù)據(jù):未觀測到的隱含變量(隱變量)Y的值:

image

完整數(shù)據(jù): 包含觀測到的隨機變量X和未觀測到的隨機變量Y的數(shù)據(jù),Z=(X,Y)

 

似然函數(shù):(似然函數(shù)的幾種寫法)

JL})D_HBNI489~H}GCRMWVJ

log似然函數(shù)為:

image

E step:用對隱變量的現(xiàn)有估計\boldsymbol\theta^{(t)}計算隱變量Y的期望

  image

其中需要用到貝葉斯公式:

image 

M step:最大化期望,獲得對隱變量更好的估計

image

 

維基中的表述是這樣子:

Given a statistical model consisting of a set \mathbf{X} of observed data, a set of unobserved latent data or missing values Y, and a vector of unknown parameters \boldsymbol\theta, along with a likelihood function L(\boldsymbol\theta; \mathbf{X}, \mathbf{Z}) = p(\mathbf{X}, \mathbf{Z}|\boldsymbol\theta), the maximum likelihood estimate (MLE) of the unknown parameters is determined by the marginal likelihood of the observed data 

       CR%M2I[QD88[N5$3(H))%ZR

However, this quantity is often intractable.

The EM algorithm seeks to find the MLE of the marginal likelihood by iteratively applying the following two steps:

Expectation step (E-step): Calculate the expected value of the log likelihood function, with respect to the conditional distribution of Y given \mathbf{X} under the current estimate of the parameters \boldsymbol\theta^{(t)}:

       A7DFNWMY)KAI]T5)_OMKRUD

Maximization step (M-step): Find the parameter that maximizes this quantity:
\boldsymbol\theta^{(t+1)} = \underset{\boldsymbol\theta} \operatorname{arg\,max} \ Q(\boldsymbol\theta|\boldsymbol\theta^{(t)}) \,

Note that in typical models to which EM is applied:

  1. The observed data points \mathbf{X} may be discrete (taking one of a fixed number of values, or taking values that must be integers) or continuous (taking a continuous range of real numbers, possibly infinite). There may in fact be a vector of observations associated with each data point.
  2. The missing values (aka latent variables) Y are discrete, drawn from a fixed number of values, and there is one latent variable per observed data point.
  3. The parameters are continuous, and are of two kinds: Parameters that are associated with all data points, and parameters associated with a particular value of a latent variable (i.e. associated with all data points whose corresponding latent variable has a particular value).

However, it is possible to apply EM to other sorts of models.

The motivation is as follows. If we know the value of the parameters \boldsymbol\theta, we can usually find the value of the latent variables Y by maximizing the log-likelihood over all possible values of Y, either simply by iterating over Y or through an algorithm such as the Viterbi algorithm for hidden Markov models. Conversely, if we know the value of the latent variables Y, we can find an estimate of the parameters \boldsymbol\theta fairly easily, typically by simply grouping the observed data points according to the value of the associated latent variable and averaging the values, or some function of the values, of the points in each group. This suggests an iterative algorithm, in the case where both \boldsymbol\theta and Y are unknown:

  1. First, initialize the parameters \boldsymbol\theta to some random values.
  2. Compute the best value for Y given these parameter values.
  3. Then, use the just-computed values of Y to compute a better estimate for the parameters \boldsymbol\theta. Parameters associated with a particular value of Y will use only those data points whose associated latent variable has that value.
  4. Finally, iterate until convergence.

The algorithm as just described will in fact work, and is commonly called hard EM. The K-means algorithm is an example of this class of algorithms.

However, we can do somewhat better by, rather than making a hard choice for Y given the current parameter values and averaging only over the set of data points associated with a particular value of Y, instead determining the probability of each possible value of Y for each data point, and then using the probabilities associated with a particular value of Y to compute a weighted average over the entire set of data points. The resulting algorithm is commonly called soft EM, and is the type of algorithm normally associated with EM. The counts used to compute these weighted averages are called soft counts (as opposed to the hard counts used in a hard-EM-type algorithm such as K-means). The probabilities computed for Y areposterior probabilities and are what is computed in the E-step. The soft counts used to compute new parameter values are what is computed in the M-step.

總結(jié):

EM is frequently used for data clustering in machine learning and computer vision.

EM會收斂到局部極致,但不能保證收斂到全局最優(yōu)。

EM對初值比較敏感,通常需要一個好的,快速的初始化過程。

 

這是我的Machine Learning課程,先總結(jié)到這里, 下面的工作是做一個GM_EM的總結(jié),多維高斯密度估計!

posted on 2010-10-20 14:44 Sosi 閱讀(2524) 評論(0)  編輯 收藏 引用 所屬分類: Courses

統(tǒng)計系統(tǒng)
青青草原综合久久大伊人导航_色综合久久天天综合_日日噜噜夜夜狠狠久久丁香五月_热久久这里只有精品
  • <ins id="pjuwb"></ins>
    <blockquote id="pjuwb"><pre id="pjuwb"></pre></blockquote>
    <noscript id="pjuwb"></noscript>
          <sup id="pjuwb"><pre id="pjuwb"></pre></sup>
            <dd id="pjuwb"></dd>
            <abbr id="pjuwb"></abbr>
            久久精品人人做人人爽| 99精品视频免费全部在线| 亚洲欧美久久久| 亚洲一区二区三区成人在线视频精品| 欧美成人免费全部观看天天性色| 在线观看免费视频综合| 麻豆9191精品国产| 欧美不卡高清| 一本色道久久88综合日韩精品 | 国产日韩欧美一区| 久久综合婷婷| 欧美国产日韩一区| 欧美一级视频一区二区| 久久久亚洲成人| 一区二区三区波多野结衣在线观看| 99视频国产精品免费观看| 国产欧美 在线欧美| 蜜臀av国产精品久久久久| 欧美成人免费网站| 欧美一区二区三区喷汁尤物| 久久综合给合| 亚洲综合另类| 卡通动漫国产精品| 亚洲一区三区视频在线观看| 欧美专区亚洲专区| 亚洲无限av看| 久久五月天婷婷| 欧美一区二区| 欧美大成色www永久网站婷| 西西裸体人体做爰大胆久久久| 久久婷婷蜜乳一本欲蜜臀| 宅男精品视频| 欧美.www| 免费不卡视频| 国产精品尤物| 99www免费人成精品| 激情五月婷婷综合| 亚洲午夜羞羞片| 99国产精品99久久久久久| 欧美一区二区三区在线免费观看| 日韩一区二区免费看| 久久免费视频观看| 欧美在线国产| 欧美色精品天天在线观看视频| 欧美成人激情视频| 伊人久久综合97精品| 亚洲午夜一级| 亚洲欧美网站| 国产精品av久久久久久麻豆网| 亚洲国产精品激情在线观看| 国产精品久久久久久模特| 亚洲精品一二三区| 亚洲美女91| 美女精品自拍一二三四| 久久蜜桃精品| 国产一区日韩二区欧美三区| 亚洲一级二级在线| 亚洲欧美欧美一区二区三区| 欧美日韩大片一区二区三区| 亚洲高清在线观看| 亚洲人久久久| 欧美精品久久久久久久久老牛影院| 欧美激情精品久久久久久免费印度 | 久久国产一区二区| 久久久999| 激情av一区| 久久男女视频| 最新国产成人在线观看| 亚洲黄网站在线观看| 麻豆精品一区二区综合av| 欧美gay视频| 亚洲黄色高清| 欧美日韩亚洲高清| 亚洲视频日本| 久久久一二三| 亚洲精品国产品国语在线app| 女人天堂亚洲aⅴ在线观看| 欧美激情第二页| 亚洲视频免费观看| 国产精品区一区二区三| 午夜视频一区二区| 久久亚洲私人国产精品va媚药| 在线观看日韩精品| 欧美日韩成人在线观看| 亚洲一区激情| 裸体丰满少妇做受久久99精品 | 国产婷婷精品| 久久天天狠狠| 亚洲精品美女久久7777777| 亚洲天堂免费观看| 狠狠色综合播放一区二区| 欧美/亚洲一区| 亚洲无线一线二线三线区别av| 久久精品国产久精国产思思| 在线看日韩av| 欧美午夜电影网| 久久午夜电影| 亚洲午夜精品视频| 欧美福利小视频| 午夜在线a亚洲v天堂网2018| 激情亚洲成人| 国产精品福利片| 美女福利精品视频| 午夜精品久久久久久久99水蜜桃 | 亚洲乱码国产乱码精品精天堂| 国产精品分类| 欧美激情影音先锋| 欧美在线免费一级片| 99伊人成综合| 欧美国产第二页| 久久久欧美精品| 亚洲一区在线视频| 亚洲欧洲日产国产网站| 国产欧美精品| 欧美色视频在线| 卡通动漫国产精品| 欧美影院在线| 亚洲综合欧美日韩| 亚洲免费观看高清完整版在线观看熊| 欧美中文在线免费| 亚洲一区二区三区四区视频| 亚洲福利久久| 国语自产偷拍精品视频偷| 欧美午夜不卡在线观看免费| 欧美国产精品专区| 另类人畜视频在线| 久久成人综合视频| 亚洲欧美日韩在线观看a三区| 亚洲精品欧美一区二区三区| 免费不卡在线观看av| 久久久五月天| 久久久噜噜噜| 久久久久成人精品| 欧美自拍丝袜亚洲| 亚洲欧美日韩国产一区| 亚洲一区二区三区色| 99精品久久免费看蜜臀剧情介绍| 校园春色国产精品| 亚洲最新视频在线播放| 精品成人一区二区三区| 免费看的黄色欧美网站| 亚洲免费一级电影| 精品1区2区3区4区| 国产精品永久免费观看| 国产精品久久久久天堂| 欧美日韩中文字幕| 欧美午夜精品理论片a级按摩| 欧美日韩一区高清| 欧美视频日韩| 国产精品久久久久久久午夜片| 国产精品久久久久久久电影| 国产精品久久久一区二区三区| 国产精品爱啪在线线免费观看| 国产精品va在线| 国产精品影片在线观看| 国产欧美日韩在线视频| 国产亚洲精品久久久| 在线精品国精品国产尤物884a| 亚洲国产黄色片| 99在线观看免费视频精品观看| 亚洲午夜精品久久| 亚洲欧洲精品一区二区三区不卡 | 久久亚洲欧洲| 亚洲高清网站| 亚洲无限av看| 欧美在线日韩在线| 欧美福利网址| 欧美视频一区在线观看| 国产在线精品二区| 亚洲日本va午夜在线影院| 一本色道久久综合精品竹菊| 午夜影院日韩| 欧美激情视频网站| 一区二区三区成人| 久久久久久久波多野高潮日日| 欧美成人精品h版在线观看| 欧美午夜一区二区| 在线电影院国产精品| 一本一本久久a久久精品综合麻豆| 性色一区二区三区| 欧美大片一区二区三区| 一区二区三区 在线观看视频 | 亚洲国产视频a| 亚洲女女做受ⅹxx高潮| 米奇777在线欧美播放| 国产精品毛片大码女人| 亚洲大胆在线| 欧美在线观看你懂的| 亚洲精品女人| 亚洲综合色丁香婷婷六月图片| 欧美电影在线观看完整版| 国产一区二区三区久久久久久久久| 99国产精品| 欧美成人自拍视频| 久久福利毛片| 国产精品色在线| 在线亚洲观看| 亚洲第一页中文字幕| 久久久伊人欧美| 国语自产精品视频在线看8查询8|