青青草原综合久久大伊人导航_色综合久久天天综合_日日噜噜夜夜狠狠久久丁香五月_热久久这里只有精品

O(1) 的小樂

Job Hunting

公告

記錄我的生活和工作。。。
<2010年10月>
262728293012
3456789
10111213141516
17181920212223
24252627282930
31123456

統(tǒng)計

  • 隨筆 - 182
  • 文章 - 1
  • 評論 - 41
  • 引用 - 0

留言簿(10)

隨筆分類(70)

隨筆檔案(182)

文章檔案(1)

如影隨形

搜索

  •  

最新隨筆

最新評論

閱讀排行榜

評論排行榜

Expectation-maximization algorithm EM算法

     In statistics, an expectation-maximization (EM) algorithm is a method for finding maximum likelihood ormaximum a posteriori (MAP) estimates of parameters in statistical models, where the model depends on unobserved latent variables. EM is an iterative method which alternates between performing an expectation (E) step, which computes the expectation of the log-likelihood evaluated using the current estimate for the latent variables, and a maximization (M) step, which computes parameters maximizing the expected log-likelihood found on the E step. These parameter-estimates are then used to determine the distribution of the latent variables in the next E step.

 

  EM算法可用于很多問題的框架,其中需要估計一組描述概率分布的參數(shù)\boldsymbol\theta,只給定了由此產(chǎn)生的全部數(shù)據(jù)中能觀察到的一部分!

  EM算法是一種迭代算法,它由基本的兩個步驟組成:

  E step:估計期望步驟

  使用對隱變量的現(xiàn)有估計來計算log極大似然

  M step: 最大化期望步驟

  計算一個對隱變量更好的估計,使其最大化log似然函數(shù)對隱變量Y的期望。用新計算的隱變量參數(shù)代替之前的對隱變量的估計,進行下一步的迭代!

 

 

觀測數(shù)據(jù):觀測到的隨機變量X的IID樣本:

image

缺失數(shù)據(jù):未觀測到的隱含變量(隱變量)Y的值:

image

完整數(shù)據(jù): 包含觀測到的隨機變量X和未觀測到的隨機變量Y的數(shù)據(jù),Z=(X,Y)

 

似然函數(shù):(似然函數(shù)的幾種寫法)

JL})D_HBNI489~H}GCRMWVJ

log似然函數(shù)為:

image

E step:用對隱變量的現(xiàn)有估計\boldsymbol\theta^{(t)}計算隱變量Y的期望

  image

其中需要用到貝葉斯公式:

image 

M step:最大化期望,獲得對隱變量更好的估計

image

 

維基中的表述是這樣子:

Given a statistical model consisting of a set \mathbf{X} of observed data, a set of unobserved latent data or missing values Y, and a vector of unknown parameters \boldsymbol\theta, along with a likelihood function L(\boldsymbol\theta; \mathbf{X}, \mathbf{Z}) = p(\mathbf{X}, \mathbf{Z}|\boldsymbol\theta), the maximum likelihood estimate (MLE) of the unknown parameters is determined by the marginal likelihood of the observed data 

       CR%M2I[QD88[N5$3(H))%ZR

However, this quantity is often intractable.

The EM algorithm seeks to find the MLE of the marginal likelihood by iteratively applying the following two steps:

Expectation step (E-step): Calculate the expected value of the log likelihood function, with respect to the conditional distribution of Y given \mathbf{X} under the current estimate of the parameters \boldsymbol\theta^{(t)}:

       A7DFNWMY)KAI]T5)_OMKRUD

Maximization step (M-step): Find the parameter that maximizes this quantity:
\boldsymbol\theta^{(t+1)} = \underset{\boldsymbol\theta} \operatorname{arg\,max} \ Q(\boldsymbol\theta|\boldsymbol\theta^{(t)}) \,

Note that in typical models to which EM is applied:

  1. The observed data points \mathbf{X} may be discrete (taking one of a fixed number of values, or taking values that must be integers) or continuous (taking a continuous range of real numbers, possibly infinite). There may in fact be a vector of observations associated with each data point.
  2. The missing values (aka latent variables) Y are discrete, drawn from a fixed number of values, and there is one latent variable per observed data point.
  3. The parameters are continuous, and are of two kinds: Parameters that are associated with all data points, and parameters associated with a particular value of a latent variable (i.e. associated with all data points whose corresponding latent variable has a particular value).

However, it is possible to apply EM to other sorts of models.

The motivation is as follows. If we know the value of the parameters \boldsymbol\theta, we can usually find the value of the latent variables Y by maximizing the log-likelihood over all possible values of Y, either simply by iterating over Y or through an algorithm such as the Viterbi algorithm for hidden Markov models. Conversely, if we know the value of the latent variables Y, we can find an estimate of the parameters \boldsymbol\theta fairly easily, typically by simply grouping the observed data points according to the value of the associated latent variable and averaging the values, or some function of the values, of the points in each group. This suggests an iterative algorithm, in the case where both \boldsymbol\theta and Y are unknown:

  1. First, initialize the parameters \boldsymbol\theta to some random values.
  2. Compute the best value for Y given these parameter values.
  3. Then, use the just-computed values of Y to compute a better estimate for the parameters \boldsymbol\theta. Parameters associated with a particular value of Y will use only those data points whose associated latent variable has that value.
  4. Finally, iterate until convergence.

The algorithm as just described will in fact work, and is commonly called hard EM. The K-means algorithm is an example of this class of algorithms.

However, we can do somewhat better by, rather than making a hard choice for Y given the current parameter values and averaging only over the set of data points associated with a particular value of Y, instead determining the probability of each possible value of Y for each data point, and then using the probabilities associated with a particular value of Y to compute a weighted average over the entire set of data points. The resulting algorithm is commonly called soft EM, and is the type of algorithm normally associated with EM. The counts used to compute these weighted averages are called soft counts (as opposed to the hard counts used in a hard-EM-type algorithm such as K-means). The probabilities computed for Y areposterior probabilities and are what is computed in the E-step. The soft counts used to compute new parameter values are what is computed in the M-step.

總結(jié):

EM is frequently used for data clustering in machine learning and computer vision.

EM會收斂到局部極致,但不能保證收斂到全局最優(yōu)。

EM對初值比較敏感,通常需要一個好的,快速的初始化過程。

 

這是我的Machine Learning課程,先總結(jié)到這里, 下面的工作是做一個GM_EM的總結(jié),多維高斯密度估計!

posted on 2010-10-20 14:44 Sosi 閱讀(2524) 評論(0)  編輯 收藏 引用 所屬分類: Courses

統(tǒng)計系統(tǒng)
青青草原综合久久大伊人导航_色综合久久天天综合_日日噜噜夜夜狠狠久久丁香五月_热久久这里只有精品
  • <ins id="pjuwb"></ins>
    <blockquote id="pjuwb"><pre id="pjuwb"></pre></blockquote>
    <noscript id="pjuwb"></noscript>
          <sup id="pjuwb"><pre id="pjuwb"></pre></sup>
            <dd id="pjuwb"></dd>
            <abbr id="pjuwb"></abbr>
            久久久久88色偷偷免费| 亚洲在线播放电影| 久久久久在线| 一区二区亚洲欧洲国产日韩| 久久影视精品| 久久亚洲精品一区| 日韩午夜三级在线| 一本一本久久a久久精品综合麻豆| 欧美在线精品一区| 亚洲视频在线观看三级| 欧美三级视频在线播放| 午夜欧美不卡精品aaaaa| 久久成人羞羞网站| 亚洲日本理论电影| 亚洲一区在线免费| 1024日韩| 亚洲少妇在线| 精品成人一区二区三区| 亚洲精品欧美一区二区三区| 国产精品免费电影| 女人香蕉久久**毛片精品| 欧美韩国一区| 久久国产福利| 欧美紧缚bdsm在线视频| 欧美影院成年免费版| 蜜臀91精品一区二区三区| 亚洲午夜女主播在线直播| 欧美亚洲网站| 中文av字幕一区| 久久精品理论片| 中文高清一区| 久久亚洲精品网站| 午夜精品视频| 欧美日韩国产成人在线| 久久成人人人人精品欧| 欧美紧缚bdsm在线视频| 久久久亚洲精品一区二区三区 | 欧美一区二区三区免费在线看| 久久九九精品99国产精品| 久久久久久久91| 午夜天堂精品久久久久 | av成人免费在线| 性久久久久久久久| 亚洲一线二线三线久久久| 男人天堂欧美日韩| 久久婷婷影院| 国产精品美女午夜av| 亚洲精品视频中文字幕| 亚洲电影网站| 久久av资源网站| 欧美一级黄色网| 欧美午夜不卡| 亚洲精品系列| 亚洲三级免费电影| 农夫在线精品视频免费观看| 久久亚洲精品视频| 国内精品99| 亚洲欧美综合网| 性伦欧美刺激片在线观看| 欧美日韩中文另类| aa级大片欧美三级| 这里只有精品视频在线| 欧美日韩国产成人在线91| 亚洲日本va午夜在线电影| 亚洲国产中文字幕在线观看| 久久性色av| 欧美大片第1页| 亚洲日本欧美在线| 欧美激情一区二区三区不卡| 亚洲精品国产品国语在线app| 欧美精品二区三区四区免费看视频| 亚洲精品久久久久中文字幕欢迎你| 亚洲黄色有码视频| 亚洲精品美女久久7777777| 久久久久国色av免费看影院 | 欧美日本簧片| 亚洲精华国产欧美| 在线午夜精品| 国产精品美女久久久免费| 中文在线一区| 久久gogo国模啪啪人体图| 国产一区二区成人久久免费影院| 亚洲电影成人| 亚洲最新色图| 国产精品热久久久久夜色精品三区 | 久久九九精品99国产精品| 久久男女视频| 亚洲欧洲免费视频| 欧美视频一区二区| 亚洲欧美综合国产精品一区| 免费成年人欧美视频| 日韩视频精品在线| 国产乱理伦片在线观看夜一区| 美女视频黄免费的久久| 亚洲三级影片| 国产精品羞羞答答| 久久成人免费网| 91久久精品网| 久久精品综合网| 日韩午夜激情| 国产一二精品视频| 欧美久久久久久蜜桃| 欧美一区二区三区四区在线观看地址 | 亚洲欧洲日本一区二区三区| 欧美极品欧美精品欧美视频| 亚洲欧美不卡| 亚洲国产日韩欧美| 久久久久久有精品国产| 一区二区三区日韩| 在线欧美视频| 国产欧美日韩精品丝袜高跟鞋| 亚洲精品在线免费观看视频| 久久se精品一区二区| 亚洲美女91| 1204国产成人精品视频| 国产精品久久久久久影视| 你懂的网址国产 欧美| 午夜综合激情| 一区二区高清在线| 亚洲国产精品视频一区| 久久精品日产第一区二区三区| 国产精品无码永久免费888| 美日韩精品免费| 欧美与黑人午夜性猛交久久久| 性欧美大战久久久久久久久| 亚洲精品少妇30p| 亚洲电影免费| 经典三级久久| 国产午夜亚洲精品理论片色戒| 一本大道久久a久久综合婷婷 | 国产亚洲精品aa午夜观看| 免播放器亚洲| 久久精品视频免费| 欧美伊久线香蕉线新在线| 亚洲一区尤物| 亚洲欧美经典视频| 亚洲色图制服丝袜| 一二三区精品| 一本久久a久久免费精品不卡| 亚洲欧美视频一区| 亚洲视频一区二区免费在线观看| 欧美日韩视频在线一区二区观看视频 | 国产欧美二区| 国产精品视频免费观看www| 国产精品劲爆视频| 国产精品第2页| 国产精品爽爽爽| 国产美女精品免费电影| 国产欧美精品一区二区三区介绍| 久久国产色av| 久久久久久色| 嫩草国产精品入口| 欧美麻豆久久久久久中文| 欧美精选一区| 国产精品国产一区二区| 国产精品一区一区三区| 欧美性猛交xxxx乱大交蜜桃 | 欧美电影打屁股sp| 欧美国产一区视频在线观看| 亚洲国产精品成人综合色在线婷婷| 99成人精品| 亚洲综合国产| 久久久久一区二区| 亚洲国产清纯| 亚洲一区二区视频| 久久久www免费人成黑人精品 | 亚洲人成网站在线播| 夜夜狂射影院欧美极品| 亚洲欧美日韩综合国产aⅴ| 久久精品女人天堂| 欧美激情在线有限公司| 国产精品任我爽爆在线播放 | 性欧美8khd高清极品| 午夜精品久久久久| 蜜桃av一区二区| 国产精品福利影院| 亚洲成人资源| 亚洲欧美日韩区| 免费看亚洲片| 亚洲视频免费看| 久久综合图片| 国产精品嫩草99a| 亚洲欧洲日本mm| 久久成年人视频| 亚洲欧洲三级| 欧美一级视频免费在线观看| 欧美成熟视频| 国产伪娘ts一区 | 久久久欧美一区二区| 欧美日韩国产a| 在线观看成人av| 午夜一区二区三区不卡视频| 欧美1区3d| 欧美亚洲免费| 国产精品高精视频免费| 亚洲美女视频在线观看| 巨胸喷奶水www久久久免费动漫| 欧美一区二区三区免费看| 亚洲国产成人不卡| 午夜精品视频在线观看|