Essay
Version 9
Preserved in Portico This version is not peer-reviewed
Tutorial on EM Algorithm
Version 1
: Received: 17 February 2018 / Approved: 20 February 2018 / Online: 20 February 2018 (15:37:19 CET)
Version 2 : Received: 22 February 2018 / Approved: 22 February 2018 / Online: 22 February 2018 (16:28:05 CET)
Version 3 : Received: 27 February 2018 / Approved: 27 February 2018 / Online: 27 February 2018 (15:06:35 CET)
Version 4 : Received: 8 September 2020 / Approved: 14 September 2020 / Online: 14 September 2020 (07:28:46 CEST)
Version 5 : Received: 22 September 2020 / Approved: 23 September 2020 / Online: 23 September 2020 (04:35:28 CEST)
Version 6 : Received: 11 October 2020 / Approved: 13 October 2020 / Online: 13 October 2020 (14:24:31 CEST)
Version 7 : Received: 19 October 2020 / Approved: 19 October 2020 / Online: 19 October 2020 (15:59:32 CEST)
Version 8 : Received: 22 October 2020 / Approved: 22 October 2020 / Online: 22 October 2020 (10:33:11 CEST)
Version 9 : Received: 21 November 2020 / Approved: 23 November 2020 / Online: 23 November 2020 (14:25:54 CET)
Version 2 : Received: 22 February 2018 / Approved: 22 February 2018 / Online: 22 February 2018 (16:28:05 CET)
Version 3 : Received: 27 February 2018 / Approved: 27 February 2018 / Online: 27 February 2018 (15:06:35 CET)
Version 4 : Received: 8 September 2020 / Approved: 14 September 2020 / Online: 14 September 2020 (07:28:46 CEST)
Version 5 : Received: 22 September 2020 / Approved: 23 September 2020 / Online: 23 September 2020 (04:35:28 CEST)
Version 6 : Received: 11 October 2020 / Approved: 13 October 2020 / Online: 13 October 2020 (14:24:31 CEST)
Version 7 : Received: 19 October 2020 / Approved: 19 October 2020 / Online: 19 October 2020 (15:59:32 CEST)
Version 8 : Received: 22 October 2020 / Approved: 22 October 2020 / Online: 22 October 2020 (10:33:11 CEST)
Version 9 : Received: 21 November 2020 / Approved: 23 November 2020 / Online: 23 November 2020 (14:25:54 CET)
How to cite: Nguyen, L. Tutorial on EM Algorithm. Preprints 2018, 2018020131. https://doi.org/10.20944/preprints201802.0131.v9 Nguyen, L. Tutorial on EM Algorithm. Preprints 2018, 2018020131. https://doi.org/10.20944/preprints201802.0131.v9
Abstract
Maximum likelihood estimation (MLE) is a popular method for parameter estimation in both applied probability and statistics but MLE cannot solve the problem of incomplete data or hidden data because it is impossible to maximize likelihood function from hidden data. Expectation maximum (EM) algorithm is a powerful mathematical tool for solving this problem if there is a relationship between hidden data and observed data. Such hinting relationship is specified by a mapping from hidden data to observed data or by a joint probability between hidden data and observed data. In other words, the relationship helps us know hidden data by surveying observed data. The essential ideology of EM is to maximize the expectation of likelihood function over observed data based on the hinting relationship instead of maximizing directly the likelihood function of hidden data. Pioneers in EM algorithm proved its convergence. As a result, EM algorithm produces parameter estimators as well as MLE does. This tutorial aims to provide explanations of EM algorithm in order to help researchers comprehend it. Moreover some improvements of EM algorithm are also proposed in the tutorial such as combination of EM and third-order convergence Newton-Raphson process, combination of EM and gradient descent method, and combination of EM and particle swarm optimization (PSO) algorithm.
Keywords
expectation maximum; EM; generalized expectation maximum; GEM; EM convergence
Subject
Computer Science and Mathematics, Probability and Statistics
Copyright: This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Comments (1)
We encourage comments and feedback from a broad range of readers. See criteria for comments and our Diversity statement.
Leave a public commentSend a private comment to the author(s)
* All users must log in before leaving a comment
Commenter: Loc Nguyen
Commenter's Conflict of Interests: Author