<?xml version="1.0" encoding="utf-8"?>
<export-example>
  <doc>
    <id>6123</id>
    <completedYear/>
    <publishedYear/>
    <thesisYearAccepted/>
    <language>eng</language>
    <pageFirst/>
    <pageLast/>
    <pageNumber/>
    <edition/>
    <issue/>
    <volume/>
    <type>reportzib</type>
    <publisherName/>
    <publisherPlace/>
    <creatingCorporation/>
    <contributingCorporation/>
    <belongsToBibliography>0</belongsToBibliography>
    <completedDate>--</completedDate>
    <publishedDate>2016-11-30</publishedDate>
    <thesisDateAccepted>--</thesisDateAccepted>
    <title language="eng">Empirical Bayes Methods, Reference Priors, Cross Entropy and the EM Algorithm</title>
    <abstract language="eng">When estimating a probability density within the empirical Bayes framework, the non-parametric maximum likelihood estimate (NPMLE) usually tends to overfit the data. This issue is usually taken care of by regularization - a penalization term is subtracted from the marginal log-likelihood before the maximization step, so that the estimate favors smooth solutions, resulting in the so-called maximum penalized likelihood estimation (MPLE).&#13;
The majority of penalizations currently in use are rather arbitrary brute-force solutions, which lack invariance under transformation of the parameters(reparametrization) and measurements.&#13;
This contradicts the principle that, if the underlying model&#13;
has several equivalent formulations, the methods of inductive inference should lead to consistent results. Motivated by this principle and using an information-theoretic point of view, we suggest an entropy-based penalization term that guarantees this kind of invariance. The resulting density estimate can be seen as a generalization of reference priors. Using the reference prior as a hyperprior, on the other hand, is argued to be a poor choice for regularization. We also present an insightful connection between the NPMLE, the cross entropy&#13;
and the principle of minimum discrimination information suggesting another method of inference that contains the doubly-smoothed maximum likelihood estimation as a special case.</abstract>
    <identifier type="issn">1438-0064</identifier>
    <identifier type="urn">urn:nbn:de:0297-zib-61230</identifier>
    <author>Ilja Klebanov</author>
    <submitter>Ilja Klebanov</submitter>
    <author>Alexander Sikorski</author>
    <author>Christof Schütte</author>
    <author>Susanna Röblitz</author>
    <series>
      <title>ZIB-Report</title>
      <number>16-56</number>
    </series>
    <subject>
      <language>eng</language>
      <type>uncontrolled</type>
      <value>parameter estimation</value>
    </subject>
    <subject>
      <language>eng</language>
      <type>uncontrolled</type>
      <value>Bayesian inference</value>
    </subject>
    <subject>
      <language>eng</language>
      <type>uncontrolled</type>
      <value>Bayesian hierarchical modeling</value>
    </subject>
    <subject>
      <language>eng</language>
      <type>uncontrolled</type>
      <value>hyperparameter</value>
    </subject>
    <subject>
      <language>eng</language>
      <type>uncontrolled</type>
      <value>hyperprior</value>
    </subject>
    <subject>
      <language>eng</language>
      <type>uncontrolled</type>
      <value>EM algorithm</value>
    </subject>
    <subject>
      <language>eng</language>
      <type>uncontrolled</type>
      <value>NPMLE</value>
    </subject>
    <subject>
      <language>eng</language>
      <type>uncontrolled</type>
      <value>MPLE</value>
    </subject>
    <subject>
      <language>eng</language>
      <type>uncontrolled</type>
      <value>DS-MLE</value>
    </subject>
    <subject>
      <language>eng</language>
      <type>uncontrolled</type>
      <value>principle of maximum entropy</value>
    </subject>
    <subject>
      <language>eng</language>
      <type>uncontrolled</type>
      <value>cross entropy</value>
    </subject>
    <subject>
      <language>eng</language>
      <type>uncontrolled</type>
      <value>minimum discrimination information</value>
    </subject>
    <subject>
      <language>eng</language>
      <type>uncontrolled</type>
      <value>reference prior</value>
    </subject>
    <subject>
      <language>eng</language>
      <type>uncontrolled</type>
      <value>Jeffreys prior</value>
    </subject>
    <collection role="ccs" number="F.">Theory of Computation</collection>
    <collection role="msc" number="62-XX">STATISTICS</collection>
    <collection role="institutes" number="num">Numerical Mathematics</collection>
    <collection role="institutes" number="compsys">Computational Systems Biology</collection>
    <collection role="persons" number="susanna.roeblitz">Röblitz, Susanna</collection>
    <collection role="persons" number="schuette">Schütte, Christof</collection>
    <collection role="projects" number="ECMath-CH6">ECMath-CH6</collection>
    <collection role="persons" number="sikorski">Sikorski, Alexander</collection>
    <file>https://opus4.kobv.de/opus4-zib/files/6123/EmpiricalBayesReferencePriorsCrossEntropyEMAlgorithm.pdf</file>
  </doc>
</export-example>
