Symbolic Maximum Likelihood Estimation

Although statistical software has long been used for maximum likelihood (ML) estimation, the focus of attention has almost always been on obtaining ML estimates (a numerical problem), rather than on deriving ML estimators (a symbolic problem; cf. Rose and Smith, 2000). mathStatica makes it possible to derive exact symbolic ML estimators from first principles with a computer algebra system.

For instance, consider the following simple problem: let [Graphics:Images/symMLE_gr_1.gif] denote a random sample of size n collected on [Graphics:Images/symMLE_gr_2.gif] where parameter [Graphics:Images/symMLE_gr_3.gif] is unknown. We wish to find the ML estimator of σ. We begin in the usual way by inputting the likelihood function into Mathematica:

[Graphics:Images/symMLE_gr_4.gif]

If we try to evaluate the log-likelihood:

[Graphics:Images/symMLE_gr_5.gif]
[Graphics:Images/symMLE_gr_6.gif]

... nothing happens! (Mathematica assumes nothing about the symbols that have been entered, so its inaction is perfectly reasonable.) But we can enhance Log to do what is wanted here using the mathStatica function SuperLog. To activate this enhancement, we switch it on:

[Graphics:Images/symMLE_gr_7.gif]
[Graphics:Images/symMLE_gr_8.gif]

If we now evaluate Log[L] again, we obtain a much more useful result:

[Graphics:Images/symMLE_gr_9.gif]
[Graphics:Images/symMLE_gr_10.gif]

To derive the first-order conditions for a maximum:

[Graphics:Images/symMLE_gr_11.gif]
[Graphics:Images/symMLE_gr_12.gif]

... we solve FOC==0 using Mathematica’s Solve function. The ML estimator [Graphics:Images/symMLE_gr_13.gif] is given as a replacement rule -> for σ:

[Graphics:Images/symMLE_gr_14.gif]
[Graphics:Images/symMLE_gr_15.gif]

The second-order conditions (evaluated at the first-order conditions) are always negative, which confirms that [Graphics:Images/symMLE_gr_16.gif] is indeed the ML estimator:

[Graphics:Images/symMLE_gr_17.gif]
[Graphics:Images/symMLE_gr_18.gif]

Finally, let us suppose that an observed random sample is [Graphics:Images/symMLE_gr_19.gif]:

[Graphics:Images/symMLE_gr_20.gif]

Then the ML estimate of σ is obtained by substituting this data into the ML estimator [Graphics:Images/symMLE_gr_21.gif]:

[Graphics:Images/symMLE_gr_22.gif]
[Graphics:Images/symMLE_gr_23.gif]

Figure 1 plots the observed likelihood (for the given data) against values of σ, noting the derived exact optimal solution [Graphics:Images/symMLE_gr_24.gif].

[Graphics:Images/symMLE_gr_25.gif]

Fig. 1: The observed likelihood and [Graphics:Images/symMLE_gr_26.gif]


References
Rose, C. and Smith, M. D. (2000), Symbolic maximum likelihood estimation with Mathematica, Journal of the Royal Statistical Society, Series D: The Statistican, 49(2), 229-240.