## Chapter 1: Introduction

1.1Mathematical Statistics with Mathematica1
AA New Approach1
BDesign Philosophy1
CIf You Are New to Mathematica2
1.2Installation, Registration and Password3
AInstallation, Registration and Password3
CHelp5
1.3Core Functions6
AGetting Started6
BWorking with Parameters8
CDiscrete Random Variables9
DMultivariate Random Variables11
EPiecewise Distributions13
1.4Some Specialised Functions15
1.5Notation and Conventions24
AIntroduction24
BStatistics Notation25
CMathematica Notation27

## Chapter 2: Continuous Random Variables

2.1Introduction31
2.2Measures of Location35
AMean35
BMode36
CMedian and Quantiles37
2.3Measures of Dispersion40
2.4Moments and Generating Functions45
AMoments45
BThe Moment Generating Function46
CThe Characteristic Function50
DProperties of Characteristic Functions (and mgf's)52
EStable Distributions56
FCumulants and Probability Generating Functions60
GMoment Conversion Formulae62
2.5Conditioning, Truncation and Censoring65
AConditional / Truncated Distributions65
BConditional Expectations66
CCensored Distributions68
DOption Pricing70
2.6Pseudo-Random Number Generation72
AMathematica's Statistics Package72
BInverse Method (Symbolic)74
CInverse Method (Numerical) 75
DRejection Method77
2.7Exercises80

## Chapter 3: Discrete Random Variables

3.1Introduction81
3.2Probability: 'Throwing' a Die84
3.3Common Discrete Distributions89
AThe Bernoulli Distribution89
BThe Binomial Distribution91
CThe Poisson Distribution95
DThe Geometric and Negative Binomial Distributions98
EThe Hypergeometric Distribution100
3.4Mixing Distributions102
AComponent-Mix Distributions102
BParameter-Mix Distributions105
3.5Pseudo-Random Number Generation109
AIntroducing DiscreteRNG109
BImplementation Notes113
3.6Exercises115

## Chapter 4: Distributions of Functions of Random Variables

 4.1 Introduction 117 4.2 The Transformation Method 118 A Univariate Cases 118 B Multivariate Cases 123 C Transformations That Are Not One-to-One; Manual Methods 127 4.3 The MGF Method 130 4.4 Products and Ratios of Random Variables 133 4.5 Sums and Differences of Random Variables 136 A Applying the Transformation Method 136 B Applying the MGF Method 141 4.6 Exercises 147

## Chapter 5: Systems of Distributions

5.1Introduction149
5.2The Pearson Family149
AIntroduction149
BFitting Pearson Densities151
CPearson Types157
DPearson Coefficients in Terms of Moments159
EHigher Order Pearson-Style Families161
5.3Johnson Transformations164
AIntroduction164
BSL System (Lognormal)165
CSU System (Unbounded)168
DSB System (Bounded)173
5.4Gram-Charlier Expansions175
BHermite Polynomials; Gram-Charlier Coefficients179
5.5Non-Parametric Kernel Density Estimation181
5.6The Method of Moments183
5.7Exercises185

## Chapter 6: Multivariate Distributions

6.1Introduction187
AJoint Density Functions187
BNon-Rectangular Domains190
CProbability and Prob191
DMarginal Distributions195
EConditional Distributions197
6.2Expectations, Moments, Generating Functions200
AExpectations200
BProduct Moments, Covariance and Correlation200
CGenerating Functions203
DMoment Conversion Formulae206
6.3Independence and Dependence210
AStochastic Independence210
BCopulae211
6.4The Multivariate Normal Distribution216
AThe Bivariate Normal216
BThe Trivariate Normal226
CCDF, Probability Calculations and Numerics229
DRandom Number Generation for the Multivariate Normal232
6.5The Multivariate t and Multivariate Cauchy236
6.6Multinomial and Bivariate Poisson238
AThe Multinomial Distribution238
BThe Bivariate Poisson243
6.7Exercises248

## Chapter 7: Moments of Sampling Distributions

7.1Introduction251
AOverview251
BPower Sums and Symmetric Functions252
7.2Unbiased Estimators of Population Moments253
AUnbiased Estimators of Raw Moments of the Population253
Bh-statistics: Unbiased Estimators of Central Moments253
Ck-statistics: Unbiased Estimators of Cumulants256
DMultivariate h- and k-statistics259
7.3Moments of Moments261
AGetting Started261
BProduct Moments266
CCumulants of k-statistics267
7.4Augmented Symmetrics and Power Sums272
ADefinitions and a Fundamental Expectation Result272
BApplication 1: Understanding Unbiased Estimation275
CApplication 2: Understanding Moments of Moments275
7.5Exercises276

## Chapter 8: Asymptotic Theory

8.1Introduction277
8.2Convergence in Distribution278
8.3Asymptotic Distribution282
8.4Central Limit Theorem286
8.5Convergence in Probability292
AIntroduction292
BMarkov and Chebyshev Inequalities295
CWeak Law of Large Numbers296
8.6Exercises298

## Chapter 9: Statistical Decision Theory

9.1Introduction301
9.2Loss and Risk301
9.3Mean Square Error as Risk306
9.4Order Statistics311
BApplications318
9.5Exercises322

## Chapter 10: Unbiased Parameter Estimation

10.1Introduction325
AOverview325
BSuperD326
10.2Fisher Information326
AFisher Information326
BAlternate Form329
CAutomating Computation: FisherInformation330
DMultiple Parameters331
ESample Information332
10.3Best Unbiased Estimators333
AThe Cramér-Rao Lower Bound333
BBest Unbiased Estimators335
10.4Sufficient Statistics337
AIntroduction337
BThe Factorisation Criterion339
10.5Minimum Variance Unbiased Estimation341
AIntroduction341
BThe Rao-Blackwell Theorem342
CCompleteness and MVUE343
DConclusion346
10.6Exercises347

## Chapter 11: Principles of Maximum Likelihood Estimation

 11.1 Introduction 349 A Review 349 B SuperLog 330 11.2 The Likelihood Function 330 11.3 Maximum Likelihood Estimation 357 11.4 Properties of the ML Estimator 362 A Introduction 362 B Small Sample Properties 363 C Asymptotic Properties 365 D Regularity Conditions 367 E Invariance Property 369 11.5 Asymptotic Properties: Extensions 371 A More Than One Parameter 371 B Non-identically Distributed Samples 374 11.6 Exercises 377

## Chapter 12: Maximum Likelihood Estimation in Practice

 12.1 Introduction 379 12.2 FindMaximum 380 12.3 A Journey with FindMaximum 384 12.4 Asymptotic Inference 392 A Hypothesis Testing 392 B Standard Errors and t-statistics 395 12.5 Optimisation Algorithms 399 A Preliminaries 399 B Gradient Method Algorithms 401 12.6 The BFGS Algorithm 405 12.7 The Newton-Raphson Algorithm 412 12.8 Exercises 418

## Appendix

A.1Is That the Right Answer, Dr Faustus?421
A.2Working with Packages425
A.3Working with = , ->, == and :=426
A.4Working with Lists428
A.5Working with Subscripts429
A.6Working with Matrices433
A.7Working with Vectors438
A.8Changes to Default Behaviour443
A.9Building Your Own mathStatica Function446

447

463

469