Lecture Notes 1: Decisions and Data. In these notes, I describe some basic ideas in decision theory. theory is constructed from

Size: px
Start display at page:

Download "Lecture Notes 1: Decisions and Data. In these notes, I describe some basic ideas in decision theory. theory is constructed from"

Transcription

1 Topics in Data Analysis Steven N. Durlauf University of Wisconsin Lecture Notes : Decisions and Data In these notes, I describe some basic ideas in decision theory. theory is constructed from The Data: d Unknown(s): θ Choices: c C Loss function: l( θ, dc, ) The goal is to construct a decision rule c( d ), a mapping from D to the set of choices C. The decision rule may be stochastic, I will ignore for expositional purposes.. Statistical decision theory The standard version of statistical decision theory is formulated along Bayesian lines: all unknowns are associated with posterior probability densities; decision rule is determined by minimization of expected loss. Specifically, one implicitly chooses c( d ) by solving the problem min,, ( θ ) µθ ( ) c C l dc d (.) for each value of d.

2 This problem illustrates the key question facing an expected loss minimizing policy analyst: how does one construct µθd ( )? To understand this conditional probability, observe that ( d ) µθ ( d ) ( d) ( ) ( ) µθ, µ d θ µθ = = µ µ ( d) (.) Since µ ( d ) is not a function of θ, (it is really nothing more than a normalization that ensures that the relevant probabilities sum to ), one can rewrite (.) as ( ) ( ) ( ) µθ d µ d θ µθ (.3) This is the classic statement that the posterior probability measure for some unknown, µθd ( ), is proportional to the product of the likelihood function µ ( d θ) and the prior probability measure µθ ( ). The prior represents the information the analyst possesses about the unkown before (i.e. prior!) to the data. For some types of unknowns (e.g. parameters) and associated regularity conditions, it will be the case that as the number of observations grows, the likelihood will swamp the prior. loss functions and standard statistical exercises The statistical decision formulation does not, at first glance, appear closely related to the standard exercises of producing an estimate of a parameter, etc. However, these types of exercises may be reproduced in their essentials by a suitable choice of the loss function. Consider the loss function ( θ) ( θ,, ) ( ) l dc = c d (.4)

3 The decision c( d ) will equal E( θ ) d produce a parameter estimate that is equal to the posterior mean of the parameter. (The proof is left as an exercise.) This posterior mean is the Bayesian analog to a point estimate in frequentist analysis. (More on this later.) priors The assignment of priors is problematic, since in many (most?) contexts, the analyst does not have a good basis for constructing µθ ( ). This is true in several sense; first for many problems one may simply have nothing to say about the ex ante relative probability of different realizations. A second problem is that what information is possessed may not quantify easily. Within Bayesian statistics, a number of approaches exist to addressing this difficulty. For our purposes, I will focus on the question of constructing priors in the case where one is ignorant i.e. one does not have any basis for discriminating between different values of the unknown. And as an example, I will consider analysis of the mean and variance of a sequence of i.i.d. random variables with mean κ and variance σ. The classical approach to specifying priors in the presence of ignorance is the principle of insufficient reason an idea which is often attributed to Bermoulli and Laplace (although the name of the principle postdates them). The basic idea of the principle is that in absence of any reason to discriminate between two values of an unknown, this is interpretable as saying that we assign equal prior probabilities to them. How does the principle of insufficient reason translate into a prior for κ? If the support of the mean, under ignorance, is (, ), then the associated prior much be constant, i.e. a uniform density on the real line, i.e. µ ( λ). Notice that in this case, the prior is improper, which means that it does not integrate to. 3

4 The case of the variance σ is more complicated. One triviality: the parameter that is typically studied is the standard deviation σ. Since the variance cannot be negative, the support of σ is, under ignorance, taken to be [ 0, ). For this case, the standard ignorant prior is taken to be µ ( σ) motivation of this formulation is that it assigns a uniform prior to support is (, ).. The σ log σ, whose This example suggests a more general problem in formulating ignorance in terms of a uniform prior. Consider a parameter vector θ and its associated transformation into another parameter vector φ ( θ) = f. Assuming all probability measures can be represented as densities, for a given prior µθ ( ), it must be the case that the associated prior for φ is df µ ( φ ) = µ ( f ( φ) ) dφ ( ) φ (.5) This means that the uniform prior is not invariant across nonlinear transformations of the unknowns, which makes little sense if the prior really captures ignorance, since ignorance about θ presumably implies ignorance about φ. One way to impose invariance of this type is via Jeffrey s prior. Recalling that Fisher s information matrix is defined as ( d ) log ( d ) logµ θ µ θ I( θ ) = E = µ ( d θ ) (.6) θ D θ Jeffrey s prior is defined as ( ) ( ) / µθ I θ (.7) 4

5 This prior addresses the invariance problem in that the form of the prior can be shown to be unchanged, but raises difficulties in terms of interpretability. One issue is that the prior depends on the form of the likelihood, which seems odd since the prior is supposed to describe beliefs about parameters.. Example. Priors and posteriors for mean of normal random variables N ( κ σ 0, 0) Suppose that i ( κσ, ) x N, i =... K with σ known; the prior for κ is. It is straightforward to show that the posterior density for the mean is K κ + 0 x σ σ 0 K µ ( κσ, κ σ ) = 0, 0, x,... xk N, + K + σ0 σ σ0 σ (.8) where x is the sample mean of the data. Notice that as K, the effects of the prior on the posterior disappear, i.e. the posterior mean converges to the sample mean. Example.. Linear regression with diffuse prior and Suppose that = β + ε yi x i i with ε ( 0, σ ) N, σ known, ε i uncorrelated, xi nonstochastic. Assume that the prior on β is uniform across described above. One can show that the posterior density of β is R L, as ( ˆ βols, σ ( ' ) ) N X X (.9) 5

6 where βˆols is the OLS estimator. For this special case, there is a nice relationship between the posterior density computed by a Bayesian and the OLS objects computed by a frequentist.. Decision theory without probabilities In this section, I discuss decision critieria when probabilities are not available for the unobservables of interest. minimax The minimax solution is to choose c so that ( θ ) min max,, c C θ l dc (.0) The minimax approach is usually associated with Abraham Wald (950). An important axiomatization is due to Itzhak Gilboa and David Schmeidler (989). The criterion implicitly means that the policymaker is extremely risk averse. There are interesting applications of the criterion in philosophy, notably John Rawls (97) difference principle. The idea of the difference principle is the following. Suppose that individuals are asked to rank different distributions of economic outcomes in a society, under the proviso that they do not know which of the outcomes will turn out to be theirs. Rawls argues that individuals will choose that distribution that maximizes the utility of the worst off person. As pointed out by Kenneth Arrow (973), this is a minimax argument. It can further be understood as the limit of a welfarist analysis. For example, suppose that individual utility is defined by u = ω α where ω denotes a scalar measure of an i i individual s outcome. Further, suppose that the social state ω is assigned the 6

7 social welfare measure i ω α α i. Then, as α, W min i u i. The limit α implies an arbitrary degree of risk aversion. But this must be true for any monotonic transformation of the social welfare function, so it is also true for a utilitarian welfare calculation as well. minimax regret An alternative approach to decisionmaking without assigning probabilities to unknowns is to choose c to minimize the maximum regret. Minimax regret was originally proposed by Leonard J. Savage (95) as an alternative to minimax which is less conservative; Charles Manski is the leading advocate of its use among current researcher; he has produced numerous important papers applying minimax regret to different contexts; see Manski (008) for a conceptual defense. Regret is defined as ( θ,, ) = ( θ,, ) min ( θ,, ) r dc l dc l dc (.) c C The minimax regret solution is θ ( θ ) minc max r, dc, (.) A recent axiomatization is due to Jorg Stoye (006a,b). Example.3. minimax, minimax regret, and expected loss Suppose that there are two possible actions by the policymaker, c and c and two possible states of the world, θ and θ. The following table the losses for each policy and state of the world, as well as the maximum loss and maximum regret. 7

8 θ θ max loss max regret c c As the table indicates, by the minimax criterion and the minimax regret criterion, the policymaker should choose c. A Bayesian would address the policy problem by assigning probability p ( p ) to θ (θ ) and computing expected losses under the two policies. It is obvious that if p is close enough to, the policymaker should choose c. Example.4. minimax and minimax regret producing different policy choices Consider the alternative loss structure θ θ max loss max regret c c Here the minimax policy choice is c whereas the minimax regret choice is c. This example illustrates the intuitive appeal of minimax regret in that the criterion focuses on differences in policy effects rather than absolute levels. Example.5. minimax regret and the independence of irrelevant alternatives property 8

9 This example modifies Example.3 by adding a third policy c 3 θ θ max loss max regret c 4 4 c c The introduction of c 3 implies that the minimax regret choice is now c. This is troubling; the fact that c 3 is available has changed the ranking of c and c. This violates the property of independence of irrelevant alternatives (IIA), which is often taken to be a natural axiom for decisionmaking. The recognition that minimax regret violates IIA is due to Herbert Chernoff (954). One way to interpret this violation is that context matters for decisionmaking, an idea that is found in a number of contexts studied in behavioral economics. It is not clear, though, that the findings on context dependent decisionmaking map that closely to the IIA violations found in minimax regret. This is something worth investigating. Example.6. Another minimax regret IIA violation Here is another case where a violation of IIA occurs. Compare the analysis of policies θ θ max loss max regret c c with the comparison of 3 policies 9

10 θ θ max loss max regret c c c c is the minimax choice in the two policy case; but if c 3 ( c ) were not available, c ( c 3 ) would be the minimax solution. This seems especially paradoxical (at least to me) since c and c 3 have symmetric structures. It is the case (see Stoye (006a,b) for formal analysis) that if one is willing to forgo IIA, one can derive minimax regret under an axiom system which includes an assume of independence of never-optimal alternatives. This means that if one adds a policy that is not optimal in any state of nature (when compared to others) that it cannot lead one to change to change one s choice among the original set. Hurwicz criterion One can mix the different approaches that have been described. For example, one can consider problems that incorporate both expected loss and minimax considerations. ( α θ ( θ ) ( α) ( θ )) minc C max l, dc, + El, dc, (.3) This was originally proposed by Leo Hurwicz (95). 3. Decision rules and risk 0

11 The risk function, defined as (, ( )),, ( ) ( ) ( ) R θ c D = l θ d c d µ d θ (.4) D is a metric for understanding the performance of a rule across the data space, given conditional on the unknown. performance across the data space by probabilities. Notice that this calculation weights Relative to the set of potential decision rules, a rule c ( ) is (strictly) dominant if it produces a (strictly) lower risk than any alternative rule for all values of θ. A decision rule is said to be inadmissible if there exists an alternative rule with lower risk for all values of θ. There is no guarantee that a unique dominant rule exists. This lack of uniqueness leads to criteria such as minimax risk. The Bayes risk of a rule is defined as Θ (,, ( )) ( ) =,, ( ) ( ) ( ) ( ) Rθ dc d µθ l θ dc d µ d θµθ (.5) Recall that µ ( θ ) µθ ( ) = µθ ( ) µ ( ) Θ D d d d. Exchanging the order of integration (ok under uninteresting regularity assumptions) and substituting this identity into (.5), the Bayes risk is Θ ( θ ( )) µθ ( ) µ ( ) l, dc, d d d (.6) D It is evident that minimizing the double integral occurs by minimizing the inner integral at each data point. Minimization of the inner integral is equivalent to the original Bayesian decision theory solution. Notice that the risk function become superfluous from this perspective.

12 A general result is that any decision rule which minimizes Bayes risk relative to a proper prior is admissible. A form of the converse is also true: any admissible rule minimizes Bayes risk under some prior. 4. Fiducial inference One way to understand the differences between Bayesian and frequentist approaches is that Bayesian approaches construct probabilities of unobservables based on observables whereas frequentist approaches are based on the analysis of the probabilities of observables based on unobservables. Statistical decision theory is based on the former. Fiducial inference is a method, originally proposed by Ronald Fisher to address this distinction. Consider a sequence of Normal ( λσ, ) random variables x i. Assume the variance is known. Then the same mean x has the probability density σ N λ,. It is straightforward to K show that this implies that x λ has the probability density σ N 0, K which is also the density of λ x. λ x is an example of what is known a a pivotal quantity, which means that its distribution does not depend on λ. The fiducial argument is that since the deviation of λ from x is defined by a pivotal quantity, this pivotal quantity characterizes the uncertainty associated with the parameter, i.e. σ σ λ x N 0, implies λ N x, K K (.7) The argument is generally rejected since, without justification, it represents a transformation of a nonrandom object, in this case λ, into a random one. A variation of Fisher s ideas has been developed by Donald Fraser (see Fraser

13 (968) for a comprehensive statement), called structural inference. It is also controversial and has yet to impact empirical work. However, unlike Fisher s original formulation I do not believe one can say it is regarded as a failure. I personally find the structural inference ideas very intriguing, although quite hard to fully understand. 3

14 References Arrow, K., (973), Some Ordinalist-Utilitarian Notes on Rawls Theory of Justice, Journal of Philosophy, LXX, 9, Chernoff, H., (954), Rational Selection of Decision Functions, Econometrica,, Fraser, D., (968), The Structure of Inference, New York: John Wiley. Hurwicz, L., (95), Some Specification Problems and Applications to Econometric Models, Econometrica, 9, Gilboa, I. and D. Schmeidler, (989), Maxmin Expected Utility with Imprecise Probabilistic Information, Journal of Mathematical Economics, 8, Manski, C., (008), Actualizing Rationality, mimeo, Northwestern University, Rawls, J., (97), A Theory of Justice, Cambridge: Harvard University Press. Savage, L. J., (96), The Theory of Statistical Decision, Journal of the American Statistical Association, 46, Stoye, J., (006a), Statistical Decisions Under Ambiguity, mimeo, NYU. Stoye, J., (006b), Axioms for Minimax Regret Choice Correspondences, mimeo, NYU. Wald, A., (950), Statistical Decision Functions, New York: John Wiley. 4

Policy Evaluation in Uncertain Economic Environments

Policy Evaluation in Uncertain Economic Environments Policy Evaluation in Uncertain Economic Environments William A. Brock, Steven N. Durlauf, Kenneth D. West 1 5/7/03 forthcoming, Brookings Papers on Economic Activity The number of separate variables which

More information

Dominance and Admissibility without Priors

Dominance and Admissibility without Priors Dominance and Admissibility without Priors Jörg Stoye Cornell University September 14, 2011 Abstract This note axiomatizes the incomplete preference ordering that reflects statewise dominance with respect

More information

Great Expectations. Part I: On the Customizability of Generalized Expected Utility*

Great Expectations. Part I: On the Customizability of Generalized Expected Utility* Great Expectations. Part I: On the Customizability of Generalized Expected Utility* Francis C. Chu and Joseph Y. Halpern Department of Computer Science Cornell University Ithaca, NY 14853, U.S.A. Email:

More information

Econ 2148, spring 2019 Statistical decision theory

Econ 2148, spring 2019 Statistical decision theory Econ 2148, spring 2019 Statistical decision theory Maximilian Kasy Department of Economics, Harvard University 1 / 53 Takeaways for this part of class 1. A general framework to think about what makes a

More information

Lecture notes on statistical decision theory Econ 2110, fall 2013

Lecture notes on statistical decision theory Econ 2110, fall 2013 Lecture notes on statistical decision theory Econ 2110, fall 2013 Maximilian Kasy March 10, 2014 These lecture notes are roughly based on Robert, C. (2007). The Bayesian choice: from decision-theoretic

More information

Lecture 2: Basic Concepts of Statistical Decision Theory

Lecture 2: Basic Concepts of Statistical Decision Theory EE378A Statistical Signal Processing Lecture 2-03/31/2016 Lecture 2: Basic Concepts of Statistical Decision Theory Lecturer: Jiantao Jiao, Tsachy Weissman Scribe: John Miller and Aran Nayebi In this lecture

More information

Recursive Ambiguity and Machina s Examples

Recursive Ambiguity and Machina s Examples Recursive Ambiguity and Machina s Examples David Dillenberger Uzi Segal May 0, 0 Abstract Machina (009, 0) lists a number of situations where standard models of ambiguity aversion are unable to capture

More information

Hypothesis Testing. Part I. James J. Heckman University of Chicago. Econ 312 This draft, April 20, 2006

Hypothesis Testing. Part I. James J. Heckman University of Chicago. Econ 312 This draft, April 20, 2006 Hypothesis Testing Part I James J. Heckman University of Chicago Econ 312 This draft, April 20, 2006 1 1 A Brief Review of Hypothesis Testing and Its Uses values and pure significance tests (R.A. Fisher)

More information

On the Measurement of Inequality under Uncertainty*

On the Measurement of Inequality under Uncertainty* journal of economic theory 75, 194204 (1997) article no. ET962280 On the Measurement of Inequality under Uncertainty* Elchanan Ben-Porath Tel-Aviv University, Ramit-Aviv, 69 978 Tel-Aviv, Israel; and Northwestern

More information

Decision theory. 1 We may also consider randomized decision rules, where δ maps observed data D to a probability distribution over

Decision theory. 1 We may also consider randomized decision rules, where δ maps observed data D to a probability distribution over Point estimation Suppose we are interested in the value of a parameter θ, for example the unknown bias of a coin. We have already seen how one may use the Bayesian method to reason about θ; namely, we

More information

Uncertain Inference and Artificial Intelligence

Uncertain Inference and Artificial Intelligence March 3, 2011 1 Prepared for a Purdue Machine Learning Seminar Acknowledgement Prof. A. P. Dempster for intensive collaborations on the Dempster-Shafer theory. Jianchun Zhang, Ryan Martin, Duncan Ermini

More information

Hypothesis Testing. Econ 690. Purdue University. Justin L. Tobias (Purdue) Testing 1 / 33

Hypothesis Testing. Econ 690. Purdue University. Justin L. Tobias (Purdue) Testing 1 / 33 Hypothesis Testing Econ 690 Purdue University Justin L. Tobias (Purdue) Testing 1 / 33 Outline 1 Basic Testing Framework 2 Testing with HPD intervals 3 Example 4 Savage Dickey Density Ratio 5 Bartlett

More information

Ridge regression. Patrick Breheny. February 8. Penalized regression Ridge regression Bayesian interpretation

Ridge regression. Patrick Breheny. February 8. Penalized regression Ridge regression Bayesian interpretation Patrick Breheny February 8 Patrick Breheny High-Dimensional Data Analysis (BIOS 7600) 1/27 Introduction Basic idea Standardization Large-scale testing is, of course, a big area and we could keep talking

More information

Policy Evaluation in Uncertain Economic Environments

Policy Evaluation in Uncertain Economic Environments WILLIAM A. BROCK University of Wisconsin STEVEN N. DURLAUF University of Wisconsin KENNETH D. WEST University of Wisconsin Policy Evaluation in Uncertain Economic Environments It will be remembered that

More information

Macroeconomics and Model Uncertainty

Macroeconomics and Model Uncertainty Macroeconomics and Model Uncertainty William A. Brock and Steven N. Durlauf October 12, 2004 JEL Classification Codes: C52, E6 Keywords: model uncertainty, robustness, stabilization policy, ambiguity aversion,

More information

Sequential Decisions

Sequential Decisions Sequential Decisions A Basic Theorem of (Bayesian) Expected Utility Theory: If you can postpone a terminal decision in order to observe, cost free, an experiment whose outcome might change your terminal

More information

Are Probabilities Used in Markets? 1

Are Probabilities Used in Markets? 1 Journal of Economic Theory 91, 8690 (2000) doi:10.1006jeth.1999.2590, available online at http:www.idealibrary.com on NOTES, COMMENTS, AND LETTERS TO THE EDITOR Are Probabilities Used in Markets? 1 Larry

More information

Econ 2140, spring 2018, Part IIa Statistical Decision Theory

Econ 2140, spring 2018, Part IIa Statistical Decision Theory Econ 2140, spring 2018, Part IIa Maximilian Kasy Department of Economics, Harvard University 1 / 35 Examples of decision problems Decide whether or not the hypothesis of no racial discrimination in job

More information

A Discussion of the Bayesian Approach

A Discussion of the Bayesian Approach A Discussion of the Bayesian Approach Reference: Chapter 10 of Theoretical Statistics, Cox and Hinkley, 1974 and Sujit Ghosh s lecture notes David Madigan Statistics The subject of statistics concerns

More information

I. Bayesian econometrics

I. Bayesian econometrics I. Bayesian econometrics A. Introduction B. Bayesian inference in the univariate regression model C. Statistical decision theory Question: once we ve calculated the posterior distribution, what do we do

More information

arxiv: v1 [cs.ai] 16 Aug 2018

arxiv: v1 [cs.ai] 16 Aug 2018 Decision-Making with Belief Functions: a Review Thierry Denœux arxiv:1808.05322v1 [cs.ai] 16 Aug 2018 Université de Technologie de Compiègne, CNRS UMR 7253 Heudiasyc, Compiègne, France email: thierry.denoeux@utc.fr

More information

UNIVERSITY OF NOTTINGHAM. Discussion Papers in Economics CONSISTENT FIRM CHOICE AND THE THEORY OF SUPPLY

UNIVERSITY OF NOTTINGHAM. Discussion Papers in Economics CONSISTENT FIRM CHOICE AND THE THEORY OF SUPPLY UNIVERSITY OF NOTTINGHAM Discussion Papers in Economics Discussion Paper No. 0/06 CONSISTENT FIRM CHOICE AND THE THEORY OF SUPPLY by Indraneel Dasgupta July 00 DP 0/06 ISSN 1360-438 UNIVERSITY OF NOTTINGHAM

More information

One-parameter models

One-parameter models One-parameter models Patrick Breheny January 22 Patrick Breheny BST 701: Bayesian Modeling in Biostatistics 1/17 Introduction Binomial data is not the only example in which Bayesian solutions can be worked

More information

Testing Restrictions and Comparing Models

Testing Restrictions and Comparing Models Econ. 513, Time Series Econometrics Fall 00 Chris Sims Testing Restrictions and Comparing Models 1. THE PROBLEM We consider here the problem of comparing two parametric models for the data X, defined by

More information

STAT 425: Introduction to Bayesian Analysis

STAT 425: Introduction to Bayesian Analysis STAT 425: Introduction to Bayesian Analysis Marina Vannucci Rice University, USA Fall 2017 Marina Vannucci (Rice University, USA) Bayesian Analysis (Part 1) Fall 2017 1 / 10 Lecture 7: Prior Types Subjective

More information

Decision Analysis. An insightful study of the decision-making process under uncertainty and risk

Decision Analysis. An insightful study of the decision-making process under uncertainty and risk Decision Analysis An insightful study of the decision-making process under uncertainty and risk Decision Theory A philosophy, articulated by a set of logical axioms and a methodology, for analyzing the

More information

U n iversity o f H ei delberg. Informativeness of Experiments for MEU A Recursive Definition

U n iversity o f H ei delberg. Informativeness of Experiments for MEU A Recursive Definition U n iversity o f H ei delberg Department of Economics Discussion Paper Series No. 572 482482 Informativeness of Experiments for MEU A Recursive Definition Daniel Heyen and Boris R. Wiesenfarth October

More information

Bayesian consistent prior selection

Bayesian consistent prior selection Bayesian consistent prior selection Christopher P. Chambers and Takashi Hayashi August 2005 Abstract A subjective expected utility agent is given information about the state of the world in the form of

More information

Lecture 2: Statistical Decision Theory (Part I)

Lecture 2: Statistical Decision Theory (Part I) Lecture 2: Statistical Decision Theory (Part I) Hao Helen Zhang Hao Helen Zhang Lecture 2: Statistical Decision Theory (Part I) 1 / 35 Outline of This Note Part I: Statistics Decision Theory (from Statistical

More information

Dynamics of Inductive Inference in a Uni ed Model

Dynamics of Inductive Inference in a Uni ed Model Dynamics of Inductive Inference in a Uni ed Model Itzhak Gilboa, Larry Samuelson, and David Schmeidler September 13, 2011 Gilboa, Samuelson, and Schmeidler () Dynamics of Inductive Inference in a Uni ed

More information

NBER WORKING PAPER SERIES MODEL UNCERTAINTY AND POLICY EVALUATION: SOME THEORY AND EMPIRICS. William A. Brock Steven N. Durlauf Kenneth D.

NBER WORKING PAPER SERIES MODEL UNCERTAINTY AND POLICY EVALUATION: SOME THEORY AND EMPIRICS. William A. Brock Steven N. Durlauf Kenneth D. NBER WORKING PAPER SERIES MODEL UNCERTAINTY AND POLICY EVALUATION: SOME THEORY AND EMPIRICS William A. Brock Steven N. Durlauf Kenneth D. West Working Paper 10916 http://www.nber.org/papers/w10916 NATIONAL

More information

Decision-making with belief functions

Decision-making with belief functions Decision-making with belief functions Thierry Denœux Université de Technologie de Compiègne, France HEUDIASYC (UMR CNRS 7253) https://www.hds.utc.fr/ tdenoeux Fourth School on Belief Functions and their

More information

The Bayesian Choice. Christian P. Robert. From Decision-Theoretic Foundations to Computational Implementation. Second Edition.

The Bayesian Choice. Christian P. Robert. From Decision-Theoretic Foundations to Computational Implementation. Second Edition. Christian P. Robert The Bayesian Choice From Decision-Theoretic Foundations to Computational Implementation Second Edition With 23 Illustrations ^Springer" Contents Preface to the Second Edition Preface

More information

Rationality and Uncertainty

Rationality and Uncertainty Rationality and Uncertainty Based on papers by Itzhak Gilboa, Massimo Marinacci, Andy Postlewaite, and David Schmeidler Warwick Aug 23, 2013 Risk and Uncertainty Dual use of probability: empirical frequencies

More information

STATS 200: Introduction to Statistical Inference. Lecture 29: Course review

STATS 200: Introduction to Statistical Inference. Lecture 29: Course review STATS 200: Introduction to Statistical Inference Lecture 29: Course review Course review We started in Lecture 1 with a fundamental assumption: Data is a realization of a random process. The goal throughout

More information

STA 732: Inference. Notes 10. Parameter Estimation from a Decision Theoretic Angle. Other resources

STA 732: Inference. Notes 10. Parameter Estimation from a Decision Theoretic Angle. Other resources STA 732: Inference Notes 10. Parameter Estimation from a Decision Theoretic Angle Other resources 1 Statistical rules, loss and risk We saw that a major focus of classical statistics is comparing various

More information

Lecture 1: Bayesian Framework Basics

Lecture 1: Bayesian Framework Basics Lecture 1: Bayesian Framework Basics Melih Kandemir melih.kandemir@iwr.uni-heidelberg.de April 21, 2014 What is this course about? Building Bayesian machine learning models Performing the inference of

More information

Bayesian inference. Fredrik Ronquist and Peter Beerli. October 3, 2007

Bayesian inference. Fredrik Ronquist and Peter Beerli. October 3, 2007 Bayesian inference Fredrik Ronquist and Peter Beerli October 3, 2007 1 Introduction The last few decades has seen a growing interest in Bayesian inference, an alternative approach to statistical inference.

More information

Definitions and Proofs

Definitions and Proofs Giving Advice vs. Making Decisions: Transparency, Information, and Delegation Online Appendix A Definitions and Proofs A. The Informational Environment The set of states of nature is denoted by = [, ],

More information

Whaddya know? Bayesian and Frequentist approaches to inverse problems

Whaddya know? Bayesian and Frequentist approaches to inverse problems Whaddya know? Bayesian and Frequentist approaches to inverse problems Philip B. Stark Department of Statistics University of California, Berkeley Inverse Problems: Practical Applications and Advanced Analysis

More information

Stat260: Bayesian Modeling and Inference Lecture Date: February 10th, Jeffreys priors. exp 1 ) p 2

Stat260: Bayesian Modeling and Inference Lecture Date: February 10th, Jeffreys priors. exp 1 ) p 2 Stat260: Bayesian Modeling and Inference Lecture Date: February 10th, 2010 Jeffreys priors Lecturer: Michael I. Jordan Scribe: Timothy Hunter 1 Priors for the multivariate Gaussian Consider a multivariate

More information

Statistics: Learning models from data

Statistics: Learning models from data DS-GA 1002 Lecture notes 5 October 19, 2015 Statistics: Learning models from data Learning models from data that are assumed to be generated probabilistically from a certain unknown distribution is a crucial

More information

CS 540: Machine Learning Lecture 2: Review of Probability & Statistics

CS 540: Machine Learning Lecture 2: Review of Probability & Statistics CS 540: Machine Learning Lecture 2: Review of Probability & Statistics AD January 2008 AD () January 2008 1 / 35 Outline Probability theory (PRML, Section 1.2) Statistics (PRML, Sections 2.1-2.4) AD ()

More information

On Backtesting Risk Measurement Models

On Backtesting Risk Measurement Models On Backtesting Risk Measurement Models Hideatsu Tsukahara Department of Economics, Seijo University e-mail address: tsukahar@seijo.ac.jp 1 Introduction In general, the purpose of backtesting is twofold:

More information

Best Guaranteed Result Principle and Decision Making in Operations with Stochastic Factors and Uncertainty

Best Guaranteed Result Principle and Decision Making in Operations with Stochastic Factors and Uncertainty Stochastics and uncertainty underlie all the processes of the Universe. N.N.Moiseev Best Guaranteed Result Principle and Decision Making in Operations with Stochastic Factors and Uncertainty by Iouldouz

More information

Preliminary Statistics Lecture 2: Probability Theory (Outline) prelimsoas.webs.com

Preliminary Statistics Lecture 2: Probability Theory (Outline) prelimsoas.webs.com 1 School of Oriental and African Studies September 2015 Department of Economics Preliminary Statistics Lecture 2: Probability Theory (Outline) prelimsoas.webs.com Gujarati D. Basic Econometrics, Appendix

More information

An Axiomatic Model of Reference Dependence under Uncertainty. Yosuke Hashidate

An Axiomatic Model of Reference Dependence under Uncertainty. Yosuke Hashidate An Axiomatic Model of Reference Dependence under Uncertainty Yosuke Hashidate Abstract This paper presents a behavioral characteization of a reference-dependent choice under uncertainty in the Anscombe-Aumann

More information

New Perspectives on Statistical Decisions under Ambiguity

New Perspectives on Statistical Decisions under Ambiguity New Perspectives on Statistical Decisions under Ambiguity Jörg Stoye Cornell University September 4, 2011 Abstract This survey summarizes and connects recent work in the foundations and applications of

More information

Bayesian Inference: Concept and Practice

Bayesian Inference: Concept and Practice Inference: Concept and Practice fundamentals Johan A. Elkink School of Politics & International Relations University College Dublin 5 June 2017 1 2 3 Bayes theorem In order to estimate the parameters of

More information

Introduction to Applied Bayesian Modeling. ICPSR Day 4

Introduction to Applied Bayesian Modeling. ICPSR Day 4 Introduction to Applied Bayesian Modeling ICPSR Day 4 Simple Priors Remember Bayes Law: Where P(A) is the prior probability of A Simple prior Recall the test for disease example where we specified the

More information

Minimax and the Value of Information

Minimax and the Value of Information Minimax and the Value of Information Evan Sadler April 4, 2014 Abstract In his discussion of minimax decision rules, Savage (1954, p. 170) presents an example purporting to show that minimax applied to

More information

Minimax-Regret Sample Design in Anticipation of Missing Data, With Application to Panel Data. Jeff Dominitz RAND. and

Minimax-Regret Sample Design in Anticipation of Missing Data, With Application to Panel Data. Jeff Dominitz RAND. and Minimax-Regret Sample Design in Anticipation of Missing Data, With Application to Panel Data Jeff Dominitz RAND and Charles F. Manski Department of Economics and Institute for Policy Research, Northwestern

More information

Relative Benefit Equilibrating Bargaining Solution and the Ordinal Interpretation of Gauthier s Arbitration Scheme

Relative Benefit Equilibrating Bargaining Solution and the Ordinal Interpretation of Gauthier s Arbitration Scheme Relative Benefit Equilibrating Bargaining Solution and the Ordinal Interpretation of Gauthier s Arbitration Scheme Mantas Radzvilas July 2017 Abstract In 1986 David Gauthier proposed an arbitration scheme

More information

Principles of Statistical Inference

Principles of Statistical Inference Principles of Statistical Inference Nancy Reid and David Cox August 30, 2013 Introduction Statistics needs a healthy interplay between theory and applications theory meaning Foundations, rather than theoretical

More information

Bayesian Econometrics

Bayesian Econometrics Bayesian Econometrics Christopher A. Sims Princeton University sims@princeton.edu September 20, 2016 Outline I. The difference between Bayesian and non-bayesian inference. II. Confidence sets and confidence

More information

Principles of Statistical Inference

Principles of Statistical Inference Principles of Statistical Inference Nancy Reid and David Cox August 30, 2013 Introduction Statistics needs a healthy interplay between theory and applications theory meaning Foundations, rather than theoretical

More information

A Bayesian perspective on GMM and IV

A Bayesian perspective on GMM and IV A Bayesian perspective on GMM and IV Christopher A. Sims Princeton University sims@princeton.edu November 26, 2013 What is a Bayesian perspective? A Bayesian perspective on scientific reporting views all

More information

Lecture 12 November 3

Lecture 12 November 3 STATS 300A: Theory of Statistics Fall 2015 Lecture 12 November 3 Lecturer: Lester Mackey Scribe: Jae Hyuck Park, Christian Fong Warning: These notes may contain factual and/or typographic errors. 12.1

More information

Why a Bayesian researcher might prefer observational data

Why a Bayesian researcher might prefer observational data Why a Bayesian researcher might prefer observational data Macartan Humphreys May 2, 206 Abstract I give an illustration of a simple problem in which a Bayesian researcher can choose between random assignment

More information

Second-Order Expected Utility

Second-Order Expected Utility Second-Order Expected Utility Simon Grant Ben Polak Tomasz Strzalecki Preliminary version: November 2009 Abstract We present two axiomatizations of the Second-Order Expected Utility model in the context

More information

Columbia University. Department of Economics Discussion Paper Series. The Knob of the Discord. Massimiliano Amarante Fabio Maccheroni

Columbia University. Department of Economics Discussion Paper Series. The Knob of the Discord. Massimiliano Amarante Fabio Maccheroni Columbia University Department of Economics Discussion Paper Series The Knob of the Discord Massimiliano Amarante Fabio Maccheroni Discussion Paper No.: 0405-14 Department of Economics Columbia University

More information

Stat 5101 Lecture Notes

Stat 5101 Lecture Notes Stat 5101 Lecture Notes Charles J. Geyer Copyright 1998, 1999, 2000, 2001 by Charles J. Geyer May 7, 2001 ii Stat 5101 (Geyer) Course Notes Contents 1 Random Variables and Change of Variables 1 1.1 Random

More information

Gaussian processes. Chuong B. Do (updated by Honglak Lee) November 22, 2008

Gaussian processes. Chuong B. Do (updated by Honglak Lee) November 22, 2008 Gaussian processes Chuong B Do (updated by Honglak Lee) November 22, 2008 Many of the classical machine learning algorithms that we talked about during the first half of this course fit the following pattern:

More information

Should all Machine Learning be Bayesian? Should all Bayesian models be non-parametric?

Should all Machine Learning be Bayesian? Should all Bayesian models be non-parametric? Should all Machine Learning be Bayesian? Should all Bayesian models be non-parametric? Zoubin Ghahramani Department of Engineering University of Cambridge, UK zoubin@eng.cam.ac.uk http://learning.eng.cam.ac.uk/zoubin/

More information

DS-GA 1002 Lecture notes 11 Fall Bayesian statistics

DS-GA 1002 Lecture notes 11 Fall Bayesian statistics DS-GA 100 Lecture notes 11 Fall 016 Bayesian statistics In the frequentist paradigm we model the data as realizations from a distribution that depends on deterministic parameters. In contrast, in Bayesian

More information

Recursive Ambiguity and Machina s Examples

Recursive Ambiguity and Machina s Examples Recursive Ambiguity and Machina s Examples David Dillenberger Uzi Segal January 9, 204 Abstract Machina (2009, 202) lists a number of situations where Choquet expected utility, as well as other known models

More information

STAT J535: Introduction

STAT J535: Introduction David B. Hitchcock E-Mail: hitchcock@stat.sc.edu Spring 2012 Chapter 1: Introduction to Bayesian Data Analysis Bayesian statistical inference uses Bayes Law (Bayes Theorem) to combine prior information

More information

Peter Hoff Statistical decision problems September 24, Statistical inference 1. 2 The estimation problem 2. 3 The testing problem 4

Peter Hoff Statistical decision problems September 24, Statistical inference 1. 2 The estimation problem 2. 3 The testing problem 4 Contents 1 Statistical inference 1 2 The estimation problem 2 3 The testing problem 4 4 Loss, decision rules and risk 6 4.1 Statistical decision problems....................... 7 4.2 Decision rules and

More information

Fiducial Inference and Generalizations

Fiducial Inference and Generalizations Fiducial Inference and Generalizations Jan Hannig Department of Statistics and Operations Research The University of North Carolina at Chapel Hill Hari Iyer Department of Statistics, Colorado State University

More information

LECTURE 5 NOTES. n t. t Γ(a)Γ(b) pt+a 1 (1 p) n t+b 1. The marginal density of t is. Γ(t + a)γ(n t + b) Γ(n + a + b)

LECTURE 5 NOTES. n t. t Γ(a)Γ(b) pt+a 1 (1 p) n t+b 1. The marginal density of t is. Γ(t + a)γ(n t + b) Γ(n + a + b) LECTURE 5 NOTES 1. Bayesian point estimators. In the conventional (frequentist) approach to statistical inference, the parameter θ Θ is considered a fixed quantity. In the Bayesian approach, it is considered

More information

Ch. 5 Hypothesis Testing

Ch. 5 Hypothesis Testing Ch. 5 Hypothesis Testing The current framework of hypothesis testing is largely due to the work of Neyman and Pearson in the late 1920s, early 30s, complementing Fisher s work on estimation. As in estimation,

More information

De Finetti s ultimate failure. Krzysztof Burdzy University of Washington

De Finetti s ultimate failure. Krzysztof Burdzy University of Washington De Finetti s ultimate failure Krzysztof Burdzy University of Washington Does philosophy matter? Global temperatures will rise by 1 degree in 20 years with probability 80%. Reading suggestions Probability

More information

Lecture 7 October 13

Lecture 7 October 13 STATS 300A: Theory of Statistics Fall 2015 Lecture 7 October 13 Lecturer: Lester Mackey Scribe: Jing Miao and Xiuyuan Lu 7.1 Recap So far, we have investigated various criteria for optimal inference. We

More information

MAXIMUM LIKELIHOOD, SET ESTIMATION, MODEL CRITICISM

MAXIMUM LIKELIHOOD, SET ESTIMATION, MODEL CRITICISM Eco517 Fall 2004 C. Sims MAXIMUM LIKELIHOOD, SET ESTIMATION, MODEL CRITICISM 1. SOMETHING WE SHOULD ALREADY HAVE MENTIONED A t n (µ, Σ) distribution converges, as n, to a N(µ, Σ). Consider the univariate

More information

Introduction to Bayesian Statistics with WinBUGS Part 4 Priors and Hierarchical Models

Introduction to Bayesian Statistics with WinBUGS Part 4 Priors and Hierarchical Models Introduction to Bayesian Statistics with WinBUGS Part 4 Priors and Hierarchical Models Matthew S. Johnson New York ASA Chapter Workshop CUNY Graduate Center New York, NY hspace1in December 17, 2009 December

More information

Economics 201B Economic Theory (Spring 2017) Bargaining. Topics: the axiomatic approach (OR 15) and the strategic approach (OR 7).

Economics 201B Economic Theory (Spring 2017) Bargaining. Topics: the axiomatic approach (OR 15) and the strategic approach (OR 7). Economics 201B Economic Theory (Spring 2017) Bargaining Topics: the axiomatic approach (OR 15) and the strategic approach (OR 7). The axiomatic approach (OR 15) Nash s (1950) work is the starting point

More information

Comment on The Veil of Public Ignorance

Comment on The Veil of Public Ignorance Comment on The Veil of Public Ignorance Geoffroy de Clippel February 2010 Nehring (2004) proposes an interesting methodology to extend the utilitarian criterion defined under complete information to an

More information

A BAYESIAN MATHEMATICAL STATISTICS PRIMER. José M. Bernardo Universitat de València, Spain

A BAYESIAN MATHEMATICAL STATISTICS PRIMER. José M. Bernardo Universitat de València, Spain A BAYESIAN MATHEMATICAL STATISTICS PRIMER José M. Bernardo Universitat de València, Spain jose.m.bernardo@uv.es Bayesian Statistics is typically taught, if at all, after a prior exposure to frequentist

More information

Lecture 3: Introduction to Complexity Regularization

Lecture 3: Introduction to Complexity Regularization ECE90 Spring 2007 Statistical Learning Theory Instructor: R. Nowak Lecture 3: Introduction to Complexity Regularization We ended the previous lecture with a brief discussion of overfitting. Recall that,

More information

PART I INTRODUCTION The meaning of probability Basic definitions for frequentist statistics and Bayesian inference Bayesian inference Combinatorics

PART I INTRODUCTION The meaning of probability Basic definitions for frequentist statistics and Bayesian inference Bayesian inference Combinatorics Table of Preface page xi PART I INTRODUCTION 1 1 The meaning of probability 3 1.1 Classical definition of probability 3 1.2 Statistical definition of probability 9 1.3 Bayesian understanding of probability

More information

Axiomatic bargaining. theory

Axiomatic bargaining. theory Axiomatic bargaining theory Objective: To formulate and analyse reasonable criteria for dividing the gains or losses from a cooperative endeavour among several agents. We begin with a non-empty set of

More information

1 Hypothesis Testing and Model Selection

1 Hypothesis Testing and Model Selection A Short Course on Bayesian Inference (based on An Introduction to Bayesian Analysis: Theory and Methods by Ghosh, Delampady and Samanta) Module 6: From Chapter 6 of GDS 1 Hypothesis Testing and Model Selection

More information

A Note on the Existence of Ratifiable Acts

A Note on the Existence of Ratifiable Acts A Note on the Existence of Ratifiable Acts Joseph Y. Halpern Cornell University Computer Science Department Ithaca, NY 14853 halpern@cs.cornell.edu http://www.cs.cornell.edu/home/halpern August 15, 2018

More information

Theories of justice. Harsanyi s approach, Rawls approach, Unification

Theories of justice. Harsanyi s approach, Rawls approach, Unification Theories of justice Section 6.4 in JR. The choice of one swf over another is ultimately a choice between alternative sets of ethical values. Two main approaches: Harsanyi s approach, Rawls approach, Unification

More information

BTRY 4830/6830: Quantitative Genomics and Genetics

BTRY 4830/6830: Quantitative Genomics and Genetics BTRY 4830/6830: Quantitative Genomics and Genetics Lecture 23: Alternative tests in GWAS / (Brief) Introduction to Bayesian Inference Jason Mezey jgm45@cornell.edu Nov. 13, 2014 (Th) 8:40-9:55 Announcements

More information

Part 2: One-parameter models

Part 2: One-parameter models Part 2: One-parameter models 1 Bernoulli/binomial models Return to iid Y 1,...,Y n Bin(1, ). The sampling model/likelihood is p(y 1,...,y n ) = P y i (1 ) n P y i When combined with a prior p( ), Bayes

More information

Social Choice Theory. Felix Munoz-Garcia School of Economic Sciences Washington State University. EconS Advanced Microeconomics II

Social Choice Theory. Felix Munoz-Garcia School of Economic Sciences Washington State University. EconS Advanced Microeconomics II Social Choice Theory Felix Munoz-Garcia School of Economic Sciences Washington State University EconS 503 - Advanced Microeconomics II Social choice theory MWG, Chapter 21. JR, Chapter 6.2-6.5. Additional

More information

STATISTICAL TREATMENT RULES FOR HETEROGENEOUS POPULATIONS

STATISTICAL TREATMENT RULES FOR HETEROGENEOUS POPULATIONS STATISTICAL TREATMENT RULES FOR HETEROGENEOUS POPULATIONS Charles Manski THE INSTITUTE FOR FISCAL STUDIES DEPARTMENT OF ECONOMICS, UCL cemmap working paper CWP03/03 STATISTICAL TREATMENT RULES FOR HETEROGENEOUS

More information

9/12/17. Types of learning. Modeling data. Supervised learning: Classification. Supervised learning: Regression. Unsupervised learning: Clustering

9/12/17. Types of learning. Modeling data. Supervised learning: Classification. Supervised learning: Regression. Unsupervised learning: Clustering Types of learning Modeling data Supervised: we know input and targets Goal is to learn a model that, given input data, accurately predicts target data Unsupervised: we know the input only and want to make

More information

A noninformative Bayesian approach to domain estimation

A noninformative Bayesian approach to domain estimation A noninformative Bayesian approach to domain estimation Glen Meeden School of Statistics University of Minnesota Minneapolis, MN 55455 glen@stat.umn.edu August 2002 Revised July 2003 To appear in Journal

More information

6 Pattern Mixture Models

6 Pattern Mixture Models 6 Pattern Mixture Models A common theme underlying the methods we have discussed so far is that interest focuses on making inference on parameters in a parametric or semiparametric model for the full data

More information

DECISIONS UNDER UNCERTAINTY

DECISIONS UNDER UNCERTAINTY August 18, 2003 Aanund Hylland: # DECISIONS UNDER UNCERTAINTY Standard theory and alternatives 1. Introduction Individual decision making under uncertainty can be characterized as follows: The decision

More information

Parametric Techniques

Parametric Techniques Parametric Techniques Jason J. Corso SUNY at Buffalo J. Corso (SUNY at Buffalo) Parametric Techniques 1 / 39 Introduction When covering Bayesian Decision Theory, we assumed the full probabilistic structure

More information

Action-Independent Subjective Expected Utility without States of the World

Action-Independent Subjective Expected Utility without States of the World heoretical Economics Letters, 013, 3, 17-1 http://dxdoiorg/10436/tel01335a004 Published Online September 013 (http://wwwscirporg/journal/tel) Action-Independent Subjective Expected Utility without States

More information

Statistical Approaches to Learning and Discovery. Week 4: Decision Theory and Risk Minimization. February 3, 2003

Statistical Approaches to Learning and Discovery. Week 4: Decision Theory and Risk Minimization. February 3, 2003 Statistical Approaches to Learning and Discovery Week 4: Decision Theory and Risk Minimization February 3, 2003 Recall From Last Time Bayesian expected loss is ρ(π, a) = E π [L(θ, a)] = L(θ, a) df π (θ)

More information

Riskiness for sets of gambles

Riskiness for sets of gambles Riskiness for sets of gambles Moti Michaeli Abstract Aumann Serrano (2008) and Foster Hart (2009) suggest two new riskiness measures, each of which enables one to elicit a complete and objective ranking

More information

David Giles Bayesian Econometrics

David Giles Bayesian Econometrics 9. Model Selection - Theory David Giles Bayesian Econometrics One nice feature of the Bayesian analysis is that we can apply it to drawing inferences about entire models, not just parameters. Can't do

More information

Probability and Information Theory. Sargur N. Srihari

Probability and Information Theory. Sargur N. Srihari Probability and Information Theory Sargur N. srihari@cedar.buffalo.edu 1 Topics in Probability and Information Theory Overview 1. Why Probability? 2. Random Variables 3. Probability Distributions 4. Marginal

More information

Mathematical Fuzzy Logic in Reasoning and Decision Making under Uncertainty

Mathematical Fuzzy Logic in Reasoning and Decision Making under Uncertainty Mathematical Fuzzy Logic in Reasoning and Decision Making under Uncertainty Hykel Hosni http://www.filosofia.unimi.it/~hosni/ Prague, 17 June 2016 Hykel Hosni (UniMi) Mathematical Fuzzy Logic in Reasoning

More information

RECURSIVE AMBIGUITY AND MACHINA S EXAMPLES 1. INTRODUCTION

RECURSIVE AMBIGUITY AND MACHINA S EXAMPLES 1. INTRODUCTION INTERNATIONAL ECONOMIC REVIEW Vol. 56, No., February 05 RECURSIVE AMBIGUITY AND MACHINA S EXAMPLES BY DAVID DILLENBERGER AND UZI SEGAL University of Pennsylvania, U.S.A.; Boston College, U.S.A., and Warwick

More information