Statistical modelling and latent variables (2)

Similar documents
STK4011 and STK9011 Autumn 2016

CHAPTER VI Statistical Analysis of Experimental Data

Part I: Background on the Binomial Distribution

LINEAR REGRESSION ANALYSIS

Chapter 5 Properties of a Random Sample

STATISTICAL PROPERTIES OF LEAST SQUARES ESTIMATORS. x, where. = y - ˆ " 1

Estimation of Stress- Strength Reliability model using finite mixture of exponential distributions

Chapter 14 Logistic Regression Models

Lecture 7. Confidence Intervals and Hypothesis Tests in the Simple CLR Model

Chapter 13, Part A Analysis of Variance and Experimental Design. Introduction to Analysis of Variance. Introduction to Analysis of Variance

Comparison of Parameters of Lognormal Distribution Based On the Classical and Posterior Estimates

Ordinary Least Squares Regression. Simple Regression. Algebra and Assumptions.

Introduction to local (nonparametric) density estimation. methods

X X X E[ ] E X E X. is the ()m n where the ( i,)th. j element is the mean of the ( i,)th., then

Multivariate Transformation of Variables and Maximum Likelihood Estimation

Chapter 4 (Part 1): Non-Parametric Classification (Sections ) Pattern Classification 4.3) Announcements

ENGI 4421 Joint Probability Distributions Page Joint Probability Distributions [Navidi sections 2.5 and 2.6; Devore sections

Point Estimation: definition of estimators

Simulation Output Analysis

STK3100 and STK4100 Autumn 2018

STK3100 and STK4100 Autumn 2017

X ε ) = 0, or equivalently, lim

22 Nonparametric Methods.

Continuous Distributions

STA302/1001-Fall 2008 Midterm Test October 21, 2008

Summary of the lecture in Biostatistics

UNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS

ECON 5360 Class Notes GMM

Analysis of Variance with Weibull Data

Chapter 8. Inferences about More Than Two Population Central Values

Lecture 8: Linear Regression

The expected value of a sum of random variables,, is the sum of the expected values:

Part 4b Asymptotic Results for MRR2 using PRESS. Recall that the PRESS statistic is a special type of cross validation procedure (see Allen (1971))

Convergence of the Desroziers scheme and its relation to the lag innovation diagnostic

Functions of Random Variables

ENGI 3423 Simple Linear Regression Page 12-01

( ) = ( ) ( ) Chapter 13 Asymptotic Theory and Stochastic Regressors. Stochastic regressors model

( ) ( ) ( ) f ( ) ( )

Lecture 3 Probability review (cont d)

Module 7. Lecture 7: Statistical parameter estimation

Bayes Estimator for Exponential Distribution with Extension of Jeffery Prior Information

best estimate (mean) for X uncertainty or error in the measurement (systematic, random or statistical) best

i 2 σ ) i = 1,2,...,n , and = 3.01 = 4.01

Lecture Notes Types of economic variables

ECONOMETRIC THEORY. MODULE VIII Lecture - 26 Heteroskedasticity

A Study of the Reproducibility of Measurements with HUR Leg Extension/Curl Research Line

Outline. Point Pattern Analysis Part I. Revisit IRP/CSR

Simple Linear Regression

THE ROYAL STATISTICAL SOCIETY 2010 EXAMINATIONS SOLUTIONS GRADUATE DIPLOMA MODULE 2 STATISTICAL INFERENCE

Analysis of System Performance IN2072 Chapter 5 Analysis of Non Markov Systems

Chapter 8: Statistical Analysis of Simulated Data

C-1: Aerodynamics of Airfoils 1 C-2: Aerodynamics of Airfoils 2 C-3: Panel Methods C-4: Thin Airfoil Theory

Special Instructions / Useful Data

Econometric Methods. Review of Estimation

Discrete Mathematics and Probability Theory Fall 2016 Seshia and Walrand DIS 10b

UNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS

Dr. Shalabh. Indian Institute of Technology Kanpur

THE ROYAL STATISTICAL SOCIETY 2016 EXAMINATIONS SOLUTIONS HIGHER CERTIFICATE MODULE 5

Lecture 3. Sampling, sampling distributions, and parameter estimation

2.28 The Wall Street Journal is probably referring to the average number of cubes used per glass measured for some population that they have chosen.

Maximum Likelihood Estimation

THE ROYAL STATISTICAL SOCIETY HIGHER CERTIFICATE

Lecture Note to Rice Chapter 8

= 2. Statistic - function that doesn't depend on any of the known parameters; examples:

THE ROYAL STATISTICAL SOCIETY GRADUATE DIPLOMA

1 Solution to Problem 6.40

Unsupervised Learning and Other Neural Networks

Chapter 4 Multiple Random Variables

1. The weight of six Golden Retrievers is 66, 61, 70, 67, 92 and 66 pounds. The weight of six Labrador Retrievers is 54, 60, 72, 78, 84 and 67.

LECTURE - 4 SIMPLE RANDOM SAMPLING DR. SHALABH DEPARTMENT OF MATHEMATICS AND STATISTICS INDIAN INSTITUTE OF TECHNOLOGY KANPUR

hp calculators HP 30S Statistics Averages and Standard Deviations Average and Standard Deviation Practice Finding Averages and Standard Deviations

Naïve Bayes MIT Course Notes Cynthia Rudin

Chapter 13 Student Lecture Notes 13-1

Class 13,14 June 17, 19, 2015

Statistics Descriptive and Inferential Statistics. Instructor: Daisuke Nagakura

CHAPTER 3 POSTERIOR DISTRIBUTIONS

BAYESIAN ESTIMATOR OF A CHANGE POINT IN THE HAZARD FUNCTION

5 Short Proofs of Simplified Stirling s Approximation

GOALS The Samples Why Sample the Population? What is a Probability Sample? Four Most Commonly Used Probability Sampling Methods

The Mathematical Appendix

IJOART. Copyright 2014 SciResPub.

{ }{ ( )} (, ) = ( ) ( ) ( ) Chapter 14 Exercises in Sampling Theory. Exercise 1 (Simple random sampling): Solution:

Logistic regression (continued)

Point Estimation: definition of estimators

3. Basic Concepts: Consequences and Properties

ESS Line Fitting

ρ < 1 be five real numbers. The

Multiple Linear Regression Analysis

EECE 301 Signals & Systems

Marcinkiewicz strong laws for linear statistics of ρ -mixing sequences of random variables

BAYESIAN INFERENCES FOR TWO PARAMETER WEIBULL DISTRIBUTION

VOL. 3, NO. 11, November 2013 ISSN ARPN Journal of Science and Technology All rights reserved.

THE ROYAL STATISTICAL SOCIETY GRADUATE DIPLOMA

Bayesian Inferences for Two Parameter Weibull Distribution Kipkoech W. Cheruiyot 1, Abel Ouko 2, Emily Kirimi 3

Bayes (Naïve or not) Classifiers: Generative Approach

Chapter 4 Multiple Random Variables

Comparing Different Estimators of three Parameters for Transmuted Weibull Distribution

CS 2750 Machine Learning Lecture 5. Density estimation. Density estimation

A Markov Chain Competition Model

Chapter 3 Sampling For Proportions and Percentages

Transcription:

Statstcal modellg ad latet varables (2 Mxg latet varables ad parameters statstcal erece Trod Reta (Dvso o statstcs ad surace mathematcs, Departmet o Mathematcs, Uversty o Oslo

State spaces We typcally have a parametrc model or the latet varables, represetg the true state o a system. Also, the dstrbuto o the observatos may deped o parameters as well as latet varables. Observatos may ote be see as osy versos o the actual state o a system. Examples o states could be: Use gree arrows or oe-way parametrc depedecy (or whch you do t provde a probablty dstrbuto requetst statstcs. D 1. The physcal state o a rocet (posto, oretato, velocty, uel-state. 2. Real water temperature (as opposed to measured temperature. 3. Occupacy a area. 4. Carryg capacty a area.

Observatos, latet varables ad parameters - erece Sometmes we are terested the parameters, sometmes the state o the latet varables, sometmes both. Impossble to do erece o the latet varables wthout also dealg wth the parameters ad vce versa. Ote, other parameters aect the latet varables tha the observatos. D D D

Observatos, latet varables ad parameters M estmato A latet varable model wll specy the dstrbuto o the latet varables gve the parameters ad the dstrbuto o the observatos gve both the parameters ad the latet varables. Ths wll gve the dstrbuto o data *ad* latet varables: (D, =( (D, But a M aalyss, we wat the lelhood, (D! D Theory (law o total probablty aga: Pr( D Pr( D, Pr( D, Pr( or ( D ( D, d ( D, ( d

Observatos, latet varables ad parameters M estmato elhood: Pr( D or ( D Pr( D,, ( D, d ( D The tegral ca ote ot be obtaed aalytcally. Pr( D,, Pr( ( d I occupacy, the sum s easy (oly two possble states Kalma lter: For latet varables as lear ormal Marov chas wth ormal observatos depedg learly o them, ths ca be doe aalytcally. Alteratve whe aalytcal methods al: umercal tegrato, partcle lters, Bayesa statstcs usg MCMC.

Occupacy as a state-space model the model words Assume a set areas, (1,,A. Each area has a set o trasects. Each trasect has a depedet detecto probablty, p, gve the occupacy. Occupacy s a latet varable or each area,. Assume depedecy betwee the occupacy state deret areas. The probablty o occupacy s labelled. So, the parameters are =(p,. Pr( =1 =. Start wth dstrbuto o observatos gve the latet varable: Pr(x,j =1 =1, =p. Pr(x,j =0 =1, =1-p, Pr(x,j =1 =0, =0. Pr(x,j =0 =0, =1. So, or 5 trasects wth outcome 00101, we wll get Pr(00101 =1, =(1-p(1-pp(1-pp=p 2 (1-p 3. Pr(00101 =0, =1 1 0 1 0=0

Occupacy as a state-space model graphc model Parameters ( : p =occupacy rate p=detecto rate gve occupacy. Oe latet varable per area (area occupacy 1 2 3 A Pr( =1 = Pr( =0 =1- The area occupaces are depedet. x 1,1 x 1,2 x 1,3 x 1,1 Data: Detectos sgle trasects. Pr(x,j =1 =1, =p. Pr(x,j =0 =1, =1-p, Pr(x,j =1 =0, =0. Pr(x,j =0 =0, =1 The detectos are depedet *codtoed* o the occupacy. Importat to eep such thgs md whe modellg! PS: What we ve doe so ar s eough to start aalyzg usg WBUGS.

Occupacy as a state-space model probablty dstrbuto or a set o trasects Probablty or a set o trasects to gve >0 detectos a gve order s Pr( 1, p (1 p, Pr( 0, 0 whle wth o detectos Pr( We ca represet ths more compactly we troduce the detcato ucto. I(A=1 A s true. I(A=0 A s alse. The 0 Pr( 1, 0, (1 I( p 0, Pr( 0 0, 1 Wth o gve order o the detecto, we pc up the bomal coecet: Pr( 1, p (1 p, Pr( 0, I( 0 (Not relevat at all or erece. The or a gve dataset, the costat s just sttg there.

Occupacy as a state-space model area-specc margal detecto probablty (lelhood For a gve area wth a uow occupacy state, the detecto probablty wll the be (law o tot. prob.: Pr( Pr( 1, Pr( 1 Pr( 0, Pr( p (1 p I( 0(1 0 Bomal (p=0.6 Occupacy (p=0.6, =0.6 Occupacy s a zero-lated bomal model

Occupacy as a state-space model ull lelhood Each area s depedet, so the ull lelhood s: Pr( A 1 p (1 p I( 0(1 We ca ow do erece o the parameters, =(p,, usg M estmato (or usg Bayesa statstcs.

Occupacy as a state-space model occupacy erece Ierece o, gve the parameters, Pr( Pr( 1 1, Pr( Pr( 0, (1 (1 p 1 Pr( 1 Pr( p 1 1 (1 (Bayes theorem: 100% or 0 PS: We preted that s ow here. However, s estmated rom the data ad s ot certa at all. We are usg data twce! Oce to estmate ad oce to do erece o the latet varables. Avoded a Bayesa settg.