Inverse problems in statistics

Transcription

1 Inverse problems in statistics Laurent Cavalier (Université Aix-Marseille 1, France) YES, Eurandom, 10 October 2011 p. 1/27

2 Table of contents YES, Eurandom, 10 October 2011 p. 2/27

3 Table of contents 1) Inverse problems Introduction Linear inverse problems with random noise SVD and sequence space model Examples YES, Eurandom, 10 October 2011 p. 2/27

4 Table of contents 1) Inverse problems 2) Adaptation and oracle inequalities Regularization methods Classes of functions Rates of convergence Adaptation Oracle inequalities Unbiased risk estimation (URE) YES, Eurandom, 10 October 2011 p. 2/27

5 Table of contents 1) Inverse problems 2) Adaptation and oracle inequalities 3) Risk hull method Penalized empirical risk Problems with URE Risk hull method Oracle inequality Proof Simulations YES, Eurandom, 10 October 2011 p. 2/27

6 Introduction There exist many fields where inverse problems appear Astronomy (Hubble satellite). Econometrics (instrumental variables). Financial mathematics (model calibration). Medical image processing (X-rays). YES, Eurandom, 10 October 2011 p. 3/27

7 Introduction There exist many fields where inverse problems appear Astronomy (Hubble satellite). Econometrics (instrumental variables). Financial mathematics (model calibration). Medical image processing (X-rays). These are problems where we have indirect observations of an object (a function) that we want to reconstruct. YES, Eurandom, 10 October 2011 p. 3/27

8 Inverse problems Let H et G be Hilbert spaces. Let A be a continuous linear operator from H into G. YES, Eurandom, 10 October 2011 p. 4/27

9 Inverse problems Let H et G be Hilbert spaces. Let A be a continuous linear operator from H into G. Given g G find f H such that Af = g. YES, Eurandom, 10 October 2011 p. 4/27

10 Inverse problems Let H et G be Hilbert spaces. Let A be a continuous linear operator from H into G. Given g G find f H such that Af = g. Solving an inverse problem Inversion of the operator A. YES, Eurandom, 10 October 2011 p. 4/27

11 Inverse problems Let H et G be Hilbert spaces. Let A be a continuous linear operator from H into G. Given g G find f H such that Af = g. Solving an inverse problem Inversion of the operator A. If A 1 is not continuous the problem is called ill-posed. YES, Eurandom, 10 October 2011 p. 4/27

12 Inverse problems Let H et G be Hilbert spaces. Let A be a continuous linear operator from H into G. Given g G find f H such that Af = g. Solving an inverse problem Inversion of the operator A. If A 1 is not continuous the problem is called ill-posed. Observe g ε a noisy version of g, then f ε = A 1 g ε could be far from f. YES, Eurandom, 10 October 2011 p. 4/27

13 Inverse problems Let H et G be Hilbert spaces. Let A be a continuous linear operator from H into G. Given g G find f H such that Af = g. Solving an inverse problem Inversion of the operator A. If A 1 is not continuous the problem is called ill-posed. Observe g ε a noisy version of g, then f ε = A 1 g ε could be far from f. Importance of the notion of noise or error. YES, Eurandom, 10 October 2011 p. 4/27

14 Linear inverse problems Let H and G two separable Hilbert spaces. Let A be a known linear bounded operator from the space H to G. YES, Eurandom, 10 October 2011 p. 5/27

15 Linear inverse problems Let H and G two separable Hilbert spaces. Let A be a known linear bounded operator from the space H to G. Let the model : Y = Af +εξ, where Y is the observation, f H unknown, A a continuous linear operator fom H into G, ξ is a white noise, ε corresponds to the noise level. YES, Eurandom, 10 October 2011 p. 5/27

16 Linear inverse problems Let H and G two separable Hilbert spaces. Let A be a known linear bounded operator from the space H to G. Let the model : Y = Af +εξ, where Y is the observation, f H unknown, A a continuous linear operator fom H into G, ξ is a white noise, ε corresponds to the noise level. Reconstruct (estimate) f with the observation Y. YES, Eurandom, 10 October 2011 p. 5/27

17 Linear inverse problems Let H and G two separable Hilbert spaces. Let A be a known linear bounded operator from the space H to G. Let the model : Y = Af +εξ, where Y is the observation, f H unknown, A a continuous linear operator fom H into G, ξ is a white noise, ε corresponds to the noise level. Reconstruct (estimate) f with the observation Y. Projection of a white noise on any orthonormal basis {ψ k } gives a sequence of i.i.d. standard Gaussian random variables. YES, Eurandom, 10 October 2011 p. 5/27

18 Singular value decomposition A major property of compact operators is that they have a discrete spectrum. YES, Eurandom, 10 October 2011 p. 6/27

19 Singular value decomposition A major property of compact operators is that they have a discrete spectrum. Suppose A A compact operator with a known basis of eigenfunctions in H: A Aϕ k = b 2 k ϕ k. YES, Eurandom, 10 October 2011 p. 6/27

20 Singular value decomposition A major property of compact operators is that they have a discrete spectrum. Suppose A A compact operator with a known basis of eigenfunctions in H: A Aϕ k = b 2 k ϕ k. Singular Value Decomposition (SVD) of A : YES, Eurandom, 10 October 2011 p. 6/27

21 Singular value decomposition A major property of compact operators is that they have a discrete spectrum. Suppose A A compact operator with a known basis of eigenfunctions in H: A Aϕ k = b 2 k ϕ k. Singular Value Decomposition (SVD) of A : Aϕ k = b k ψ k, A ψ k = b k ϕ k, where b k > 0 are the singular values, {ϕ k } o.n.b. on H, {ψ k } o.n.b. on G. YES, Eurandom, 10 October 2011 p. 6/27

22 Singular value decomposition A major property of compact operators is that they have a discrete spectrum. Suppose A A compact operator with a known basis of eigenfunctions in H: A Aϕ k = b 2 k ϕ k. Singular Value Decomposition (SVD) of A : Aϕ k = b k ψ k, A ψ k = b k ϕ k, where b k > 0 are the singular values, {ϕ k } o.n.b. on H, {ψ k } o.n.b. on G. A linear bounded compact operator between two Hilbert spaces may really be seen as an infinite matrix. YES, Eurandom, 10 October 2011 p. 6/27

23 Projection on {ψ k } Projection of Y on {ψ k } : YES, Eurandom, 10 October 2011 p. 7/27

24 Projection on {ψ k } Projection of Y on {ψ k } : Y,ψ k = Af,ψ k +ε ξ,ψ k YES, Eurandom, 10 October 2011 p. 7/27

25 Projection on {ψ k } Projection of Y on {ψ k } : Y,ψ k = f,a ψ k +ε ξ,ψ k YES, Eurandom, 10 October 2011 p. 7/27

26 Projection on {ψ k } Projection of Y on {ψ k } : Y,ψ k = f,a ψ k +ε ξ,ψ k = b k f,ϕ k +ε ξ,ψ k YES, Eurandom, 10 October 2011 p. 7/27

27 Projection on {ψ k } Projection of Y on {ψ k } : Y,ψ k = f,a ψ k +ε ξ,ψ k = b k f,ϕ k +εξ k where {ξ k } standard Gaussian sequence i.i.d., by projection of a white noise ξ on the o.n.b. {ψ k }. YES, Eurandom, 10 October 2011 p. 7/27

28 Sequence space model Equivalent Sequence space model y k = b k θ k +εξ k, k = 1,2,..., where {θ k } coefficients of f, ξ k N(0,1) i.i.d., b k 0 singular values. YES, Eurandom, 10 October 2011 p. 8/27

29 Sequence space model Equivalent Sequence space model y k = b k θ k +εξ k, k = 1,2,..., where {θ k } coefficients of f, ξ k N(0,1) i.i.d., b k 0 singular values. Estimate θ = {θ k } with the observation Y = {Y k }. Use L 2 risk, it is equivalent to estimate f. YES, Eurandom, 10 October 2011 p. 8/27

30 Sequence space model Equivalent Sequence space model y k = b k θ k +εξ k, k = 1,2,..., where {θ k } coefficients of f, ξ k N(0,1) i.i.d., b k 0 singular values. Estimate θ = {θ k } with the observation Y = {Y k }. Use L 2 risk, it is equivalent to estimate f. Remark that b k 0 weaken the signal θ k. YES, Eurandom, 10 October 2011 p. 8/27

31 Sequence space model Equivalent Sequence space model y k = b k θ k +εξ k, k = 1,2,..., where {θ k } coefficients of f, ξ k N(0,1) i.i.d., b k 0 singular values. Estimate θ = {θ k } with the observation Y = {Y k }. Use L 2 risk, it is equivalent to estimate f. Remark that b k 0 weaken the signal θ k. Ill-posed problem. YES, Eurandom, 10 October 2011 p. 8/27

32 Inversion We have to invert in some sense the operator A. YES, Eurandom, 10 October 2011 p. 9/27

33 Inversion We have to invert in some sense the operator A. Thus, we obtain the model : where σ k = b 1 k. X k = b 1 k y k = θ k +εσ k ξ k, k = 1,2,... YES, Eurandom, 10 October 2011 p. 9/27

34 Inversion We have to invert in some sense the operator A. Thus, we obtain the model : X k = b 1 k y k = θ k +εσ k ξ k, k = 1,2,... where σ k = b 1 k. In the case where the problem is ill-posed the variance term grows to infinity. YES, Eurandom, 10 October 2011 p. 9/27

35 Inversion We have to invert in some sense the operator A. Thus, we obtain the model : X k = b 1 k y k = θ k +εσ k ξ k, k = 1,2,... where σ k = b 1 k. In the case where the problem is ill-posed the variance term grows to infinity. In this model the aim is to estimate {θ k } by use of {X k }. When k is large the noise in X k may then be very large, making the estimation difficult. YES, Eurandom, 10 October 2011 p. 9/27

36 Inversion We have to invert in some sense the operator A. Thus, we obtain the model : X k = b 1 k y k = θ k +εσ k ξ k, k = 1,2,... where σ k = b 1 k. In the case where the problem is ill-posed the variance term grows to infinity. In this model the aim is to estimate {θ k } by use of {X k }. When k is large the noise in X k may then be very large, making the estimation difficult. (see Donoho (1995), Mair and Ruymgaart (1996), Johnstone (1999) and C. and Tsybakov (2002)...). YES, Eurandom, 10 October 2011 p. 9/27

37 Difficulty of inverse problems YES, Eurandom, 10 October 2011 p. 10/27

38 Difficulty of inverse problems σ k 1 : Direct problem. YES, Eurandom, 10 October 2011 p. 10/27

39 Difficulty of inverse problems σ k 1 : Direct problem. σ k k β, β > 0 : Mildly ill-posed problem. YES, Eurandom, 10 October 2011 p. 10/27

40 Difficulty of inverse problems σ k 1 : Direct problem. σ k k β, β > 0 : Mildly ill-posed problem. σ k exp(βk), β > 0 : Severely ill-posed problem. YES, Eurandom, 10 October 2011 p. 10/27

41 Difficulty of inverse problems σ k 1 : Direct problem. σ k k β, β > 0 : Mildly ill-posed problem. σ k exp(βk), β > 0 : Severely ill-posed problem. Parameter β is called degree of ill-posedness. YES, Eurandom, 10 October 2011 p. 10/27

42 Examples There exist many examples of operators for which the SVD is known : Standard Gaussian white noise (any basis). YES, Eurandom, 10 October 2011 p. 11/27

43 Examples There exist many examples of operators for which the SVD is known : Standard Gaussian white noise (any basis). Convolution (Fourier) blurred images. YES, Eurandom, 10 October 2011 p. 11/27

44 Examples There exist many examples of operators for which the SVD is known : Standard Gaussian white noise (any basis). Convolution (Fourier) blurred images. Tomography (difficult basis) X-Rays. YES, Eurandom, 10 October 2011 p. 11/27

45 Examples There exist many examples of operators for which the SVD is known : Standard Gaussian white noise (any basis). Convolution (Fourier) blurred images. Tomography (difficult basis) X-Rays. Instrumental variables (Fourier) econometrics. YES, Eurandom, 10 October 2011 p. 11/27

46 Direct model A very specific inverse problem since, in this case, the operator is A = I. YES, Eurandom, 10 October 2011 p. 12/27

47 Direct model A very specific inverse problem since, in this case, the operator is A = I. However, most of the results on inverse problems will apply in this framework. YES, Eurandom, 10 October 2011 p. 12/27

48 Direct model A very specific inverse problem since, in this case, the operator is A = I. However, most of the results on inverse problems will apply in this framework. Model often called a direct model, since we have at our disposal direct observations and not indirect ones. YES, Eurandom, 10 October 2011 p. 12/27

49 Direct model A very specific inverse problem since, in this case, the operator is A = I. However, most of the results on inverse problems will apply in this framework. Model often called a direct model, since we have at our disposal direct observations and not indirect ones. In this case, the sequence space model may be obtained by projection on any orthonormal basis {ψ k }. YES, Eurandom, 10 October 2011 p. 12/27

50 Equivalence with nonparametric regression This model is an idealized version of the standard nonparametric regression : Y i = f(x i )+ξ i, i = 1,...,n, where (X 1,Y 1 ),..,(X n,y n ) are observed (we may assume X i [0,1]), f is an unknown function in L 2 (0,1), and ξ i are i.i.d. zero-mean Gaussian random variables of variance σ 2. YES, Eurandom, 10 October 2011 p. 13/27

51 Equivalence with nonparametric regression This model is an idealized version of the standard nonparametric regression : Y i = f(x i )+ξ i, i = 1,...,n, where (X 1,Y 1 ),..,(X n,y n ) are observed (we may assume X i [0,1]), f is an unknown function in L 2 (0,1), and ξ i are i.i.d. zero-mean Gaussian random variables of variance σ 2. Equivalence between the models (Brown and Low (1996), Nussbaum (1996)). YES, Eurandom, 10 October 2011 p. 13/27

52 Equivalence with nonparametric regression This model is an idealized version of the standard nonparametric regression : Y i = f(x i )+ξ i, i = 1,...,n, where (X 1,Y 1 ),..,(X n,y n ) are observed (we may assume X i [0,1]), f is an unknown function in L 2 (0,1), and ξ i are i.i.d. zero-mean Gaussian random variables of variance σ 2. Equivalence between the models (Brown and Low (1996), Nussbaum (1996)). Noise level is related to number of observations by ε 1/ n. YES, Eurandom, 10 October 2011 p. 13/27

53 Circular convolution The framework of deconvolution is perhaps one of the most well-known inverse problem. It is used in many applications as econometrics, physics, astronomy, medical image processing. For example, it corresponds to the problem of a blurred signal that one wants to recover from indirect data. YES, Eurandom, 10 October 2011 p. 14/27

54 Circular convolution The framework of deconvolution is perhaps one of the most well-known inverse problem. It is used in many applications as econometrics, physics, astronomy, medical image processing. For example, it corresponds to the problem of a blurred signal that one wants to recover from indirect data. Consider the following convolution operator Af(t) = r f(t) = 1 0 r(t x)f(x)dx, x [0,1], where r is a known 1-periodic symetric real convolution kernel in L 2 [0,1]. In this model, A is a linear bounded self-adjoint operator from L 2 [0,1] to L 2 [0,1]. YES, Eurandom, 10 October 2011 p. 14/27

55 Hubble satellite One famous example of an inverse problem of deconvolution is the blurred images of the Hubble space telescope. YES, Eurandom, 10 October 2011 p. 15/27

56 Hubble satellite One famous example of an inverse problem of deconvolution is the blurred images of the Hubble space telescope. The Hubble satellite was launched into low-earth orbit outside of the disturbing atmosphere in order to provide images. YES, Eurandom, 10 October 2011 p. 15/27

57 Hubble satellite One famous example of an inverse problem of deconvolution is the blurred images of the Hubble space telescope. The Hubble satellite was launched into low-earth orbit outside of the disturbing atmosphere in order to provide images. Unfortunately, a manufacturing error in the main mirror was detected, causing severe spherical aberrations in the images. YES, Eurandom, 10 October 2011 p. 15/27

58 Hubble satellite One famous example of an inverse problem of deconvolution is the blurred images of the Hubble space telescope. The Hubble satellite was launched into low-earth orbit outside of the disturbing atmosphere in order to provide images. Unfortunately, a manufacturing error in the main mirror was detected, causing severe spherical aberrations in the images. Therefore, astronomers employed inverse problem techniques (Richardson-Lucy algorithm) to improve the blurred images (see Adorf (1995)). YES, Eurandom, 10 October 2011 p. 15/27

59 Images of the Hubble satellite YES, Eurandom, 10 October 2011 p. 16/27

60 Blurred cameraman (a) (b) YES, Eurandom, 10 October 2011 p. 17/27

61 Blurred kangaroo YES, Eurandom, 10 October 2011 p. 18/27

62 Convolution model Define then the following model Y(t) = r f(t)+ε ξ(t), x [0,1], where Y is observed, f is an unknown periodic function in L 2 [0,1] and ξ(t) is a white noise on L 2 [0,1]. YES, Eurandom, 10 October 2011 p. 19/27

63 Convolution model Define then the following model Y(t) = r f(t)+ε ξ(t), x [0,1], where Y is observed, f is an unknown periodic function in L 2 [0,1] and ξ(t) is a white noise on L 2 [0,1]. The SVD basis is then clearly here the Fourier basis {ϕ k (t)}. YES, Eurandom, 10 October 2011 p. 19/27

64 Convolution model Define then the following model Y(t) = r f(t)+ε ξ(t), x [0,1], where Y is observed, f is an unknown periodic function in L 2 [0,1] and ξ(t) is a white noise on L 2 [0,1]. The SVD basis is then clearly here the Fourier basis {ϕ k (t)}. We make the projection on {ϕ k (t)}, in the Fourier domain, and obtain y k = b k θ k +εξ k, where b k = r(x)cos(2πkx)dx for even k, θ k are the Fourier coefficients of f, and ξ k are i.i.d. N(0,1). YES, Eurandom, 10 October 2011 p. 19/27

65 Computerized tomography In medical X-ray tomography one tries to have an image of the internal structure of an object. This image is characterized by a function f. However, there is no direct observations of f. YES, Eurandom, 10 October 2011 p. 20/27

66 Computerized tomography In medical X-ray tomography one tries to have an image of the internal structure of an object. This image is characterized by a function f. However, there is no direct observations of f. Suppose that one observes the attenuation of the X-rays. Denote by I 0 and I 1 the initial and final intensity, x is the position on a given line L and I(x) is the attenuation for a small x. One has then I(x) = f(x)i(x) x. YES, Eurandom, 10 October 2011 p. 20/27

67 Computerized tomography This corresponds from a mathematical point of view to I (x) I(x) = f(x), YES, Eurandom, 10 October 2011 p. 21/27

68 Computerized tomography This corresponds from a mathematical point of view to I (x) I(x) = f(x), and then by integration ( I1 ) log(i 1 ) log(i 0 ) = log I 0 = L f(x)dx. YES, Eurandom, 10 October 2011 p. 21/27

69 Computerized tomography This corresponds from a mathematical point of view to I (x) I(x) = f(x), and then by integration ( I1 ) log(i 1 ) log(i 0 ) = log I 0 = L f(x)dx. Thus observing I 1 /I 0 is equivalent to the observation of exp( f(x)dx). By measuring attenuation of X-rays, one L observes cross section of the body. YES, Eurandom, 10 October 2011 p. 21/27

70 Tomography scan YES, Eurandom, 10 October 2011 p. 22/27

71 Tomography reconstruction YES, Eurandom, 10 October 2011 p. 23/27

72 Radon transform This problem corresponds to the reconstruction of an unknown function f in IR d based on observations of its Radon transform Rf, i.e. integrals over hyperplanes Rf(s,u) = f(v)dv,, v: v,s =u where u [ 1,+1], s S d 1,S d 1 = {v IR d, v = 1} is the unit sphere in IR d. YES, Eurandom, 10 October 2011 p. 24/27

73 Radon transform This problem corresponds to the reconstruction of an unknown function f in IR d based on observations of its Radon transform Rf, i.e. integrals over hyperplanes Rf(s,u) = f(v)dv,, v: v,s =u where u [ 1,+1], s S d 1,S d 1 = {v IR d, v = 1} is the unit sphere in IR d. The Radon transform is a suitable tool for the problem of tomography, because Rf(s, u) represents the integral of f over the hyperplane {v IR d, v,s = u}. YES, Eurandom, 10 October 2011 p. 24/27

74 Tomography model The model is the following Y(s,u) = Rf(s,u)+εξ(s,u), s S d 1, u [ 1,+1], where ξ is a white noise. YES, Eurandom, 10 October 2011 p. 25/27

75 Tomography model The model is the following Y(s,u) = Rf(s,u)+εξ(s,u), s S d 1, u [ 1,+1], where ξ is a white noise. In this case, the Radon operator R is linear bounded and compact. YES, Eurandom, 10 October 2011 p. 25/27

76 Tomography model The model is the following Y(s,u) = Rf(s,u)+εξ(s,u), s S d 1, u [ 1,+1], where ξ is a white noise. In this case, the Radon operator R is linear bounded and compact. The SVD basis is known for the Radon transform. However, this basis is very difficult to compute. YES, Eurandom, 10 October 2011 p. 25/27

77 Tomography model The model is the following Y(s,u) = Rf(s,u)+εξ(s,u), s S d 1, u [ 1,+1], where ξ is a white noise. In this case, the Radon operator R is linear bounded and compact. The SVD basis is known for the Radon transform. However, this basis is very difficult to compute. There exist many different models of tomography (X-rays tomography, positron emission tomography, attenuated tomography, tomography in quantum physics and so on). The model are then very different, but linked to the Radon operator. YES, Eurandom, 10 October 2011 p. 25/27

78 Instrumental variables An economic relationship between a response variable Y and a vector X of explanatory variables is represented by Y i = f(x i )+U i, i = 1,...,n, where f has to be estimated and U i are the errors. YES, Eurandom, 10 October 2011 p. 26/27

79 Instrumental variables An economic relationship between a response variable Y and a vector X of explanatory variables is represented by Y i = f(x i )+U i, i = 1,...,n, where f has to be estimated and U i are the errors. This model does not characterize the function f if U is not constrained. The problem is solved if E(U X) = 0. YES, Eurandom, 10 October 2011 p. 26/27

80 Instrumental variables An economic relationship between a response variable Y and a vector X of explanatory variables is represented by Y i = f(x i )+U i, i = 1,...,n, where f has to be estimated and U i are the errors. This model does not characterize the function f if U is not constrained. The problem is solved if E(U X) = 0. In many structural econometrics models some components of X are endogeneous. YES, Eurandom, 10 October 2011 p. 26/27

81 Instrumental variables An economic relationship between a response variable Y and a vector X of explanatory variables is represented by Y i = f(x i )+U i, i = 1,...,n, where f has to be estimated and U i are the errors. This model does not characterize the function f if U is not constrained. The problem is solved if E(U X) = 0. In many structural econometrics models some components of X are endogeneous. If Y denotes wages and X, level of education, among other variables. The error U includes, ability, not observed, but influences wages. YES, Eurandom, 10 October 2011 p. 26/27

82 Instrumental variables An economic relationship between a response variable Y and a vector X of explanatory variables is represented by Y i = f(x i )+U i, i = 1,...,n, where f has to be estimated and U i are the errors. This model does not characterize the function f if U is not constrained. The problem is solved if E(U X) = 0. In many structural econometrics models some components of X are endogeneous. If Y denotes wages and X, level of education, among other variables. The error U includes, ability, not observed, but influences wages. High ability tends to have high level of education, then education and ability are correlated, and thus X and U also. YES, Eurandom, 10 October 2011 p. 26/27

83 Instrumental variables Nevertheless, suppose that we observe another set of data, W i where W is called an instrumental variable for which E(U W) = E(Y f(x) W) = 0. YES, Eurandom, 10 October 2011 p. 27/27

84 Instrumental variables Nevertheless, suppose that we observe another set of data, W i where W is called an instrumental variable for which E(U W) = E(Y f(x) W) = 0. This equation characterizes f by a Fredholm equation of the first kind. Estimation of the function f is in fact an ill-posed inverse problems. YES, Eurandom, 10 October 2011 p. 27/27

85 Instrumental variables Nevertheless, suppose that we observe another set of data, W i where W is called an instrumental variable for which E(U W) = E(Y f(x) W) = 0. This equation characterizes f by a Fredholm equation of the first kind. Estimation of the function f is in fact an ill-posed inverse problems. Not exactly our model of Gaussian white noise, but closely related. YES, Eurandom, 10 October 2011 p. 27/27

86 Instrumental variables Nevertheless, suppose that we observe another set of data, W i where W is called an instrumental variable for which E(U W) = E(Y f(x) W) = 0. This equation characterizes f by a Fredholm equation of the first kind. Estimation of the function f is in fact an ill-posed inverse problems. Not exactly our model of Gaussian white noise, but closely related. Since the years 2000, the framework of inverse problems has been the topic of many articles in the econometrics literature, see Florens (2003) and Hall and Horowitz (2005). YES, Eurandom, 10 October 2011 p. 27/27