Projection Methods for Model Reduction of Large-scale Systems

Transcription

1 Projection Methods for Model Reduction of Large-scale Systems Kyle A. Gallivan School of Computational Science and Information Technology Florida State University Support: NSF, DOE, IBM Corporation, Belgian Programme on Interuniversity Poles of Atraction 1

2 Topics Brief summary of previous work in computational science and engineering Model reduction of a system represented by: a differential equation a differential equation and data data, a parameterization and an optimization metric 2

3 Research Goal To understand the interaction and tradeoffs between the Algorithms Applications Architectures (software and hardware, small to large scale) to create systems that contribute to science and engineering 3

4 Activities develop efficient and reliable algorithms develop theory that yields algorithmic and implementation insights develop analytical and empirical modeling to identify the appropriate architecture to achieve performance goals adapt and tune the version of the algorithm for that architecture/application combination create proof-of-concept, research-grade, pre-production software assist in the improvement of the architectures (software and hardware) create software systems that facilitate all of the above look for similarity between applications for technology transfer 4

5 Example of multidisciplinary activity over time Algorithm Performance Compiler Code BLAS3 Evaluation kernel/cache/ parallel model Dense linear load/store window algebra model analysis Structured Perfect Club FALCON problems analysis Current: Cache models and iterative compilation Chains of Recurrences for performance restructuring 5

6 Technique Code Parallel Direct Method PY12M Reordering,Pivoting for multiclusters MCSPARSE, H* Itetative/Direct hybrid (nonsymmetric) EN-like methods PARASPAR Sparse Least Squares CIMGS LTI model reduction RK Family Large-scale Riccati-based stabilization TSQR Family Parallel MG BoomerAMG coarsening addition 6

7 Application/Architecture/Algorithm Interaction Perfect Club hand-tuned versions (constructs still relevant in restructuring) GFDL-IMGA Ocean Circulation model Mediterranean basin, multicluster mapping Electromagnetics Frequency selective screen simulation, extended model reduction algorithms to two-parameters and additional frequency dependece Circuit simulation Hierarchical relaxation (waveform, nonlinear, linear), extensive implementation analysis of memory system, control system and runtime library of Cedar starting point for model reduction 7

8 Current Application/Algorithm Interaction Geodesic searches for image recognition (Liu, Srivastava) Autofocus for synthetic aperture radar (Munson) Efficient high-fidelity evaluation of models with stochastic parameters (Hussaini) 8

9 Collaborators Model Reduction: P. Van Dooren, E. Grimme, A. Van den Dorpe Incremental Dominant Spaces: P. Van Dooren, Y. Chahlaoui, C. Baker Grassmann and Projection Search: A. Srivastava, X. Liu 9

10 Goal Efficiently approximate a large-scale dynamical system with a reduced-order model that is accurate in response and form. smaller in dimension and faster to use relative to some task. can be produced efficiently relative to the amount preprocessing time allowed by subsequent use 10

11 continuous-time ẋ(t) = G(x(t), u(t)) y(t) = H(x(t), u(t)) ẋ(t) = A(t)x(t)+B(t)u(t) y(t) = C(t)x(t)+D(t)u(t) ẋ(t) = Ax(t) + Bu(t) y(t) = Cx(t) + Du(t) discrete-time x(k + 1) = G(x(k), u(k)) y(k) = H(x(k), u(k)) x(k+1) = A(k)x(k)+B(k)u(k) y(k) = C(k)x(k) + D(k)u(k) x(k + 1) = Ax(k) + Bu(k) y(k) = Cx(k) + Du(k) Last two cases have extensively been studied 11

12 Linear Time Invariant Systems (Implicit) Eẋ(t) = Ax(t) + Bu(t) y(t) = Cx(t) + Du(t). u(t) R m, y(t) R p, x(t) R N, N >> m, p Find another system driven with the same input Ê ˆx(t) = ˆx(t) + ˆBu(t) ŷ(t) = Ĉˆx(t) + ˆDu(t), but ŷ(t) R p, ˆx(t) R n. Find low order model with small output error y(t) ŷ(t) 12

13 Uses accelerate simulation of a large system, e.g., exploiting large LTI interconnect subsystem in nonlinear ODEs for circuit simulation accelerate simulation of a large LTI system that represents a peturbation to a solution of interest for a large nonlinear system preprocessing to create a smaller computationally tractably problem that is solved and whose solution is lifted back to the large space, e.g., stabilizing feedback 13

14 What kind of approximations? Transfer functions T(s) = C(sI N A) 1 B + D, ˆT(s) = Ĉ(sI n Â) 1 ˆB + ˆD, Infinity norm T(.) ˆT(.). = sup ω T(jω) ˆT(jω) 2, Hankel norm Balanced truncation T(.) ˆT(.) 2 H. = σ 2 i (H) Apply a balancing state space transformation and truncate to leading n n block of transformed A (and conformally on other matrices) This is often used due to bound relative to the. norm. 14

15 Modal Approximation Suppose for simplicity that all the poles of T(s) are different. T(s) = κ (s β 1)...(s β N 1 ) (s α 1 )...(s α N ) γ 1 = γ N s α 1 s α N Idea : Keep the k dominant modes of the partial fraction expansion ˆT(s) = γ 1 s α γ k s α k. Advantages : Preserves stability, not always easy to determine which poles are dominant in an efficient manner for large systems 15

16 Rational interpolation / moment matching Let T(s) = C(sE A) 1 B. Consider the interpolation set I = {(σ 1, k 1 ),...,(σ r, k r )}. construct a reduced order transfer function ˆT(s) = Ĉ(sI k Â) 1 ˆB, that interpolates R(s) at σ i up to the k i -th Taylor expansion term 1 i r, 1 j k i : d j 1 {R(s)} dsj 1 = dj 1 s=σi ds j 1 { ˆR(s)} s=σi Ĉ(σ i I k Â) j ˆB = C(σ i I N A) j B 16

17 Complexity For dense systems these approximations require O(N 3 ) operations due to linear system solving singular value decomposition eigenvalue decomposition Not acceptable for large sparse systems We want to use Projection sparse linear system solvers sparse eigenvalue solvers sparse matrix equation solvers, e.g., Lyapunov, Sylvester 17

18 Asymptotic Waveform Expansion compute the moments T i construct the Padé approximation ˆT(s) via forming and solving a system of linear equations for parameters numerically unreliable to form moments resulting linear system is typically very ill-conditioned. standard approach for circuit simulation before mid 90s completely replaced by projection-based approaches 18

19 Modal approximation via projection Given T(s), ˆT(s) can be constructed by a projection technique : Take Z, V C N k with Z T V = I k such that [ ] AV = V diag α 1 α k [ ] Z T A = diag α 1 α k Z T ˆT(s) = CV (si k Z T AV ) 1 Z T B [ ] = Ĉ diag 1 1 s α 1 s α ˆB 1 γ 1 = γ k. s α 1 s α k 19

20 Padé, Projections, and Krylov ˆT(s) = Ĉ(sI k Â) 1 ˆB interpolates T(s) at I = {(0, 2k)} ĈÂ i ˆB = CA i B, 1 i 2k ˆT 1 ˆT2 ˆTk ˆT 2 ˆT 3... ˆT k = ˆT k ˆTk+1 ˆT2k 1 T 1 T 2 T k T 2 T 3... T k T k T k+1 T 2k 1 ĈÂ 1. ĈÂ k [ ] ˆB Â Â 1... = CA 1. CA k [ A A 1 B... ] Â = Z T AV 20

21 Projected input and output matrices ˆT 1 ˆT 2. ˆT k = ĈÂ 1. ĈÂ k ˆB = CA 1. CA k B = T 1 T 2. T k ˆB = Z T B Ĉ [ Â 1 ˆB... Â k ˆB ] = C [ A 1 B... A k B ] Ĉ = CV 21

22 Rational Lanczos and Arnoldi The projection matrices define Krylov spaces and the projected matrices can be computed efficiently via several iterative methods, e.g., Lanczos, Arnoldi etc. Â = Z T AV ˆB = Z T B Ĉ = CV I k = Z T V Im(V ) = K k (A 1, A 1 B) Im(Z) = K k (A T, A T C T ) 22

23 Multipoint is required g(iw) actual system rational Lanczos, k= pt Lanczos at w=0, k=15 1 pt Lanczos at w=1e5, k= frequency (w) 23

24 Brief history Background: Padé approximation for model reduction; Shamash, etc., 1975 Partial realization and Krylov subspaces; Gragg and Lindquist, 1983 Markov and Padé approximation and Krylov subspaces; Multipoint asserted but not developed; Villemagne and Skelton, 1987 Asymptotic Waveform Evaluation; Pillage, Rohrer, etc.,

25 Later papers use the Lanczos algorithm, a fast iterative way to form Krylov subspaces and force momement matching onto it. Lanczos-based Model Reduction: Structural Dynamics: Noor-Omid, Craig, etc., Control: Boley, Kasenally, Van Dooren, etc., Circuits: Gallivan, Grimme, Van Dooren; Feldman, Freund; etc.,

26 Unifying theory and algorithms for projection-based: Lanczos theory and algorithmic framework: Freund et al., present Complete rational Krlov family Grimme, Gallivan, Van den Dorpe, Van Dooren, 1997-present Balanced truncation for large scale LTI systems: Antoulas and Sorensen, current Generality of projection Genin, Van den Dorpe, Van Dooren and Gallivan, current 26

27 Basic Theorem of Projection Villemagne and Skelton; Grimme, Gallivan, Van den Dorpe, Van Dooren and K k=1 K k=1 ) K Jbk ((A σ (k) E) 1 E, (A σ (k) E) 1 B ImX K Jck ((A σ (k) E) T E T, (A σ (k) E) T C T) ImY T (j k) i = ˆT (j k) i, j k = 1, 2,...,J (k), where k = 1, 2,...K, J (k). = Jbk + J ck provided all of the moments exist, i.e., the matrices are nonsingular. 27

28 Including modal matching Note only inclusion is required in theorem so other bases can be added to gain satisfy constraints Theorem Let ImX i, ImY i be left and right invariant subspaces of the regular pencil (λe A) with given spectrum, then the reduced order pencil (λê Â) =. Y T (λe A)X has left (right) invariant subspace with the same spectrum if ImX i ImX ImY i ImY. This extension allows to incorporate matching poles of the original system into the reduced order system, e.g., to stabilize it. Approximation of DAE requires often that the reduced order system has the same algebraic conditions (these corresponds to an invariant subspace at 28

29 Practicality Extremely flexible characterization Rational Power Method, Dual Rational Arnoldi, Rational Lanczos are easily defined as members of a general Rational Krylov family. Parallelism available at many levels Efficiency requires : sparse approximations to linear system solutions lose exact moment matching but Galerkin conditions hold exploit shifted form of systems for preconditioning 29

30 Practicality Point Placement and Selection: choose the next point at which a moment is added carefully causes synchronization postprocessing can be used to get rid of some of the unnecessary basis vectors added increases parallelism but adds cleanup operations clustered version of these two is natural 30

31 Practicality Use error estimate (.). = ˆT(.) T(.) ˆT(.) is our current approximation and T(.) is a second approximation of T(s), based on adding interpolation points to that of ˆT(.) or on interlacing points. Point selection based on the frequency response (ω) real points simpler than complex points and yield general trends and behavior in left half plane imaginary points get local information best mix is often best Stopping criterion is based on H norm of (.). 31

32 Stability Stability/Passivity of ˆT(s) may require further projection on the reduced system, e.g., restarted Arnoldi (Grimme, Sorensen, Van Dooren). Stability/Passivity can be guaranteed when imposing only k moments to match and using the other degrees of freedom to yield a passive reduced order model (Freund,Feldman) Recent work by Sorensen shows that constraining the interpolation points appropriately guarantees passivity is preserved but it does not control approximation error a hybrid using those constraints and extra Rational Krylov steps may solve the problem. 32

33 Comparison Optimal and rational approximations 15th order approximation of 120 th order CD player Frequency responses in db th order Hankel norm approximation Legend : Optimal Hankel norm approximation 33

34 Frequency responses in db th order balanced truncation Legend : - - Balanced truncation approximation 34

35 Frequency responses in db th order RK with points 1,2,2,2,2,2,2,2,2,3,4,4,5,5,Inf Legend : Rational Krylov approximation 35

36 Approximate Balanced Truncation Antoulas and Sorensen Motivated by the fact that Hankel singular values drop off rapidly Avoids the need to specify interpolation conditions Targeted for black box software Large sparse Sylvester equation solver ARPACK attempts to get at the dominant spaces of G o and G c no convergence proof no rigorous error bound works very well in practice 36

37 The basic problem A T G o + G o A + C T C = 0 and AG c + G c A T + BB T = 0.. where G c = CC T., G o = O T O, [ O = C T A T C T (A T ) 2 C T [ ] C = B AB A 2 B ] Given approximations V in and Z in to bases of the dominant spaces, V up G c V in and Z up G o Z in without knowing G c or G o. 37

38 A key kernel (Sorensen and Zhou) Given approximations V in and Z in to bases of the dominant spaces, V up G c V in and Z up G o Z in by solving AV up F T r + EV up Q T r + BH T r = 0 A T Z up F l + E T Z up Q l + C T H l = 0 where F l, Q l and H l are functions of Z in and F r, Q l and H l are functions of V in A, E C N N, B C N m, C C p N, F r,l, Q r,l C n n, H r C n m. and H l C n p A series of such projectors are produced. What are they? 38

39 Key question We have Projections producing rational interpolation Projections producing modal approximations A series of Sylvester equations producing approximate balanced truncation based on approximating the Gramians Are they related or fundamentally different approaches? 39

40 Sylvester and rational interpolation (SISO) Suppose V and Z solve AV F T r + EV Q T r + BH T r = 0 A T ZF l + E T ZQ l + C T H l = 0 A, E C N N, B C N 1, C C 1 N, F r,l, Q r,l C n n, H r C n 1. and H l C n 1 [ ] [ ] Suppose also sf r Q r H r and sfl T Q T l Hl T are full rank for all s. (This and SISO imply all their generalized eigenvalues have single Jordan blocks.) 40

41 Equivalence Theorem The reduced-order system ˆT(s) = Ĉ(sÊ Â) 1 ˆB obtained by using projectors that have the same image V and Z is such that T(s) ˆT(s) = O(s + σ k ) b k+c k 1 k K, where σ k is a generalized eigenvalue of (F r, Q r ) and (F l, Q l ), whose single Jordan blocks are of size b k and c k respectively provided se A and sê Â are invertible at σ k. 41

42 Interpolation and projection (SISO) For SISO minimal T(s) of McMillan degree N and minimal ˆT(s) of McMillan degree n projection and interpolation are equivalent! T(s) = C(sI N A) 1 B ˆT(s) = Ĉ(sI n Â) 1 ˆB, n < N = Z, V C N n, Z T V = I n Z T AV = Â, Z T B = ˆB, CV = Ĉ. Sketch of the proof : If Mc Millan degree E(s) 2n = Multipoint Padé, Otherwise = common poles = Multipoint Padé + Modal Truncation. Consequence : ˆT(s) interpolates T(s) ˆT(s) is a good choice! 42

43 There is always a Sylvester equation! Part (F 1, 0): AV 1 + V 1 F 1 = 0 = Modal Approximation Part (F i, G i ) observable (each F i is one Jordan block): λ [ ] AV i + V i... + B = 0 1 λ ( Im(V i ) = K k (A + λi) 1, (A + λi) 1 B ) Observable part of (F, G) Interpolation Non Obs. part of (F, G) Modal Approximation 43

44 SISO Consequences The series of Sylvester equations in the approximate balanced truncation produce a series of rational interpolation approximations There the interpolation conditions are implicitly specified by controlling the choice of F r, Q r F l and Q l! Controlling the choice of F r, Q r F l and Q l by specifying the spectra in Jordan form yields a Rational Krylov method directly from the recursion to solve the Sylvester equation!! Sylvester equations and the shift-and-invert based Rational Krylov algorithms are two algorithmic approaches that produce the same reduced order models. 44

45 What about MIMO? Check Sylvester Let V =. [ ] v 1 v n and y i C m 1 with all Jordan blocks of size 1. λ 1 [ ] AV + V... + B y 1 y n = 0 Choose Z so Z T V = I n and construct ˆT(s) It follows that for any 1 i n v i = (λ i I n A) 1 By i T(λ i )y i = ˆT(λ i )y i λ n 45

46 Rational Interpolation : match T (j) (σ) = ˆT (j) (σ) j = 1,, k σ Left Tangential Interpolation : [ ] [ ] ξ 1 ξ 2 T(σ) = ξ 1 ξ 2 ˆT(σ) ξ 1 t 11 (σ) + ξ 2 t 21 (σ) = ξ 1ˆt 11 (σ) + ξ 2ˆt 21 (σ) ξ 1 t 12 (σ) + ξ 2 t 22 (σ) = ξ 1ˆt 12 (σ) + ξ 2ˆt 22 (σ) Right Tangential Interpolation : T(σ) µ 1 µ 2 = ˆT(σ) µ 1 µ 2 Two-Sided Tangential Interpolation combines both 46

47 Tangential Interpolation If the range of [ (λi A) 1 B (λi A) k B ] η 0 η k η 0 is in Im(V ) then the vector polynomial y(s) = k 1 i=0 η i(s λ) i satisfies a right tangential interpolation condition (T(s) ˆT(s))y(s) = O(s λ) k (essentially this requires a nontrivial Jordan block for λ) Left tangential interpolation conditions can be derived similarly. 47

48 Tangential interpolation and projection/sylvester Tangential interpolation via projectors produced from a Sylvester equation is a powerful MIMO model reduction tool. It may be the key that allows a rigorous understanding of approximate balanced truncation and rational Krylov-based approaches for MIMO and facilitate their unification. It is possible to show that it is not always possible to find projectors that produce ˆT(s) from T(s) in the case of MIMO systems key restrictions present for SISO are lost. So tangential interpolation and projection are not universal for MIMO. However, Van den Dorpe and Genin have conjectured that any ˆT(s) of degree n can be produced from any T(s) of degree N via tangential interpolation defined projectors for practical situations in large-scale model reduction, i.e., when N n is sufficiently large. 48

49 Conclusions for equation-based model reduction We have successfully transferred techniques from the linear algebra and control communities into the circuit simulation community the Rational Krylov approaches are now standard. projection-based approaches provide a flexible family of model reduction methods We have extended, clarified and unified the theoretical understanding of the problem in a way that includes algorithmic and implementation insight, i.e., our theory guides our efficient algorithm design. We have aided in the dispersal of the updated knowledge and algorithms to other applications, e.g., electromagnetics application by Michielssen and Weile. 49

50 Conclusions for equation-based model reduction The recent advances in understanding the generality and links to other approaches promise a new round of advances from an algorithmic and library point of view. Progress is needed on the specification of the interpolation conditions required to satisfy both error requirements and property preservation Combined implicit and explicit specification strategies that use Sylvester and Rational Krylov methods look very promising. We are currently thinking about how to apply this to model reduction in a closed-loop control setting and a linear time-varying setting 50

51 Incremental tracking of dominant singular spaces Given A m n, approximate by a rank k factorization B m k C k n min A BC 2, k m, n C A B Applications in : Image compression Information retrieval Image recognition Model reduction (P.O.D) 51

52 Idea: Windowing Track dominant spaces with a sequence of windows SVD s of dimension m (l + k) k R V T A = 2 1 = U +E k + l 1: expand by appending l columns (Gram Schmidt) 2: contract by deleting l columns (SV D update) Total cost: 10mnk (Givens) or 8mnk (Householder) instead of O(mn 2 ) 52

53 Details (for l = 1) of step i, i = k + 1,...,n PSfrag Expand: appending column a + (Gram Schmidt): 4mk flops R ˆr V T U û ˆρ 1 = Û ˆRˆV T Downdate: removing smallest singular value of ˆR: 6mk flops R + 0 T V+ µ i U + = (ÛG u) (G T u ˆRG v ) (G T v ˆV T ) 53

54 P 3 P 2 P 1 ρ ρ ρ ρ η 1 φ 1 ρ ρ ρ η 2 0 φ 2 ρ ρ η φ 3 ρ η 4 Z 1 Z 2 Z 3 = ρ ρ ρ ρ ρ ρ µ 4 1 The elimination and fill-in structure for the two-sided algorithm with k = 3. (Z i eliminates η i 54

55 An evolution equation ẋ = F(x) is replaced by a reduced order equation ȧ = U H k F(U k a) = f(a) information can be recovered by integrating the reduced order equation rather than interpolating between saved states (columns of A) If V T k is tracked then U kσ k V T k e i x i can be used instead of integration. The form of the differential equation influences the cost of the production of the reduced order system and whether or not moving between the reduced state space and the original state space can be avoided 55

56 Perturbation Theorem The recursive algorithm produces approximate matrices V (i), Q (i) and R (i) that satisfy exactly the perturbed equation [A(:, 1 : i) + E] V (i) = Q (i) R(i), ( V (i) + F) T ( V (i) + F) = I k, with the bounds (up to O(u 2 ) terms) : E F ɛ e A 2, ɛ e 26k 3/2 nu, F F ɛ f 9k 3/2 nu. and in practice ɛ e 26k 2 u, ɛ f 9k 2 u. Note these bounds do not depend on m, the largest dimension of A 56

57 Orthogonality Theorem Let (a given matrix) V R n k select k columns of the matrix A R m n, and let A V = QR, Q T Q = I k, with R upper triangular, be its exact QR factorization. Let A V + G = Q R, G F = ɛ g A 2 u A 2, (1) be a computed version, where Q = Q + Q, R = R + R. Then under a mild assumption, we can bound the loss of orthogonality in Q as follows: Q T Q Ik F 2ɛ g κ 2 (R)κ R (A V ) 2ɛ g κ 2 2(R), ɛ g u. 57

58 Estimate quality of approximation Quality of rank k approximation  = Û ˆΣˆV T of A = UΣV T estimated by : Canonical angles : cosθ k. = U T (:, k)û 2, cosϕ k. = V T (:, k)ˆv 2 tanθ k tan ˆθ k. = ˆµ 2 Singular values : Approximation error : (ˆσ 2 k ˆµ2 ), tanϕ k tan ˆϕ k. = ˆµˆσ 1 (ˆσ 2 k ˆµ2 ) ˆσ i σ i ˆµ2 2ˆσ i E 2. = µ σk+1 ˆµ = max µ i µ (true error) 58

59 Gap γ : , σ k+1 = true sv s σ i (A), approximated sv s ˆσ (n) 1,..., ˆσ (n) k, dismissed sv s µ k+1,... µ n < σ k+1 σ 1 = ˆσ 1 = σ 2 = ˆσ 2 = σ 3 = ˆσ 3 = σ 4 = ˆσ 4 = σ 5 = ˆσ 5 = µ = ˆµ = cos θ k = cos ˆθ k = cos φ k = cos ˆφ k =

60 Gap γ : , σ k+1 = true sv s σ i (A), approximated sv s ˆσ (n) 1,..., ˆσ (n) k, dismissed sv s µ k+1,... µ n < σ k+1 σ 1 = ˆσ 1 = σ 2 = ˆσ 2 = σ 3 = ˆσ 3 = σ 4 = ˆσ 4 = σ 5 = ˆσ 5 = µ = ˆµ = cos θ k = cos ˆθ k = cos φ k = cos ˆφ k =

61 How quickly do we track the subspaces? Gap γ : Gap γ : How cos θ (i) k evolves with the time step i 61

62 Information retrieval words n = O(10 3 ) U Low rank approximation useful for: pages m = O(10 6 ) A L k = O(10 1 ) Low memory requirement O(k(m + n)) Fast queries Ax L(U x) in O(k(m + n)) time Approximation obtained in O(kmn) time using windowing 62

63 Image sequences Each column of A is one image Original : m = 28341, n = 100 Approximation : k = 6 63

64 Image sequences Each column of A is one image Original : m = 28341, n = 100 Approximation : k = 6 64

65 Image sequences Each column of A is one image Original : m = 28341, n = 100 Approximation : k = 6 65

66 Image sequences Each column of A is one image Original : m = 28341, n = 100 Approximation : k = 6 66

67 Related Work Efficient Arrowhead - Chandrasekaran, Manjunath, Wang, Winkeler, and Zhang, Graphical Models and Image Processing Sequential Karhunen-Loeve (SKL) - Levy and Lindenbaum, 2000, IEEE Trans. Image Processing. Incremental SVD (IncSVD) - Chahlaoui, Gallivan, and Van Dooren,2000 CIRO 2000, to appear SIMAX. 67

68 Complexity Algorithm One-sided Two-sided SKL 2mn k2 +3kl+2l 2 l 2mn k2 +3kl+2l 2 l + n2 k 2 +n 2 kl l IncSVD (G) 10mnk + 4mnl 10mnk + 4mnl + 3n 2 k IncSVD (HH) 8mnk + 4mnl 8mnk + 4mnl + n2 k 2 +n 2 kl l 68

69 Currrent and Future Work Finalizing empirical and analytical investigation of dynamic block size and loss of orthogonality Beginning study of PODs to represent perturbed trajectories when analyzing trajectories of differential equations with stochastic parameters. Beginning algorithm/architecture interaction study for incremental tracking on latest graphics attached processors for problems similar to those address by SKL 69

70 Searching on the Grassmann manifold for projectors Collection of images D R n grouped into classes a priori Training data selected by human to represent essential characteristics of each class Seeking a subspace with dimension d with U R d to encode the images U T D = C. Standard bases often depend upon statistical assumptions that may not be accurate enough for given data and that may have nothing to do with the metric for recognition We search the Grassmann manifold via a stochastic gradient method to optimize the recognition metric on the test data We assume that substantial amounts of off-line preprocessing are allowed in order to improve on-line recognition. 70

71 Motion on the Grassmann manifold Assume you have a basis S 0 for a space S and you wish to move along the Grassmann manifold. The trajectory can be given in terms of bases but only their spaces are of interest. S(t) = Qe 0 B T B 0 Q T S 0 Q T S 0 = I d 0 B R n d d are the directional velocities and represent the degrees of freedom of motion on the manifold from a given point. This can be evaluated in O(nd 2 ) for each time point. 71

72 Recognition versus t 100 Recognition rate (%) Iterations n = 10304, (that is ), d = 5, k train = 5, and k test = 5. X 0 = U PCA 72

73 Recognition versus t 100 Recognition rate (%) Iterations n = 10304, (that is ), d = 5, k train = 5, and k test = 5. X 0 = U ICA 73

74 Recognition versus t 100 Recognition rate (%) Iterations n = 10304, (that is ), d = 5, k train = 5, and k test = 5. X 0 = U FDA. 74