IMPLICIT COUPLING FRAMEWORK FOR MULTI-PHYSICS RESERVOIR SIMULATION

Transcription

1 IMPLICIT COUPLING FRAMEWORK FOR MULTI-PHYSICS RESERVOIR SIMULATION A DISSERTATION SUBMITTED TO THE DEPARTMENT OF ENERGY RESOURCES ENGINEERING AND THE COMMITTEE ON GRADUATE STUDIES OF STANFORD UNIVERSITY IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY Ruslan Rin October 2017

2 2017 by Ruslan Rin. All Rights Reserved. Re-distributed by Stanford University under license with the author. This work is licensed under a Creative Commons Attribution- Noncommercial 3.0 United States License. This dissertation is online at: ii

3 I certify that I have read this dissertation and that, in my opinion, it is fully adequate in scope and quality as a dissertation for the degree of Doctor of Philosophy. Hamdi Tchelepi, Primary Adviser I certify that I have read this dissertation and that, in my opinion, it is fully adequate in scope and quality as a dissertation for the degree of Doctor of Philosophy. Pavel Tomin, Co-Adviser I certify that I have read this dissertation and that, in my opinion, it is fully adequate in scope and quality as a dissertation for the degree of Doctor of Philosophy. Louis Durlofsky Approved for the Stanford University Committee on Graduate Studies. Patricia J. Gumport, Vice Provost for Graduate Education This signature page was generated electronically upon submission of this dissertation in electronic format. An original signed hard copy of the signature page is on file in University Archives. iii

4 Abstract The growth in the complexity of the subsurface flow processes of interest and the continuous increase in computational power have created the need for simulation frameworks that can be extended to handle multi-physics problems. Having a generalized framework reduces the development cost of incorporating additional physical mechanisms and exploring new numerical formulations and solution algorithms. In addition, a general simulation framework is necessary as we the transition from the existing paradigm, in which a small number of specific process models are coupled in a very limited number of predefined combinations, to a new paradigm, where many different nonlinear processes can be coupled in arbitrary permutations. The flexibility of such a framework allows us to investigate new nonlinear coupling configurations across the different physics processes and improve our understanding of the nature of the interactions. Such a flexible simulation framework will allow us to improve the robustness and computational efficiency of modeling coupled systems by finding new configurations and more efficient solvers for a given physics process. The goal of this work is the development and implementation of a general sequential reservoir simulation framework for multi-physics problems. The limitations associated with existing general-purpose platforms include: (1) Limited coupling strategies for a predetermined small set of physical processes. (2) The absence of accurate and consistent comparisons between the different coupling strategies. (3) Highly specialized linear solvers for a predefined set of physical processes. (4) Limited paths for the extension to new physics and additional complex features. In contrast to the existing platforms, the new framework can accommodate any coupling strategy across the different nonlinear physics modules. This flexibility is achieved using a sub-problem tree structure and a global status table. These components are integrated with the subsets iv

5 through an automatic differentiation layer (library). The exploration of different coupling strategies facilitates the study of new multi-physics preconditioners. The modular design with transparent subset-based interfaces greatly improves code extensibility. In addition, we developed a unique mapping operators concept that enables advanced coupling strategies, including (parallel) domain-decomposition and multiscale approaches. We have designed, developed, implemented, and tested a new GENeral Implicit Coupling framework for multi-physics problems (GENIC) that allows for rapid prototyping and customization. This GENIC framework is the next generation of the Automatic-Differentiation General- Purpose Research Simulator (AD-GPRS) at Stanford. GENIC represent a flexible and efficient computational laboratory for advanced reservoir simulation that allows researchers to try their ideas rapidly and specialize the preferred strategies efficiently. We demonstrate the power and flexibility of the GENIC framework in modeling complex multi-physics problems using several nonlinear problems that involve several mechanisms of flow and transport, and we show that the GENIC framework allows for investigating the full range of possible coupling strategies. We explore the use of existing and novel sequential coupling schemes for flow-thermal problems (i.e., multi-component multiphase flow and transport of mass and energy). We investigate the convergence properties of three different schemes: constant pressure, constant density, and a hybrid scheme where constant pressure is used for single-phase blocks, and constant density is used for two-phase blocks. The hybrid approach performed the best out of the three schemes tested. This hybrid method was able to avoid the divergent, or slow converging, behavior in the singlephase (constant density) and the two-phase (constant pressure) regions. Next, we present a new approach for coupling flow and thermal displacement processes modified sequential fully implicit (m-sfi) method. We demonstrate m-sfi method for several challenging examples where standard sequential approaches fail; specifically, we show that m-sfi takes a similar number of time steps as the full implicit method (FIM). In cases where accurate front prediction is possible and the two-phase region is limited, such as strictly injection or production problems, the m-sfi approach outperforms FIM. However, when most of the reservoir has two phases, the overall performance of the m-sfi method is more expensive than FIM due to the increased cost of the sequential iterations. We then demonstrate the applicability of the GENIC framework for flow-mechanics v

6 problems using three examples: (1) Mandel s problem; (2) Dean s problem; (3) an extended SPE 10 problem. We tested two existing coupled properties : fixed-stress and fixed-strain. The results are consistent with published results, where the fixed-strain approach is only conditionally stable and the fixed-stress is unconditionally stable. In addition, we compare the performance of the sequential coupling strategy with FIM for the extended SPE 10 problem. For moderate and weak coupling, the sequential approach performs better than FIM. However, strongly coupled problems where both physics are highly nonlinear poses remains an open challenge for sequential schemes. We conclude with a solution for a flow-thermal-mechanics problem using three different coupling strategies: fully coupled, sequentially coupled, and nested sequentially coupled. This is the first example - to our knowledge - where both fully coupled and sequentially coupled approaches are applied to this three-physics problem. When the mechanics problem is linear, and the coupling strength is weak, the sequential coupling strategy performs faster than FIM; however, as the coupling strength involving mechanics increases, sequential coupling strategies become less effective. vi

7 Acknowledgments I would like to thank my advisor, Hamdi Tchelepi, for his help and constant encouragement. Hamdi has provided me with latitude and freedom to explore ideas and direct my research. He has taught me how to work independently and think creatively and has helped me to grow as a researcher. More importantly, Hamdi has supported me during a hard time in Stanford when I took an academic leave of absence, and he encouraged me to come back to pursue my Ph.D. degree. Throughout the entire process, he has found a way to foster my growth and development through insightful research advice, kind words, and in a multitude of other ways, and I am deeply grateful for his generosity and unconditional support. My second advisor, Pavel Tomin, played a key role in both my professional and personal development. I am most grateful for the strong professional relationship and friendship we were able to build. I thank Pavel for spending countless hours working together on the foundation of the new research platform. His hard work and passion for research have had a profound impact on my motivation to do the best research possible. Pavel contributed to virtually every single aspect of this work, and it would have been impossible to accomplish it without his advice. I would like to thank Denis Voskov for his help and support during my M.S. research, and serving as co-adviser in the early stages of my Ph.D. program. Denis greatly contributed to my professional development and choice of research area. Many thanks to Professor Lou Durlofsky and Professor Khalid Aziz for reading my dissertation and providing valuable comments and suggestions. I also thank Professor Biondo L. Biondi for being the chair during my defense. I would like to acknowledge Timur Garipov for valuable discussions on geomechanics, Yang Wong for collaboration in the development of thermal physics, Kirill Terekhov for his contribution in the idea of subsets, vii

8 and Eiko Rutherford, Joanna Sun and Sandy Costa for providing administrative assistance during my stay in the ERE department. Special thanks to Michael Connolly for providing extremely valuable feedback on my dissertation. My wonderful time at Stanford was enriched by colleagues and friends at Stanford, including Sergey Klevtsov, Karine Levonyan, Yongduk Shin, Sumeet Trehan, Waqas Ali, Rustem Zaydullin, Jincong He, Motonao Imai, Da Huo, Mehrdad Shirangi, Elnur Aliyev, Amir Salehi, Nicola Castelletto, Oleg Volkov, Jason Chiang, Yimin Liu, and Olga Fuks. Most importantly, I want to thank my parents, Nuriya and Mansur, for their support and constant encouragement, and my wife, Yuliya, for her love, our son, and endless support. viii

9 Contents Abstract Acknowledgments iv vii 1 Introduction Multi-Physics Problems Coupling Methods for Multi-Physics Problems Current Progress in Multi-Physics Simulator Development Outline General Implicit Coupling Framework (GENIC) Mathematical Formulation Multi-Component Transport Conservation of Energy Momentum Balance Wells Coupling Strategies Framework Design Subsets Sub-Problem Tree Structure and Global Status Table Coupling Terms Coupled Properties Mapping Operators Solving Algorithm ix

10 3 Numerical Applications Flow and Thermal Coupling Example 1. Avdonin s Problem Example 2. Injection Flash Front Summary Modified SFI for Flow-Thermal Problems Example 3. 3D m-sfi Case Example 4. m-sfi SPE Summary Flow and Mechanics Coupling Example 5. Mandel Problem Example 6. Depletion of a Box-Shaped Reservoir (Deans Problem) Example 7. Water Flooding Problem Summary Coupled Flow, Thermal, and Mechanics Example 8. Cold Water Injection Summary Future Extensions of GENIC Automatic Coupled Properties Parallelization Multi-Scale Framework Conclusions and Future Work Conclusions Recommendations for Future Work Nomenclature 111 Bibliography 115 A Programming Model of AD-GPRS 133 A.1 The Structure of AD-GPRS x

11 A.2 Flow Sequence B Two-Phase Prototype 143 xi

12 List of Tables 2.1 Example of physics modules (sub-problems), their hypothetical subset properties and corresponding unknowns Example of a variable set for three-phase black-oil problem expanded by inactive status Parameters for Avdonin s problem Parameters for Injection Flash Front problem Single-phase injection flash front. Comparison between sequential coupling schemes, such as Constant Pressure (CP), Constant Density (CD), Hybrid (H) and fully coupled method (FIM) Two-phase injection flash front. Comparison between sequential coupling schemes, such as Constant Density (CD), Hybrid (H) and fully coupled method (FIM). Note: Constant Pressure does not converge for two-phase case Parameters for 3D model for m-sfi study m-sfi 3D case for blocks. Comparison between different coupling schemes: FIM, SEQ, m-sfi and MSFI-N m-sfi 3D case for low grid resolution (L) and finer grid resolution (H). Comparison between different coupling schemes: FIM, SEQ, m-sfi and MSFI-N Parameters for SPE 10 model for m-sfi study m-sfi SPE 10 upscaled case with top four layers for grid. Comparison between different coupling schemes: FIM, m-sfi and MSFI-N. 74 xii

13 3.10 m-sfi SPE 10 case for very coarse grid resolution (L) and original grid resolution (top four layers) (H). Comparison between different coupling schemes: FIM, m-sfi and MSFI-N Rock and fluid properties used for Mandel s problem Mandel s problem. Convergence between fixed-strain and fixed-stress methods, and FIM Rock properties for Dean s model Number of cells and total degrees of freedom (DOF) for the simulation cases Normalized simulation time for Water Flooding Problem Main parameters for Cold Water Injection problem Cold Water Injection case. Comparison between FIM and SEQ schemes.. 95 A.1 Sub-problems, subsets and properties xiii

14 List of Figures 2.1 Key principles of the GENIC framework Schematic representation of fully implicit coupling Schematic representation of mixed implicit coupling Example of possible coupling strategies for a flow-thermal-mechanics problem Example of a flow sub-problem made up of two gridblock with subsets and corresponding unknowns Example of mixing formulations between flow and thermal sub-problems Activation logic of subsets for different space discretization schemes for a flow sub-problem Example of flow-thermal-mechanics problem Subset applications for domain decomposition and multiscale Example of extended fixed-stress split strategy for a flow-thermal-mechanics problem Subset and subproblem activation state Subset activation logic for different subsets Example 1 for sub-problem tree and corresponding global status table Example 2 for sub-problem tree and corresponding global status table Example 3 for sub-problem tree and corresponding global status table Example of a coupled property Modified sequential implicit (m-sfi) approach for a flow-thermal problem Schematic representation of m-sfi approach Extension of global status table concept for mapping operators xiv

15 2.20 Example of global status table for m-sfi coupling strategy Schematic representation of Avdonin s problem (uniform linear spacing) Avdonin s problem. Comparison of the analytical and numerical solutions of the temperature distribution and the temperature evolution Avdonin s problem. Sequentially implicit method convergence profile after the first time step t = days Schematic representation of Injection Flash Front problem (uniform radial spacing) Single-phase injection flash front. Pressure and temperature distribution after 100 days (50 grid cells) Single-phase injection flash front. Pressure and temperature profile at x = 167 m (50 grid cells) Grid refinement for single-phase radial one-dimensional injection scenario. Pressure and temperature distribution after 100 days Grid refinement for single-phase radial one-dimensional injection scenario. Pressure and temperature profile at x = 167 m Two-phase radial one-dimensional injection scenario. Pressure and temperature distribution after 100 days (50 grid cells) Two-phase radial one-dimensional injection scenario. Pressure and temperature profile at x = 167 m (50 grid cells) Grid refinement for two-phase radial one-dimensional injection scenario. Pressure and temperature distribution after 100 days Grid refinement for two-phase radial one-dimensional injection scenario. Pressure and temperature profile at x = 167 m Schematic of the three-dimensional model D m-sfi case. Pressure and temperature profile for grid at production well block D m-sfi case. Pressure (a) and temperature (b) distribution for grid after 10 years of production at 16th layer xv

16 3.16 3D m-sfi case. Relative error between FIM and MSFI-N approach for pressure (a) and temperature (b) for grid after 10 years at 16th layer Grid refinement for 3D m-sfi case. Pressure and temperature profile at production well block Schematic of the SPE 10 setup: permeability (top) and porosity (bottom) maps SPE 10 m-sfi case. Pressure, temperature, and water saturation profile for grid at central block of top layer SPE 10 m-sfi case. Pressure (left) and temperature (right) distribution for grid after 100 days at top layer SPE 10 m-sfi case. Relative error between FIM and MSFI-N approach for pressure (left) and temperature (right) distribution for grid after 100 days at top layer Schematic representation of different operator-splitting strategies for flowmechanics problems Sketch of the setup for Mandel s problem Mandel s problem. Comparison of the analytical and the numerical solutions for displacement and pressure Mandel s problem. Sequentially implicit method convergence profile after first time step t = days Dean s model geometry Dean s problem solution comparison Water Flooding Problem. Properties distribution Water Flooding Problem. Comparison of FIM (solid color) and SEQ (dashed) simulation time for single-phase setup Water Flooding Problem. Comparison of FIM (solid color) and SEQ (dashed) simulation time for two-phase setup Water Flooding Problem. Comparison of FIM (solid color) and SEQ (dashed) simulation time for the two-phase plastic setup Cold Water Injection problem description xvi

17 3.33 Cold Water Injection. Sequentially implicit method convergence profile after first time step t = days Saturation and temperature distributions in the middle of the reservoir at the end of simulation for FIM approach (left) and difference in them between FIM and SEQ simulations (right) Oil production and water injection rates. FIM and SEQ approaches produce nearly identical results Schematic representation of the solution for sequential mechanics-thermal coupling Example of update for coupled properties A dependency graph for a flow-thermal-mechanics problem Scheme of the interaction between simulator and the ADETL library in the GENIC framework. Both mappers and subsets are part of ADETL Differences between regular and parallel mapping operators Parallel implementation through parallel mapping operators for a flowthermal problem for n computation nodes Schematic representation of multi-scale sub-problems and corresponding sub-domains Interaction between multi-scale sub-problems and their coupled properties - basis functions ϕ i A.1 Overall structure of the entire simulator A.2 Structure of SubProblemTree class A.3 Structure of SubProblem class A.4 Class diagram for subsets A.5 Class diagram for formulations A.6 Class diagram for LinearSystemBase B.1 Class diagram for subproblems and their interaction with subsets in Two- Phase Prototype B.2 Class diagram for subsets in Two-Phase Prototype xvii

18 Chapter 1 Introduction Reservoir simulation is a primary tool for planning and managing subsurface flow processes including oil/gas recovery, subsurface water resources, and sequestration of carbon dioxide. Reservoir simulation is the art and science of developing numerical techniques to solve the equations that describe mass and heat flow in subsurface porous geologic formations and implementing these numerical methods in computer software (Aziz and Settari, 1979). In the last three decades, there has been a strong interest across academia, industry, and the national laboratories to develop general-purpose computational tools and frameworks to simulate various dynamic processes, and Reservoir Simulation technology is a prime example of such efforts. In the context of Reservoir Simulation, general-purpose refers to the ability to apply a software built using a unified computer code capable of simulating different subsurface flow processes (e.g., water flooding, steam injection, miscible/co 2 gas injection, chemical flooding). Developing a general-purpose reservoir simulator is challenging because the requirements on robustness and efficiency are extreme. That is, the reservoir simulator is expected to model different recovery processes of practical interest, while maintaining the highest standards of robustness and efficiency for the particular process being modeled at a given time. In a research setting, such as the Reservoir Simulation Research group at Stanford (SUPRI-B), an additional critical requirement is for the simulation framework to be modular and extensible; that is, the framework must serve as an effective research platform whereby multiple researchers work on various aspects including adding new physics modeling capabilities, developing advanced numerical methods 1

19 CHAPTER 1. INTRODUCTION 2 and solution schemes, and using the simulator as an integral part of investigating and understanding the complex interactions associated with subsurface flow processes. In the SUPRI-B group, significant efforts have been invested into developing a flexible research simulation platform. The efforts date back to the 1980s. Aziz and Wong (1989) developed a mathematical framework that accommodates different formulations to describe thermal-compositional processes at the so-called Darcy scale. Wong (1988) implemented and compared the most important black-oil and compositional formulations that were available in the literature at the time in a unified simulation code base. The work by Wong and Aziz provided important insights about the choice of variables and different formulations, especially for compositional problems. Their effort was a significant step toward realizing a general-purpose reservoir simulator. Several efforts led by Aziz and his students were invested. Important examples include the graduate work of Verma (1996), Nogaret (1996), and Byer (2000). The first truly general-purpose research reservoir simulator was GPRS, which was developed from scratch by Hui Cao as a part of his Ph.D. work (Cao, 2002). Since then, GPRS has evolved quite significantly; nevertheless, the connection back to Cao s General- Purpose Research Simulator (GPRS) is strong. At the time, the primary objective of developing the GPRS simulator was to allow for flexible nonlinear compositional formulations (e.g., natural-variables and mass/molar variables), so that researchers can compare existing methods and propose new and improved models. GPRS is distinguished from the previous efforts in its extensible modular design and object-oriented computer code written in C++. The code design of GPRS reflected the physical components associated with oil-field development. Each field was split into several spatial domains, and each domain includes one reservoir and all of the wells that are in that reservoir. In addition, each domain also had an equation selector (module to select a subset of the governing equations) and a linear solver. In GPRS, the Jacobian matrices were calculated separately for the reservoir part and for the wells part. The equation selector was used to recast the Jacobian using the desired variables. In addition, one was able to specify different levels of implicitness in the treatment of the variables (Cao, 2002). The first version of GPRS included black-oil and isothermal compositional models, unstructured grids and Multi-Point Flux Approximation (MPFA) discretization schemes. Cao proposed a general formulation approach in

20 CHAPTER 1. INTRODUCTION 3 order to test and validate different choices for the variable sets used to describe compositional flow. He integrated different formulations into the simulation framework, including IMPES (implicit pressure and explicit saturations and mole fractions), IMPSAT (implicit pressure and saturations and explicit mole fractions), AIM (Adaptive Implicit Method), and FIM (Fully Implicit Method). Note that GPRS had a connection-based design that allowed for the use of structured and unstructured grids (Lim, 1995). Jiang (2007) made significant contributions to GPRS in terms of architectural design, extensibility, computational efficiency, and new advanced well modeling capabilities. He designed and implemented a new framework, in which the facilities model was treated as a separate component and promoted to the same level as the reservoir component. The framework enabled the combination of any available facilities object (e.g., standard wells, multi-segment wells, and well groups). Jiang proposed a general discrete wellbore model that accommodates complex (surface) network topology with general branching, loops, and multiple exits. Jiang extended the GPRS linear solver capability with novel Multi-Level Block Sparse (MLBS) data storage of the matrices. One of the lasting contributions of Jiang is the advanced multi-stage linear solution framework, including the two-stage CPR (Constrained Pressure Residual) preconditioning approach, which was extended to support general wellbore models. His advanced linear solvers with multi-stage preconditioners made it possible to simulate complex compositional displacements using highly detailed models with unstructured grids and complex multi-segment wells (Jiang and Tchelepi, 2009). Several students and researchers in SUPRI-B expanded the capabilities of GPRS. Notable efforts include reactive transport Fan (2010), phase-equilibrium computations Pan and Tchelepi (2010, 2011), and tie-line based compositional simulation Voskov and Tchelepi (2009a,b). The next highly significant development was due to Younis (Younis, 2011). He developed and implemented the Automatically Differentiable Expression Templates Library (ADETL). ADETL provides generic representations of variables and discretized expressions on a graph and automatically computes the sparse analytical Jacobian. Unlike existing AD packages at the time (Shiriaev and Griewank, 1996; Griewank et al., 1996; Bendtsen and Stauning, 1996; Bischof et al., 2008), the computational performance of ADETL was demonstrated to be reasonably on par with hand-coded differentiation routines. In addition,

21 CHAPTER 1. INTRODUCTION 4 ADETL provides the capability to define a variables formulation and performs variable switching, ordering, and alignment behind the scenes. Using the library, large-scale numerical simulations can be performed rapidly by simply writing the discrete residual equations without the need for hand differentiation and hand-crafted performance tuning. The Ph.D. work of Younis led to the first automatic differentiation platform for GPRS, which we call AD-GPRS (Younis, 2011). Zhou (2012) made significant contributions to the development of AD-GPRS. He improved the efficiency of the ADETL library with high-performance memory allocation and data storage, block-sparse operators, and interfaces for variable switching and ordering (Zhou, 2009). In addition, Zhou integrated advanced well modeling capabilities, the MLBS data structure, and the BILU and CPR preconditioners into AD-GPRS. Zhou (2012) also developed a unified code-base for both Two-Point Flux Approximation (TPFA) and MPFA in space, and for any combination of FIM, AIM, IMPES, and IMPSAT in time. This code unification designed and implemented by Zhou (Zhou, 2012) differentiates AD-GPRS from GPRS where each functionality was often implemented as a new code branch. His contributions allowed for a smooth transition from GPRS to AD-GPRS as the default platform for reservoir simulation research at Stanford. AD-GPRS was initially developed with focus on isothermal compositional displacement processes with advanced capabilities, such as a variety of nonlinear formulations (Voskov and Tchelepi, 2012), multi-stage linear solvers with powerful preconditioners Zhou et al. (2013), complex multi-segment wells (Zhou, 2012), optimization packages (Voskov and Volkov, 2013), nonlinear finite-volume approximations (Terekhov et al., 2017), and flexible restart capabilities (Iskhakov, 2013). There has been significant growth in the complexity of simulation models of practical interest, such as unconventional gas production, thermal Enhanced Oil Recovery (EOR), enhanced geothermal systems, and Steam Assisted Gravity Drainage (SAGD). These problems are highly nonlinear with complex interactions of a wide range of physical phenomena that operate at different spatial and temporal scales. In the last few years, AD-GPRS has been extended to support thermalcompositional, EOS-based multiphase flow and transport (Zaydullin et al., 2014), chemical reactions (Farshidi, 2016), geothermal simulation (Wong et al., 2017), and coupled flow and geomechanics (Garipov et al., 2015, 2016a).

22 CHAPTER 1. INTRODUCTION 5 AD-GPRS has evolved into a powerful computational platform for modeling nonlinear multi-component multiphase flow processes in heterogeneous reservoirs. Nevertheless, the addition of new physical mechanisms into the AD-GPRS framework has proved to be very challenging. For example, the incorporation of the geomechanics module required a full understanding of a wide range of unrelated capabilities in the simulator, including details of the Jacobian-matrix construction for the flow and thermal parts. Moreover, the extension of a particular physics module to additional constitutive relations and formulations had become quite complex despite using the automatic-differentiation layer. This made the further extension to new physics quite difficult and created the need for a general simulation framework applicable to different and new physical processes and numerical methods, including multiscale formulations (Jenny et al., 2003, 2006; Zhou, 2010). The availability of a generalized framework can decrease the development cost of incorporating additional physical mechanics and exploring new formulations and solution methods. In addition, a general simulation framework would aid in the transition from the existing paradigm, in which a small number of process models are coupled together in a limited number of predefined combinations, to the new paradigm, where many different processes are coupled in arbitrary permutations. The flexibility of such a framework allows us to investigate new nonlinear coupling configurations across the different physical processes and improve our understanding of the coupled nature of the processes. This would enable us to improve the performance and stability of coupled systems by finding new coupling solution schemes and more efficient nonlinear and linear solvers for a given physics. The goal of this work is the development and implementation of such a multi-physics reservoir simulation research platform. 1.1 Multi-Physics Problems In reservoir engineering, many subsurface processes of interest are associated with Enhanced Oil Recovery (EOR). Examples include gas injection, chemical flooding, and thermal recovery methods. These displacement processes are characterized by complex interactions across the multiple components that make up the fluid-rock system. The interactions often lead to the presence of multiple fluid phases with inter-phase mass transfer as

23 CHAPTER 1. INTRODUCTION 6 a function of space and time. The multi-component transport and the multiphase flow are coupled in a highly nonlinear manner. Moreover, the changes in the pressure and temperature distributions inside the subsurface formation often lead to changes in the mechanical stresses, which impact the solid structure of the porous medium and affect the distribution of fluids. The complex interactions among different physical mechanisms pose strict requirements on the numerical methods and solution strategies. Together with numerical stability, nonlinear convergence, and truncation errors, the solution strategy should account for different types of coupling; this demands a robust multi-physics simulator. One example of a complex subsurface processes is Steam Assisted Gravity Drainage (SAGD) a widely used recovery technique for heavy oil and bitumen reservoirs (Butler, 1985; Li and Chalaturnyk, 2007). The process involves multi-physics phenomena, such as fluid thermal expansion, viscosity reduction, rock thermal dilation, and complex phase behavior. Typical SAGD operations involve a pair of horizontal wells located one a few meters above the other (Butler, 1985). Steam is injected into the upper well to form a steam chamber and to mobilize the reservoir oil. The mobilized oil drains to the production well that lies below. Significant mechanical stresses associated with this process increase the risk of fracturing the reservoir or the cap-rock. In the case of hydrate-bearing sediments, when methane is produced from the hydrate reservoirs, flow is tightly coupled to the mechanics because the changes of pressure and temperature can have a significant impact on the effective stresses in the porous medium (Rutqvist and Moridis, 2009; Moridis et al., 2013). Moreover, changes in the stress and temperature fields influence hydrate formation or dissociation. This can cause lead to wellbore instabilities or possible methane emission to the surface due to geological failure (Kim et al., 2011). In environmental engineering, land subsidence due to the exploitation of subsurface resources, its damage to surface infrastructure, and its impact on our environment have been the subject of extensive studies in the subsurface modeling of fluid flow and geomechanics (White and Morton, 1997; Hermansen et al., 1997; Hansen et al., 1995; Settari et al., 2008). Compaction of unconsolidated aquifer systems that accompanies excessive groundwater pumping is by far the single largest cause of subsidence in the United States (Gai, 2004). The increasing development of land and water resources threatens to exacerbate the existing subsidence problem while initiating new ones. Surface subsidence may also be induced by

24 CHAPTER 1. INTRODUCTION 7 hydrocarbon withdrawal from weak formations and chalk reservoirs. While reservoir compaction itself has been widely recognized as an additional driving mechanism for increasing oil and gas recovery, its side effects are undesirable. The most obvious one is surface, or sea-floor settlement, which may create environmental problems and cause damage to oilfield structures and seabed pipelines (Gai, 2004; Chaussard et al., 2013). One well-known case is the excessive groundwater removal that has caused Venice to start sinking (Lubick, 2002). The same problem has occurred in other cities, such as Tokyo, Shanghai, Houston, and New Orleans (Nicholls, 1995; Chaussard et al., 2013), where the simulation of groundwater usage enabled effective environmental regulation to reduce groundwater use. The effective treatments for these problems can be accurately assessed with flow-mechanics simulations. In nuclear waste management, despite more than thirty years of active research and development (Ames and Rai, 1978; Abrajano and Bates, 1987; Wronkiewicz et al., 1996; Gaucher et al., 2004), there is no operating geologic repository for highly radioactive waste. This is due to the long times (thousands of years) for which predictions need to be performed, and where the overall scenario cannot be verified entirely, as it is deduced from many individual observations made by different laboratories or in the field. These issues can be resolved by use of numerical simulations which allow modeling over a wide range of parameters and longer time scale. In nuclear waste disposal, radioactive materials generate heat, which can perturb an initial stable condition, causing fluid flow or deformation. Modeling of this process requires the use of a coupled flow-thermal-mechanics and chemical reactions simulator (Rutqvist and Tsang, 2003; Grambow et al., 2014). Additionally, there is great interest in the development of ceramics materials that can withstand radiation damage over millennia without losing their properties; the ceramic crystalline structure can be found from numerical simulations when a large number of scales can be considered (Devanathan et al., 2010). Recent studies (Devanathan et al., 2010; Verma et al., 2015) reveal a demand for additional scales (quantum and atomic) and physics (quantum mechanical calculations, classical molecular dynamics simulations, kinetic Monte Carlo and phase field methods) for a correct prediction. This further increased interest in a multi-physics simulator.

25 CHAPTER 1. INTRODUCTION 8 In earthquake engineering, there is building legislation that attempts to ensure that structures would suffer no damage in moderate events and would not collapse in major events. However, this legislation does not guarantee earthquake-proof buildings. In addition to damage caused by severe ground shaking, damage to structures during earthquakes can be due to the soil response (such as ground cracking, liquefaction, lateral spreading, and landslides) and to the effects of adjacent structures, either through direct contact or through structure-soil-structure interaction (Elnashai and Di Sarno, 2008). Numerical simulation of soil-structure allows for the design of structures that meet specific performance criteria set by the structure s owners and regulators. This improves the prediction of the structural response and enables the design of earthquake-resistant buildings. For most soil responses, a mechanical simulator is enough (McKenna, 2011). However, in the case of liquefaction (a state in which fluid pressure in a porous medium becomes greater than the forces holding the solid together, that makes solid behave like fluid), a complex flow-elasto-plastic simulator is needed (Carey et al., 2016). Each of the above-mentioned fields has its set of simulators; however, it is still challenging to extend a software for additional physics within the same field or employ similar ideas with slightly different governing equations towards another field. We believe it is important to develop a flexible multi-physics simulation platform capable of accurate modeling of complex subsurface processes. 1.2 Coupling Methods for Multi-Physics Problems In the reservoir engineering community, the interest in different coupling strategies emerged at early stages of numerical simulations. While the fully coupled approach the Fully Implicit Method (FIM) - ensures unconditionally stable numerical solutions for well-posed problems and is a straightforward method of solving systems of equations, it is computationally expensive and cannot effectively handle problems of practical interest when the computational resources are limited (Aziz and Settari, 1979). This has led to the development of sequential methods for coupled flow (pressure) and transport (saturation) problems. The IMplicit Pressure, Explicit Saturation (IMPES) was developed by Sheldon et al. (1959), Stone and Garder (1961), and Danilov et al. (1968). The main idea in IMPES is to

26 CHAPTER 1. INTRODUCTION 9 split the coupled system into a pressure equation, which is solved implicitly in time, and saturation equations that are advanced using explicit time approximation. Because of the explicit treatment of saturation, IMPES is only conditionally stable and the time step is limited by a CFL-condition (Coats, 2000, 2003). IMPES is often considered as a special form of the FIM method with only one implicit variable, since it can be obtained from FIM by applying algebraic reduction (Aziz et al., 2005). While IMPES is computationally efficient for a single time step, the total cost of an IMPES simulation can easily exceed that of FIM given the prohibitively small time steps that are required in real-field applications. Nevertheless, IMPES remains a popular method in the community to solve large-scale problems, and it is still used today. The IMplicit Pressure and SATuration(s) (IMPSAT) sequential method was developed to overcome the time step limitations of the IMPES method by incorporating implicit treatment of saturations without solving simultaneously for pressure and saturations. IMPSAT was proposed by MacDonald and Coats (1970) and later reported by Spillete et al. (1973), Coats et al. (1974) and Coats (1976). The method consists of two steps. The first step is to obtain an implicit pressure solution in exactly the same way as for the IMPES method; however, the saturations are updated by solving the transport problems with implicit time stepping (Watts, 1986). IMPSAT is an operator splitting method, in which the flow problem (pressure field) and the transport problem (saturations) are decoupled and solved sequentially. However, for compressible flow, mass is generally not conserved for one of the fluid phases. As a result, the mass-balance errors are proportional to the areal variation of ρ n /ρ w (Aziz and Settari, 1979; Coats et al., 1974) and can be significant for large time steps. With growing computational power, FIM became the main approach for solving coupled multiphase flow and transport problems and the development of sequential methods for such problems slowed down significantly. The Sequential Fully Implicit was developed and integrated with the Multiscale Finite-Volume method (MsFV) Jenny et al. (2006). In the SFI scheme, each time step consists of an outer loop and two inner loops. The first inner loop deals with the pressure; the second inner loop deals with the saturations; the outer loop deals with the iterating on all the variables. If only a single outer-loop iteration is performed, we obtain the IMPSAT method. If a single iteration of both the outer and inner Newton loops is performed, we obtain the IMPES method (Jenny et al., 2005). The

27 CHAPTER 1. INTRODUCTION 10 SFI method for MsFV was tested for large-scale (millions of grid blocks) multiphase flow problems in highly heterogeneous domains, and the coupling scheme has been shown to be robust and computationally efficient, even for large time steps. However, the method requires a multiscale framework and has not been used extensively in traditional (i.e., single scale) compositional simulators. While FIM is a popular coupling method for the pressure-transport problem, the situation is different for flow-mechanics problems. The coupling between the flow and mechanics codes has been the subject of intense activity in the last decade. The coupling schemes can be split into two broad categories (Settari and Walters, 2001): iteratively coupled and fully coupled. Similar to the pressure-transport problem, FIM for flow-mechanics ensures unconditionally stable numerical solutions for well-posed problems (Jha and Juanes, 2007). However, the fully coupled approach is computationally expensive and requires the development of a unified flow-geomechanics simulation framework (Hu et al., 2013; Yang et al., 2014). In addition, the simulation domain for the mechanical problem is usually larger than the domain for the flow problem, since it includes the overburden, base, and sides. Consequently, the computational cost of the mechanical problem at each simulation time step can be quite high. Another disadvantage of the fully coupled approach consists of the fact that the resulting linear system associated with the FIM Jacobian has a mixed character of near-elliptic and hyperbolic components. The fully coupled approach for flow and mechanics requires complex preconditioning procedures for the efficient solution of the linear system (White and Borja, 2011; White et al., 2016; Castelletto et al., 2015; Klevtsov et al., 2016). The development and application of sequential-implicit solution schemes are motivated by a simple idea to avoid the solution of a complex full system with mixed character. In sequential coupling, the system of governing equations is divided into several physical subproblems. Each subproblem can be solved by a different method, which includes dedicated nonlinear and linear solvers. In modern reservoir simulation, different types of sequential coupling between the flow and geomechanics are used. Initially, the coupled solution of flow and geomechanics were implemented in explicitly coupled fashion (Park, 1983; Zienkiewicz et al., 1988), where each physical process was solved separately with a partial update of the solution between different processes. In this

28 CHAPTER 1. INTRODUCTION 11 approach, the convergence of the full system of the governing equations often was not checked. While this method can be efficient and accurate in some situations, it does not guarantee the correctness of the solution. Later, a loosely coupled scheme was proposed for the solution of flow and geomechanics problems (Bevillon and Masson, 2000; Settari and Walters, 2001). This approach is similar to the explicitly coupled scheme with an occasional solution of the coupling between different physics. The loosely coupled scheme may achieve a very good performance (Minkoff et al., 2003b; Samier et al., 2008) with a reasonable match to the results of a fully coupled solution; however, it is not stable and it is not clear when the full system should be resolved. Several sequential-implicit approaches have been proposed (Prevost, 1997; Settari and Walters, 2001; Thomas et al., 2003; Tran et al., 2004; Gai, 2004; Wheeler and Gai, 2007). Sequential-implicit formulations have several advantages over FIM, such as the use of separate modules for each physical process, use of specialized linear solvers and preconditioners for different physics components, and separate treatment of the different computational domains (Mainguy and Longuemare, 2002). Moreover, for flow-mechanics problems, such an approach can provide accurate and unconditionally stable solutions (Armero and Simo, 1992; Kim et al., 2013; Mikelic and Wheeler, 2013). Due to an increased interest in the Reservoir Simulation community, several implementations of the sequentialimplicit method for a flow-thermal-geomechanics simulation have been developed. However, existing reservoir simulators capable of solving challenging problems are limited to a specific coupling strategy, namely, an iterative sequential approach. Examples include: the Subsurface Transport Over Multiple Phases (STOMP) (White and Oostrom, 2006), TOUGHT+FLAC (Rutqvist, 2011), IPARS-JAS3D (Minkoff et al., 2003a), and Open- GeoSys (Kolditz et al., 2012). However, without reference solutions of the fully coupled physics, it is often not clear whether the obtained solutions are accurate, let alone having the detailed assessment of the computational performance. Issues related to the accuracy of the computed solutions can be addressed when a fully coupled multi-physics simulator is used; examples include DuMuX (Flemisch et al., 2011) and MRST (Lie, 2014; Keilegavlen and Nordbotten, 2017). Unfortunately, published results are reported for small-scale problems and for relatively simple multi-physics interactions. To our knowledge, none of the available simulators are capable of running both fully-implicit and sequential-implicit methods

29 CHAPTER 1. INTRODUCTION 12 within the same code platform for challenging nonlinear problems, such as SAGD problems. There is also a lack of systematic studies of the robustness and efficiency of different nonlinear coupling strategies. An available comparison (Markert et al., 2010) performed in PANDAS (Ehlers and Ellsiepen, 1998) is limited to simplified flow-mechanics problems without nonlinear effects. Moreover, this study employs a finite-element methodology, while large-scale compositional simulations are heavily reliant on conservative finitevolume approximations. This motivates the development of a framework that allows for accurate comparison between coupling strategies. 1.3 Current Progress in Multi-Physics Simulator Development We can categorize numerical simulators into three groups. The first group consists of highly specialized, fast, parallel, state-of-the-art simulators for a specific set of problems where no more than two physics modules are solved simultaneously. Examples include compositional-thermal and flow-mechanics. All commercial simulators belong to this group. The second group is a set of fully-coupled multi-physics simulators. These simulators are either non-extendable for additional physics, or have limited scalability. There are ongoing efforts to overcome these limitations by developing efficient multi-physics preconditioners, such as in the case for compositional-mechanics (White and Borja, 2011; White et al., 2016; Castelletto et al., 2015; Klevtsov et al., 2016). The third group is represented by sequential iterative multi-physics simulators.the sequential coupling is often rigid, does not permit extension to new physics, and the order of the involved physics modules and coupling terms are fixed. In summary, available multi-physics simulators have several issues. First, they exhibit poor scalability for fully coupled approach caused by a limited number of efficient multiphysics preconditioners. Second, the design of coupling strategies is severely limited due to the extreme complexity of simulator development. Third, to the best our knowledge, an accurate and consistent comparison of different coupling strategies for multi-physics problems has not been performed. Finally, available simulators in reservoir engineering are

30 CHAPTER 1. INTRODUCTION 13 not extendable for additional physics and processes, discretization domains, and coupling terms. This work is not the first attempt to create a flexible multi-physics simulator. Developing process-rich simulators with great fidelity is a common challenge across many disciplines. In most cases, the focus is on the richness of the modeled processes and not on providing wide flexibility in the nonlinear coupling strategy across the different physics modules. Some of the strategies used to manage complexity in a multi-physics simulator include: data managers (Larson et al., 2005; Slattery et al., 2013), dependency graphs (Berzins et al., 2011), and process couplers (Redler et al., 2010; Craig et al., 2012). The Data Manager is a software that uses database solutions to enable users to manage structures of simulation and process data across the complete product lifecycle. Data managers are common in simulations that involve multiple domains, such as atmosphere-ocean-land climate models (Redler et al., 2010), or in engineering applications with solid and fluid domains (Larson et al., 2005). Data managers help achieve solutions of process-rich simulation, but they do not focus on the scalability of multi-physics problems. Dependency graphs is a system representing dependencies of several objects to each other and started getting attention in extreme scale computing, where it is important to identify the correct order of the final complex system and identify concurrency in the processes (Kale et al., 2007; Agullo et al., 2009; Notz et al., 2012). Dependency graphs are used in combination with process couplers - a software that links together and mediates interactions between models (physics modules). Couplers are abstractions with different implementations; no standard reference architecture for couplers has emerged. The functions typically associated with couplers include managing data transfer between two, or more, models (often in parallel), interpolating field data when the grid resolution, or discretization, differs between these models, ensuring conservation of physical quantities across the coupling boundary, accumulation and averaging of physical quantities (e.g., when the timesteps are different in different modules), and coordinating execution of the constituent models (Dunlap et al., 2013). Examples of frameworks based on process couplers include the Community Earth System Models - CESM (Kay et al., 2015), CPL7 (Craig, 2010), and OASIS (Redler et al., 2010). These types of efforts result in clean, modular code; however, the coupling is limited to sequential schemes and is allowed only between different sub-domains.

31 CHAPTER 1. INTRODUCTION 14 The recent work of Coon et al. (2016) is a conceptual model for managing complexity in multi-physics codes using a process tree, a dependency graph, and a data manager. These ideas were applied to develop the Arctic Terrestrial Simulator, which uses the process coupler approach to manage the coupling among the process models present in the same sub-domain. In addition, dependency graphs and the associated evaluation process are used extensively to manage and organize shared data requirements among different process representations. While the ATS framework is an example of a very flexible platform, there are some possible disadvantages. The first is that it requires the developer to cede some control and change their approach to software development through a significant learning curve. The developer is forced to implement small blocks of code with significant code redundancy. Additionally, there are performance concerns. The overhead of traversing, maintaining, and evaluating dependency graphs can entail indirection and therefore wallclock time and memory requirements to the simulation. Finally, implementation of both fully coupled and sequentially coupled approaches requires significant additional efforts. Here, we developed a modular multi-layer architecture for the multi-physics framework that can employ any coupling strategy across the modules. This level of flexibility is achieved using a sub-problem tree structure, a global status table, and the integration of these concepts with subsets through an automatic differentiation layer. The modular design with transparent subset-based interfaces greatly improves code extensibility. In addition, we have employed a unique mapping operators concept, which enables advanced coupling strategies, including domain-decomposition and multiscale approaches. We have designed, developed, implemented and tested a novel general implicit coupling framework for multiphysics problems (GENIC) that allows for rapid customization. This GENIC framework constitutes the next generation of the Automatic-Differentiation General-Purpose Research Simulator (AD-GPRS), and it represents a flexible and efficient reservoir simulation research laboratory in which students and researchers can easily test their ideas. 1.4 Outline The dissertation proceeds as follows. In Chapter 2, we present the governing equations for flow, thermal, mechanics, and wells, and the different coupling strategies between them.

32 CHAPTER 1. INTRODUCTION 15 Then, we describe the design of the framework based on three key principles: modular design, extension to new physics, and seamless coupling. We elaborate on how to add new functionality for to existing physics models, how to add new physics, how to control domain discretization and formulations, how to specify coupled properties and coupling strategies. Finally, we describe the general algorithm and mapping operations for advanced sequential coupling strategies. In Chapter 3, we demonstrate numerical applications of the framework using several multi-physics problems with different coupling strategies. For each physics coupling, we apply different coupling strategies and splitting operators. We demonstrate examples with two (flow-thermal, flow-mechanics) and three (flow-thermal-mechanics) physics. We show that the best choice of the coupling strategy depends on the coupling strength across the physics models, which depends strongly on where a particular problem lies in the multiphysics parameter space. Finally, we demonstrate the flexibility of the framework by trying several new coupling methods for highly nonlinear flow-thermal problems. In Chapter 4, we describe how we can further extend the GENIC framework to solve large-scale multi-physics problems using automatic coupled properties, parallel domain decomposition, and multiscale formulations. These new capabilities can be developed in alignment with the key principles of the framework and preserve its flexibility and efficiency. We conclude this thesis in Chapter 5 with a summary and suggestions for future research.

33 Chapter 2 General Implicit Coupling Framework (GENIC) The primary design objective of the General Implicit Coupling Framework (GENIC) reservoirsimulation framework is the creation of a platform where different coupling strategies for the simulation of complex physical processes can be designed and implemented with great flexibility. For a given set of coupled nonlinear processes, such a capability makes it possible to find the best solution strategy in terms of robustness and computational efficiency. The GENIC platform is based on three key principles. The first principle is modular design. Each module represents a particular physics process that can be described by a set of conservation equations and associated constitutive relations. We are interested in dynamic processes that are nonlinear and tightly coupled to other physical processes. Isothermal compositional displacement processes are one example; compositional simulation involves multi-component transport across multiple flowing fluid phases with interphase mass transfer. Another important problem space is thermal Enhanced Oil Recovery (EOR), of which steam injection is a primary example. In thermal EOR processes, one must keep track of energy transfer in addition to mass transfer. A growing area of interest is poromechanics whereby the interactions of the fluid with the solid skeleton must be accounted for, and a momentum balance for the solid-fluid aggregate is required. The GENIC design allows for testing each physics module independently, or in combination with other physics modules. For example, compositional-thermal, thermal-mechanics, 16

34 CHAPTER 2. GENERAL IMPLICIT COUPLING FRAMEWORK (GENIC) 17 thermal-reactions, or compositional-thermal-mechanics type of a multi-physics problem can be solved. Here, compositional refers to the coupled mass conservation equations of multiple species; thermal refers to the energy balance; mechanics refers to the Biot formulation of poromechanics; reactions refers to the accounting for chemical reactions (both equilibrium and non-equilibrium) in the species mass conservation equations. Importantly, only the specified physics modules are involved in constructing the nonlinear solution strategy in GENIC, and the details of the construction are applied at Run-Time. Since each module is independent of the other ones, the requirements for a researcher to contribute to GENIC are focused on knowledge of the relevant physics. This is expected to widen the user base and allow individuals to contribute to the module of his/her expertise, if they have adequate programming skills. Modularity, or component-based design, is particularly important for modeling nonlinear multi-physics. Modularity facilitates decoupling of multi-physics problems, but it is really critical for methodical management of solution methods for coupled nonlinear problems. Modularity and extensibility go hand in hand. Additionally, the modular structure reduces memory and processing consumption by optimizing the required resources for large-scale problems of practical interest. The second key principle is physics extension. The GENIC framework enables a general representation of a given physics with flexible discretization in both space and time, the choice and ordering of variables for each physics. GENIC also allows for associating different physical regions with different physics. For example, the compositional module can be discretized in space using finite-volume methods, such as Two-Point Flux Approximation (Aziz and Settari, 1979), Multi-Point Flux Approximation (Aavatsmark et al., 1998, 2010; Edwards and Rogers, 1998), Nonlinear Finite Volume Approximation (Le Potier, 2005; Nikitin et al., 2014; Terekhov et al., 2017) or Mixed-Finite Element Methods (Matringe, 2008). For time discretization, the Fully Implicit Method (FIM), IMPES (Sheldon et al., 1959; Stone and Garder, 1961; Danilov et al., 1968; Coats, 2000) or Adaptive Implicit Method (Thomas and Thurnau, 1983; Russell, 1989; Cao, 2002; De Loubens et al., 2009; Maes et al., 2013). GENIC provides wide flexibility in choosing the primary variables and the ordering of equations and unknowns; these include the natural-variables (Coats, 1980), molar variables (Young and Stephenson, 1983; Fussell and Fussell, 1979; Chien et al., 1985) or the tie-simplex space (Zaydullin et al., 2013). This flexibility is available for every

35 CHAPTER 2. GENERAL IMPLICIT COUPLING FRAMEWORK (GENIC) 18 physics module. Because the modules are independent of each other, different combinations of discretization and choice of variable sets are allowed. For example, one can solve a compositional-thermal problem with a choice of pressure p, phase saturations S p, molar fractions x cp and temperature T. Another choice can be pressure p, overall composition z c and molar enthalpy h p. Similar flexibility is simultaneously allowed for discretization in time and space. For example, one can apply FIM and Nonlinear Finite Volume Approximation for time and space discretization of a compositional module, and at the same time, discretize the thermal module with AIM and Two-Point Flux Approximation. In addition, physics extension is aligned with modular design: each extension is implemented as a separated sub-module and can be activated and deactivated independently. There is no resource allocation for non-active sub-modules - this enables the extension of capabilities without the performance drop. The last key principle is seamless coupling. The platform provides seamless coupling between the physics modules without resorting to conditional branches and intermediate interfaces. GENIC treats terms that are coupled across multiple physics modules efficiently. This principle enables a choice of coupling strategy for a given problem on the fly. For example, the same problem can be run with FIM, Sequential Fully Implicit (SEQ) (Jenny et al., 2006) or hybrid (FIM-SEQ) solution methods by using an input parameter for the coupling logic (solution scheme). The framework design is not limited to the specific number or ordering of the physics modules; therefore, wide flexibility is achieved to investigate coupling strategies in order to find the most effective strategy for the given multi-physics problem. For three or more physics modules, nested FIM-SEQ constructions are allowed. This level of coupling is not available in other multi-physics simulators. In fact, existing simulators are usually limited by the number of the physics modules and how they can be sequenced. Building a framework that allows for seamless coupling is one of the key developments in this work. Figure 2.1 shows the interaction between the key principles. Modularity brings clarity and simplicity to the software development; physics extensions provide simulator capabilities, and seamless coupling maintains flexibility of the framework. The different principles are synergistic, and combining them effectively has been one of the primary challenges in developing GENIC. For example, the implementation of physics extension as a sub-module

36 CHAPTER 2. GENERAL IMPLICIT COUPLING FRAMEWORK (GENIC) 19 Figure 2.1: Key principles of the GENIC framework. enables efficient resources management, when no resources are allocated for unused capabilities. The seamless coupling of different physics modules is the key to multi-physics integration: without a modular design, the platform would not be extensible to new physics, while without seamless integration only one coupling method (FIM or SEQ) would be allowed. Finally, the choice of coupling strategy requires various coupled properties such as fixed-stress or fixed-strain for compositional-mechanics coupling. Such flexibility is provided by seamless coupling of coupled properties implemented with physics extensions principle. This chapter is organized as follows. First, we give a brief description of the governing equations and the different coupling strategies. Then, we describe in detail the design of the framework based on the key principles. 2.1 Mathematical Formulation The GENIC framework is applicable for an arbitrary number of physics modules as long as we can specify the physics with a set of governing partial differential equations, unknowns, and computational domain. Presently, the framework is capable of solving compositional, thermal, mechanical, and chemical processes with the presence of wells. For brevity, we limit the description here to thermal multi-component flow in deformable subsurface formations with wells. The full set of governing equations that describes these processes

37 CHAPTER 2. GENERAL IMPLICIT COUPLING FRAMEWORK (GENIC) 20 includes the conservation of mass, energy, and momentum, and well constraints. The appropriate constitutive relations and boundary conditions are assumed to be fully specified Multi-Component Transport The mass balance equations for n c components that can exist in n p fluid phases can be written as: R F t ( φ ) ( n p np ) x i j ρ j S j + x i j ρ j v j + j=1 j=1 n p x i j ρ j q j = 0, i = 1..n c, (2.1) j=1 where i is the index of a component, j the index of a phase, φ the porosity, x i j the phase compositions, ρ j the phase density, S j the phase saturation, q j the source term. Phase velocity, v j, is given by the Darcy s law v j = k k r j µ j ( p j ρ j g ), (2.2) where k is the absolute permeability, k r j the relative permeability of the phase, µ j the viscosity of the phase, and g is the gravity. These equations are supplemented with linear constraints for the phase compositions and the phase saturations: n c i=1 x i j = 1, j = 1..n p, (2.3) n p S j = 1, (2.4) j=1 and the equality condition for fugacities (instantaneous thermodynamic phase equilibrium is assumed): f i j ( p,t,xi j ) fik (p,t,x ik ) = 0, j k = 1..n p,i = 1..n c. (2.5) Here, we employ the natural-variables formulation (Coats, 1980; Voskov and Tchelepi,

38 CHAPTER 2. GENERAL IMPLICIT COUPLING FRAMEWORK (GENIC) ), where the unknowns are the pressure, p, phase saturations S j, and phase compositions, x i j Conservation of Energy We assume thermal equilibrium between the rock and the fluids. It follows that the totalenergy balance equation can be expressed as: R T t ( φ ) ( n p np ) u j ρ j S j + (1 φ)u r + h j ρ j v j (κ T ) + j=1 j=1 n p h j ρ j q j = 0, j=1 (2.6) where κ is the thermal-conductivity coefficient, u j the phase internal energy, u r the rock internal energy, and h j the phase enthalpy. The unknown related to the energy equations is the temperature T. All the thermal properties are evaluated based on p, T, and x i j using correlations and a cubic Equation of State (EoS). The details of the thermal-compositional implementation are given elsewhere (Zaydullin et al., 2014) Momentum Balance The mechanical deformation equations are formulated in terms of total stresses, bulk strains, pore pressure, and temperature. They are written as follows (Nikolaevskij, 1990; Jaeger et al., 2009): R M σ + ρg = 0, (2.7) where ρ = ρ s (1 φ)+ρ f φ is the overall-mass density, ρ s is the density of the solid skeleton and ρ f is the average fluid density. The total-stress tensor, σ, contains the contributions of the fluid and solid-skeleton (Zienkiewicz et al., 1999): σ = Cε e bp, (2.8) where ε e is the second-order elasticity strain tensor, b is the second-order tensor of Biot coefficients, C is the forth-order elasticity moduli tensor. The fluid pressure is defined as n p P = S j p j. j=1

39 CHAPTER 2. GENERAL IMPLICIT COUPLING FRAMEWORK (GENIC) 22 We follow the small deformation theory and assume that the total strain tensor, ε, can be decomposed into the three components representing elastic, plastic, and thermal contributions: ε = ε e + ε p + ε θ, (2.9) so the elastic strain can be expressed as ε e = ε ε p ε θ. (2.10) We also assume that the dilation induced by the temperature is isotropic (Ottosen and Ristinmaa, 2005), i.e., ε θ = αt 1 (2.11) with α being the skeleton linear thermal expansion/dilation coefficient. Then, combining Eq. 2.9 and Eq. 2.8, we obtain: σ = C (ε ε p ) at bp, (2.12) where we define the thermal expansion tensor, a = αc1. In the following, we also assume that the Biot s tensor is isotropic, i.e. b = b1, where b is the Biot s coefficient. Eq. 2.7 and Eq together with the total-strain definition, ε = 1 ( u + T u ) (u is 2 the displacement vector), provide a framework for modeling the deformation behavior of natural saturated porous formations. A material model needs to be added to complete the description. We use the Drucker Prager (Drucker and Prager, 1952) model to compute the stress-strain response in Eq To determine the plastic behavior, the classical associated Drucker Prager model employs the following yield surface F (Borja, 2013): F = J 2 + B I 1 + A, (2.13) where I 1 and J 2 are volumetric and deviatoric invariants of the effective stress, and A and B are material parameters. We assume that the Drucker Prager yield surface circumscribes

40 CHAPTER 2. GENERAL IMPLICIT COUPLING FRAMEWORK (GENIC) 23 the Mohr-Coulomb yield surface, and consequently express A, B as follows: A = 6 c cosψ 2 sinψ, B =, (2.14) 3(3 sinψ) 3(3 sinψ) where ψ is the internal friction angle and c is the cohesion. This model primarily captures plastic dilation, and the size of the yield surface is controlled by the internal state variables, such as softening and hardening parameters. In this work, we employ an isotropic softening/hardening rule: ċ = H v ε p v H t Ṫ, (2.15) where H v and H t are two material parameters, ε p v = tr(ε p ) is the volumetric plastic strain, and dot defines an increment. The first term on the right hand side of Eq allows the yield surface to grow with plastic volumetric compaction (hardening) and shrink with plastic dilation (softening), and the second terms allows shrinking of the yield surface due to temperature increase. Following Coussy (2004) we introduce the porosity as the ratio of the volume of the connected porous space to the total volume. The relationship between the porosity and stress/strain is assumed to be: φ φ p = φ o + (b φ o)(1 b) K d (P P o ) + b(ε v ε p v ε v,0 ) α φ (T T o ). (2.16) Here, K d is the local drained bulk modulus, ε v = tr(ε) is the volumetric total strain, and α φ is the volumetric thermal dilation coefficient related to the porosity. The reference porosity, pressure, and volumetric strain are denoted as φ o, P o, and ε v,o, respectively. Assuming that the irreversible porosity is defined as φ p = bε p v, we obtain: φ = φ o + (b φ o)(1 b) K d (P P o ) + b(ε v ε v,o ) α φ (T T o ). (2.17) We use a widely used model for the permeability calculation (David et al., 1994): k = k 0 ( φ φ 0 ) m. (2.18)

41 CHAPTER 2. GENERAL IMPLICIT COUPLING FRAMEWORK (GENIC) 24 The permeability exponent, m, defines how permeability deviates from a reference value k o Wells A general multi-segment well is represented by a set of conservation equations written for each segment. The mass balance equation for each segment is: R W,F t n p j=1ρ j α j x i j z n p ρ j V s, j x i j + j=1 n p ρ j x i j q j = 0, i = 1..n c (2.19) j=1 where α j is the in-situ phase fraction (holdup) of phase j, V s, j is the superficial phase velocity (flux over the surface area), q j is the inflow of phase j to the well, z is the coordinate along the well, and n p j=1 ρ jx i j q j represents the mass source/sink through the wellbore. The energy balance equation for each segment is: R W,T t ( n p ρ j α j u j + V ) in, 2 j j=1 2 + z ( n p ρ j α j V in, j h j + V ) in, 2 j j=1 2 n p n p ρ j α j Vin, 2 j + q loss ρ j h j q j = 0 (2.20) j=1 j=1 gcosθ where V in, j is the interstitial velocity of phase j (V in, j = V s, j /α j), gcosθ is the gravitational component along the well, in which θ is the inclination angle of the segment from vertical, q loss is the heat loss from the wellbore fluid to surroundings, and it is calculated using the following relation: q loss = 2πr w u to (T w T ) (2.21) where u to is overall heat transfer coefficient, r w is the wellbore radius and T w is the temperature of wellbore fluids. We close the system of equations Eq with linear constraints for x i j and α j, Eq. 2.3, and thermodynamic phase equilibrium equations for components, Eq The details of the multi-segment well implementation and modeling of phase fractions inside the wellbore are given elsewhere (Zhou, 2012).

42 CHAPTER 2. GENERAL IMPLICIT COUPLING FRAMEWORK (GENIC) Coupling Strategies We define the set of a given physics as. Next, we introduce the following notation to describe a particular coupling strategy Ψ: Ψ = ϒ A [A 1,A 2,...,A n ], (2.22) where ϒ A is a type of coupling such as FIM or SEQ, i.e. ϒ A {FIM, SEQ}, and A i either represents a particular physics module (A i ), or refers to another coupled set A i = ϒ B [B 1,...,B k ], where B j is defined similarly to A i. Such formulation enables a nested coupling strategy. For example, solving fully coupled physics A 1 and A 2 followed by sequentially coupled physics A 3, can be written as Ψ = SEQ[FIM[A 1,A 2 ], A 3 ]. Notice that ϒ = FIM is invariant with respect to the order of arguments; on the other hand, ϒ = SEQ is sensitive to the order of the arguments. Here, we assume = {FLOW, THERMAL, MECHANICS}, where FLOW, THER- MAL, MECHANICS represent multi-component flow (Eq ), thermal (Eq. 2.6), and mechanics (Eq ). For a given we introduce a vector of unknowns, x = (x F,x T,x M ), where x F are the flow unknowns, x T are the thermal unknowns, and x M are the mechanics unknowns. The corresponding residuals are R F (x F,x T,x M ), R T (x F,x T,x M ), and R M (x F,x T,x M ). Notice that there is no assumption about which set of residuals depends on which unknowns in the general case. So, some residuals may depend only on a subset of the unknowns. In the case of a compositional-thermal-mechanics problem, the residuals for each physics module R F,R T,R M are assumed to depend on all the unknowns x F,x T,x M. The two commonly used coupling strategies for the compositional-thermal-mechanics multi-physics problem available in the literature are briefly outlined below. Option 1: Fully implicit coupling: solve all the equations simultaneously (Figure 2.2): R n+1 F RT n+1 RM n+1 ( x n+1 F,xT n+1,x n+1 ( x n+1 F,xT n+1,x n+1 M,xn) = 0, M,xn) = 0, ( x n+1 F,xT n+1,xm n+1,xn) = 0. (2.23)

43 CHAPTER 2. GENERAL IMPLICIT COUPLING FRAMEWORK (GENIC) 26 Figure 2.2: Fully implicit coupling: Ψ = FIM[FLOW, THERMAL, MECHANICS]. Figure 2.3: Fully coupled flow and energy-balance, followed by sequentially coupled mechanics: Ψ = SEQ[FIM[FLOW,THERMAL], MECHANICS]. Using Newton s method leads to the following system: R F/ x F R F/ x T R F/ x M R T/ x F R T/ x T R T/ x M R M/ x F R M/ x T R M/ x M δx F δx T = δx M R F R T R M. (2.24) Option 2: Fully coupled flow and energy-balance, followed by sequentially coupled mechanics (Figure 2.3): Step 1: Solve R n+1 F R n+1 T ( xf n+1,xt n+1,x n+1/2 ( x n+1 F,xT n+1,x n+1/2 M,x n) = 0, M,x n) = 0. (2.25) or in the linearized matrix form [ R F/ x F R T/ x F R F/ x T R T/ x T ][ ] δxf = δx T [ RF R T ]. (2.26) Step 2: Solve matrix form R n+1 M (xn+1 F,xT n+1,xm n+1,xn ) = 0. (2.27) [ R M/ x M ][δx M ] ] = [R M. (2.28) We repeat Steps 1 and 2 until global convergence is achieved.

44 CHAPTER 2. GENERAL IMPLICIT COUPLING FRAMEWORK (GENIC) 27 (a) (b) (c) Figure 2.4: Example of other possible coupling strategies for a flow-thermal-mechanics problem: (a) Ψ = SEQ[SEQ[FLOW,MECHANICS],THERMAL] (b) Ψ = SEQ[FLOW, SEQ[MECHANICS,THERMAL]] (c) Ψ = SEQ[FLOW, MECHANICS, THERMAL]. Dashed box represents nested sequentially coupled blocks. There are 13 different coupling strategies for problems involving multiphase flow, thermal effects, and mechanical deformation (Figure 2.4 depicts some of them): FIM[FLOW, THERMAL, MECHANICS] SEQ[FIM[FLOW, THERMAL], MECHANICS] SEQ[FLOW, FIM[THERMAL, MECHANICS]] SEQ[MECHANICS, FIM[FLOW, THERMAL]] SEQ[FIM[MECHANICS, FLOW], THERMAL] SEQ[THERMAL, FIM[FLOW, MECHANICS]] SEQ[FIM[THERMAL, MECHANICS], FLOW] SEQ[FLOW, THERMAL, MECHANICS] SEQ[FLOW, MECHCANICS, THERMAL] SEQ[THERMAL, FLOW, MECHANICS] SEQ[THERMAL, MECHANICS, FLOW] SEQ[MECHANICS, FLOW, THERMAL] SEQ[MECHANICS, THERMAL, FLOW] For problems with four physics modules (e.g., flow, thermal, mechanics, reactions), the number of possible coupling strategies is 49. This would require implementing a large number of different simulators, which is simply not feasible. Although not all coupling strategies are potentially effective, numerical analysis of nonlinear solution schemes is often limited to simplified physics (Kim et al., 2013; Kim, 2015). In modeling coupled nonlinear dynamics problems, it is often the case that one has to experiment with different schemes to figure out what works best for which problem class. The objective here is to build an advanced reservoir-simulation framework that allows for implementing, testing, and comparing different coupling strategies in a unified manner.

45 CHAPTER 2. GENERAL IMPLICIT COUPLING FRAMEWORK (GENIC) Framework Design We employ a modular design by splitting a given multi-physics problem into sub-problems. Each sub-problem represents a particular physical process, e.g., flow, thermal, reactions, mechanics, wells. A sub-problem contains a set of subsets, where we define unknowns and properties. We employ physics extension principle by defining the governing equations, computational domain, spatial discretization and nonlinear formulation for each subproblem. As a result, the framework can be viewed as a collection of large modules (subproblems) and their sub-modules (subsets). We arrange these pieces together to solve a particular multi-physics problem. For example, the choice of subsets for a compositionalthermal problem with Two-Point Flux Approximation, natural-variables formulations, and FIM is different from a set of subsets for a compositional-mechanics problem with Nonlinear Finite Volume approximation, molar-variables formulation, and AIM. Flexibility in coupling the different sub-problems is of prime importance. The objective is to provide wide flexibility in the number of different physics modules and how they are coupled. The choice of the coupling strategy is a part of the simulation input. To allow this level of flexibility, two important questions have to be answered. First: how to solve each sub-problem for a particular coupling strategy? Second: how to stitch all the subproblems together to produce a global numerical solution for a given timestep? To deal with the first question, we devise a tree strategy for sub-problems. The second question is handled using a global status table. In the following section, we outline the basis of sub-problems organization, and we elaborate the details of the sub-problem tree and the global status table. Then, we describe the strategy for accommodating complex coupling terms and properties Subsets The concept of subsets lies at the core of the GENIC framework. A subset is an abstract computational domain with a defined set of unknowns and properties. A particular combination of subsets describes a sub-problem. For example, the flow physics module may have a cell-based subset with unknowns, such as pressure p, phase saturations S j and properties, such as relative permeability k r j and phase viscosity µ j. In addition, we may define

46 CHAPTER 2. GENERAL IMPLICIT COUPLING FRAMEWORK (GENIC) 29 (a) (b) (c) Figure 2.5: Example of a flow sub-problem of two blocks with following hypothetical subsets and corresponding unknowns: (a) cell-based (pressure p and phase saturation S p ), (b) face-based (phase velocity v p ) and (c) boundary-based unknowns (boundary flux b). c l a s s F l o w C e l l S u b s e t : p u b l i c SubSet { F l o w C e l l S u b s e t ( n c e l l s, nphase, ncomp,... ) ADD SCALAR( p r e s ) / / p r e s s u r e ADD VECTOR( s a t, nphase ) / / phase s a t u r a t i o n ADD MATRIX( xcp, nphase, ncomp ) / / molar f r a c t i o n of component c i n phase p... / / any o t h e r p r o p e r t y } c l a s s FlowFaceSubset : p u b l i c SubSet { FlowFaceSubset ( n f a c e s, nphase,... ) ADD VECTOR( p h a s e f l u x, nphase ) / / molar phase f l u x... / / any o t h e r p r o p e r t y } c l a s s T h e r m a l C e l l S u b s e t : p u b l i c SubSet { T h e r m a l C e l l S u b s e t ( n c e l l s, nphase,... ) ADD SCALAR( temp ) / / t e m p e r a t u r e ADD VECTOR( e n t h a l p y, nphase ) / / phase e n t h a l p y... / / any o t h e r p r o p e r t y } Listing 2.1: Simplified code example: cell- and face-based subsets for flow, and cell-based subset for thermal sub-problems. Number of blocks is different for each subset (number of cells for cell-based, and number of faces/connections for face-based subsets). a connection-based subset with unknowns defined on the interfaces of a control-volume (e.g. phase velocities v j ) and properties defined on boundaries as shown in Figure 2.5. The geomechanics module employs vertex-based displacements, u, and cell-based properties, such as the fluid pressure p, volumetric strain ε v, and volumetric stress σ v. In Listing 2.1 we define three subsets: FlowCellSubset, FlowFaceSubset and ThermalCellSubset for cell and face-based compositional properties and cell-based thermal properties. Notice that we do not establish pairing between subsets and sub-problems because the same subset can be

47 CHAPTER 2. GENERAL IMPLICIT COUPLING FRAMEWORK (GENIC) 30 Table 2.1: Example of physics modules (sub-problems), their hypothetical subsets properties and corresponding unknowns. We can extend capabilities of existing sub-problems by adding additional subsets or extending existing ones. Sub-problem Subset Unknowns and properties cells p, S j, x i j, µ, ρ,... FLOW faces v j boundaries boundary conditions cells T, h j, u j, h tot,... THERMAL faces v e j boundaries boundary conditions vertices u cells p MECHANICS faces traction values integration points ε, σ, C... boundaries boundary conditions used by different sub-problems as needed. Table 2.1 represents three physics modules: Flow, Thermal, and Mechanics with their hypothetical subsets. Each subset has a list of related properties. We can extend this example with additional physics modules, subsets and unknowns. Listing 2.2 demonstrates the relationship between a sub-problem and its subsets. Notice that we can reuse subsets from other sub-problems. This is important for well sub-problems, or advanced applications of the framework such as multi-scale formulations (Jenny et al., 2003; Lunati and Jenny, 2006; Hajibeygi et al., 2008), or domain decomposition (Killough and Wheeler, 1987; Wheeler et al., 1999). We define status as a specific choice and ordering of the problem unknowns. The status change may involve dynamic variable switching, which is used in various formulations of multiphase flow models when phases appear or disappear (Voskov and Tchelepi, 2012). Moreover, it is often the case that the number of unknowns is different for each status, which implies that the size of the residual vector and the Jacobian matrix can change. The Automatically Differentiable Expression Templates Library (ADETL) (Younis, 2011;

48 CHAPTER 2. GENERAL IMPLICIT COUPLING FRAMEWORK (GENIC) 31 c l a s s FlowProblem : p u b l i c SubProblem { FlowProblem (... ) c r e a t e S u b s e t s ( ) ; F l o w C e l l S u b s e t * f l o w c e l l s u b s e t ; FlowFaceSubset * f l o w f a c e s u b s e t ;... } FlowProblem : : c r e a t e S u b s e t s ( ) { / / c r e a t e each s u b s e t f l o w c e l l s u b s e t = new F l o w C e l l S u b s e t ( n c e l l s, nphase, ncomps,... ) ; f l o w f a c e s u b s e t = new FlowFaceSubset ( n f a c e s, nphase ) ; / / add s u b s e t s i n t o t h e g l o b a l J a c o b i a n a d d S u b s e t ( f l o w c e l l s u b s e t ) ; a d d S u b s e t ( f l o w f a c e s u b s e t ) ; } Listing 2.2: Simplified code example: flow sub-problem consists of cell and face subsets. Zhou, 2012) provides a capability to define a status table (a set of statuses with a corresponding order of unknowns) and automatically rearranges the column structure and enumeration of the variables when the status changes. A variable formulation defines a choice of a status table. We apply different formulations (i.e. status tables) of the variables for each subset independently. The resulting mixed formulation for a multi-physics problem is formed using the subsets of those individual formulations through a combination of related status tables. For example, Figure 2.6 depicts two formulations for flow (natural and molar) and two formulations for thermal (temperature and enthalpy) problems. Instead of defining a new formulation for a coupled compositional-thermal problem, a mixed formulation is automatically employed based on applied formulations for each involved physics. This leads to four possible mixed formulations. The automatic construction of mixed formulation is of great importance since the number of formulations for a coupled multi-physics problems grows very quickly with the number of formulations for each physics. Listing 2.3 demonstrates related code where each formulation for each sub-problem is enabled independently. We first define formulations for each physics. In the presented case, a compositional formulation such as natural or molar is the implementation of InterfaceFlowFormulation class, while thermal formulation is the implementation of InterfaceThermalFormulation. Then, we independently assign formulations for each physics. Since each subset is introduced independently from other subsets, we may enable or

49 CHAPTER 2. GENERAL IMPLICIT COUPLING FRAMEWORK (GENIC) 32 c l a s s I n t e r f a c e F l o w F o r m u l a t i o n : p u b l i c F o r m u l a t i o n { I n t e r f a c e F l o w F o r m u l a t i o n (... ) / / used s u b s e t s F l o w C e l l S u b s e t * f l o w c e l l s u b s e t ; FlowFaceSubset * f l o w f a c e s u b s e t ; c o n s t T h e r m a l C e l l S u b s e t * t h e r m c e l l s u b s e t ; / / used o nly t o a c c e s s t e m p e r a t u r e... } c l a s s I n t e r f a c e T h e r m a l F o r m u l a t i o n : p u b l i c F o r m u l a t i o n { I n t e r f a c e T h e r m a l F o r m u l a t i o n (... ) T h e r m a l C e l l S u b s e t * t h e r m c e l l s u b s e t ; / / no need f o r flow s u b s e t s... } S i m u l a t o r : : s e t F o r m u l a t i o n s (... ) { / / p i c k any i m p l e m e n t a t i o n of I n t e r f a c e F l o w F o r m u l a t i o n / / N a t u r a l / Molar F o r m u l a t i o n s a r e i m p l e m e n t a t i o n of I n t e r f a c e F l o w F o r m u l a t i o n pflowproblem >s e t F o r m u l a t i o n ( new N a t u r a l F o r m u l a t i o n ) ; } / / p i c k any i m p l e m e n t a t i o n of I n t e r f a c e T h e r m a l F o r m u l a t i o n / / Thermal / E n t h a l p y F o r m u l a t i o n a r e i m p l e m e n t a t i o n of I n t e r f a c e T h e r m a l F o r m u l a t i o n pthermalproblem >s e t F o r m u l a t i o n ( new T h e r m a l F o r m u l a t i o n ) : Listing 2.3: Simplified code example: each sub-problem has its own formulation (mixed Natural and Thermal formulation). c l a s s F V D i s c r e t i z a t i o n F o r m u l a t i o n : p u b l i c F o r m u l a t i o n { F V D i s c r e t i z a t i o n F o r m u l a t i o n (... ) F l o w C e l l S u b s e t * f l o w c e l l s u b s e t ; / / used s u b s e t s... } c l a s s M F E D i s c r e t i z a t i o n F o r m u l a t i o n : p u b l i c F o r m u l a t i o n { M F E D i s c r e t i z a t i o n F o r m u l a t i o n (... ) F l o w V e r t i c e s S u b s e t * f l o w v e r t i c e s s u b s e t ; / / used s u b s e t s... } c l a s s FlowProblem : p u b l i c SubProblem { FlowProblem (... ) / / used s u b s e t s, a l l r e q u i r e d s u b s e t s f o r both d i s c r e t i z a t i o n a r e p r e s e n t / / n o t e t h a t f a c e / b o u n d a r i e s s u b s e t s a r e n o t shown f o r b r e v i t y F l o w C e l l S u b s e t * f l o w c e l l s u b s e t ; F l o w V e r t i c e s S u b s e t * f l o w v e r t i c e s s u b s e t ;... } S i m u l a t o r : : s e t F o r m u l a t i o n s (... ) { i f ( finitevolumemethod ) / / f l o w v e r t i c e s s u b s e t i s d i s a b l e d and n e v e r a l l o c a t e d f o r FV pflowproblem >s e t F o r m u l a t i o n ( F V D i s c r e t i z a t i o n F o r m u l a t i o n ) ; e l s e i f ( mixedfiniteelementmethod ) / / f l o w c e l l s u b s e t i s d i s a b l e d and n e v e r a l l o c a t e d f o r MFE pflowproblem >s e t F o r m u l a t i o n ( M F E D i s c r e t i z a t i o n F o r m u l a t i o n ) ; } Listing 2.4: Simplified code example: FV and MFE space discretization for a flow sub-problem. Each class below contains face and boundaries subset which are not shown for brevity.

50 CHAPTER 2. GENERAL IMPLICIT COUPLING FRAMEWORK (GENIC) 33 Sub-problem Formulations Unknowns FLOW natural p, S p, x cp FLOW molar p, z c THERMAL temperature T THERMAL enthalpy h p (a) Mixed Formulations natural and temperature natural and enthalpy molar and temperature molar and enthalpy (b) Unknowns p, S p, x cp, T p, S p, x cp, h p p, z c, T p, z c, h p Figure 2.6: Example of mixing natural and molar formulations for a flow sub-problem, and temperature and enthalpy formulations for a thermal sub-problem: (a) list of sub-problem and formulations, (b) possible mixed formulations with unknowns. Flow Subsets FV MFE cell-based face-based boundaries vertices-based Figure 2.7: Activation logic of subsets for different space discretization schemes for a flow sub-problem. disable any subset for a given problem throughout a simulation run, or at the initialization stage. When a subset is disabled, it is removed at the Jacobian level and corresponding memory is freed. We can create an alternative sub-problem package without changing its core functionality by enabling/disabling related subsets. For example, we might introduce a Mixed Finite Element (MFE) formulation for the flow sub-problem by adding additional subsets defined on the vertices (Figure 2.7). The residuals of both Finite Volume (FV) and MFE methods contain similar terms (accumulation, fluxes, and source/sink) and vary only in the subset choice, and how these terms are calculated. Listing 2.4 demonstrates the reuse of subsets using formulations FV DiscretizationFormulation and MFE DiscretizationFormulation for FV and MFE space discretization. Sub-problem FlowProblem contains all subsets from both formulations, however, only part of them are

51 CHAPTER 2. GENERAL IMPLICIT COUPLING FRAMEWORK (GENIC) 34 Figure 2.8: Example of flow-thermal-mechanics problem. S M, S F, S T, S Tiso represent all subsets related to mechanics, flow, thermal and iso-thermal sub-problems correspondingly. Mechanics sub-problem includes over- and under-burden and, therefore, is larger than a reservoir. The reservoir domain S F has two parts: thermal S T and iso-thermal S Tiso. used at any moment. The selected subsets depend on the chosen formulation. The overall flexibility of the framework depends strongly on how subsets are designed and implemented. Subsets can be identical in terms of the variable sets, yet a subset can be applied to different domains. At the same time, different subsets can belong to different parts of the overall problem. For example, we may have a small domain for the flow sub-problem and a large domain for the mechanics sub-problem which is often the case for problems with geomechanical effects. Figure 2.8 shows an example of a flow-thermalmechanics problem, where S M, S F, S T, S Tiso represent all subsets related to mechanics, flow, thermal and iso-thermal sub-problems, respectively. Each of those subsets has different domain size: mechanics subsets S M include over- and under-burden and, therefore, cover a larger domain than the reservoir. The reservoir domain S F has two parts: thermal S T and iso-thermal S Tiso. Listing 2.5 is a code example of how different size of subsets is supported in the framework. Notice that mapping of properties between different physics requires explicit management. For example, both cells A and B are the first elements for the mechanics S M and the reservoir S F domains correspondingly. However, the cell B has another index in the domain S M. Such mapping between physics defined on different domains should be provided on the input stage. In addition, for low-order transport equations, we can separate the flux part from the accumulation part (Maes et al., 2016) using connection-based subsets. Similarly, we can decouple other terms for any sub-problem. The GENIC framework supports all of the

52 CHAPTER 2. GENERAL IMPLICIT COUPLING FRAMEWORK (GENIC) 35 s i z e t nsm = 1000; / / number of b l o c k s i n mechanics domain s i z e t nsf = 2 00; / / number of b l o c k s i n r e s e r v o i r domain ( flow ) s i z e t nst = 1 60; / / number of b l o c k s i n t h e r m a l domain s i z e t nstiso = nsf nst ; / / number of b l o c k s i n i s o t h e r m a l domain pmechanicsproblme >m e c h c e l l s u b s e t = new MechCellSubset (nsm,... ) pflowproblem >f l o w c e l l s u b s e t = new F l o w C e l l S u b s e t ( nsf,... ) ; pthermalproblem >t h e r m a l c e l l s u b s e t = new T h e r m a l C e l l S u b s e t ( nst,... ) ; pthermalproblem >i s o t h e r m a l c e l l s u b s e t = new I s o t h e r m a l C e l l S u b s e t ( nstiso,... ) ;... / / d e f i n e e x p l i c i t mapping Listing 2.5: Simplified code example: different size domain for each sub-problem. (a) (b) Figure 2.9: (a) Domain decomposition example: S 1,..S 9 represents identical flow-thermal subsets related to different parts of a global simulation domain, shaded cells represent additional boundaries subsets b 1,..b 9 (b) Multiscale example: a primary coarse grid (solid thick lines) and the dual coarse grid (dashed lines) form subsets S c and S d correspondingly; the dual grid naturally defines a partition of the fine-mesh into internal (white cells), edge (shaded cells), and node points (grey cells) of the dual grid - subsets S f, S e and S n. above-mentioned capabilities. We elaborate on the details of coupling terms in Section Additionally, subsets may be used for domain decomposition and multi-scale approaches. We employ flow-thermal subsets S 1,..S k (where k is a number of domains) to solve a problem using a domain decomposition, where for each domain we specify a related boundaries subset (see Figure 2.9a). Alternatively, subsets of properties defined on a fine grid, dual and coarse grids allow for multiscale formulations and solution schemes (Figure 2.9b). While GENIC framework helps with a structure to solve these problems, additional code modifications are required to fully support these capabilities. These modifications are discussed in detail in Chapter 4.

53 CHAPTER 2. GENERAL IMPLICIT COUPLING FRAMEWORK (GENIC) 36 (a) (b) Figure 2.10: Example of extended fixed-stress split strategy for a flow-thermal-mechanics problem: (a) a schematic representation of split and (b) the corresponding tree structure. Note that Linear solver A is applied for both FLOW and THERMAL sub-problems, while Linear solver B is applied only for ME- CHANICS sub-problem Sub-Problem Tree Structure and Global Status Table A tree structure provides a general representation of a given coupling scheme defined by Eq Each inner node in the tree defines a coupling strategy ϒ (FIM or SEQ) that is applied to the child nodes. Each terminal node represents a particular physics problem with a dedicated linear solver applied to a single or multi-physics block. Figure 2.10 explains this concept for a flow-thermal-mechanics problem. We have one head node with the global coupling strategy Ψ = SEQ[FIM[FLOW,THERMAL], MECHANICS], one internal node for coupling between two physics Ψ 2 = FIM[FLOW,THERMAL], and two terminal nodes for flow-thermal and mechanics physics. We extend the status table concept to allow for flexible coupling. First, we mark all possible statuses within a status table with non-fixed unknowns as active. Second, we add an auxiliary inactive status when all unknowns are fixed (i.e. derivatives are not formed). Consequently, a subset can be active, or inactive, at any given moment. Table 2.2 shows an example of a variable set for black-oil model with a corresponding Jacobian block structure. We define a sub-problem as active when any of its enabled subsets is active and inactive otherwise (Figure 2.11). An active sub-problem forms a corresponding part of the Jacobian matrix. Similarly, for a sub-problem in the inactive state, its Jacobian part is not formed. Notice that auxiliary subsets used by a sub-problem with only defined properties do not form derivatives once related unknowns are in the inactive state. Figure 2.12 shows activa-

54 CHAPTER 2. GENERAL IMPLICIT COUPLING FRAMEWORK (GENIC) 37 Table 2.2: Example of a variable set for three-phase black-oil problem expanded by inactive status. No derivatives are formed for inactive subset status. Local Subset Status p S o S g R S Jacobian Block Structure Global State Status O, W J 3x3 active O, G J 3x3 active inactive Figure 2.11: Subset and subproblem activation state. Sub-problem is inactive only when all enabled subsets S 1 and S 2 are inactive, disabled subsets S 3 and S 4 do not affect the active status of its sub-problem. tion logic of subsets for cell-based subset S C and face-based subset S F. Subset S C defines both unknowns X C and properties Y C, while subset S F is an auxiliary subset and contains only properties Y F, which are a function of unknowns from subset S C. Both active subsets form a full Jacobian matrix. When subset S C is active only related part of Jacobian is formed. However, when S F is the only active subset - Jacobian is empty because properties Y F depend on unknowns X C which are fixed. / / G l o b a l S t a t u s Table c o n s t r u c t e d from SEQ[ FIM [FLOW,THERMAL], MECHANICS] p l u s FIM / / s t a t u s FLOW THERMAL MECHANICS / / [ 0 ] t r u e t r u e f a l s e / / [ 1 ] f a l s e f a l s e t r u e / / [ 2 ] t r u e t r u e t r u e d o u b l e d t = 0. 1 ; / / c u r r e n t t i m e s t e p s i z e i n days / / one i m p l i c i t s e q u e n t i a l i t e r a t i o n f o r e x t e n d e d f i x e d s t r e s s s p l i t subproblems. s e t G l o b a l S t a t u s ( 0 ) ; / / a c t i v a t e FLOW & THERMAL subproblems. s o l v e ( d t ) ; / / s o l v e FIM [FLOW, THERMAL] subproblems. s e t G l o b a l S t a t u s ( 1 ) ; / / a c t i v a t e MECHANICS subproblems. s o l v e ( d t ) ; / / s o l v e MECHANICS o nly / / f u l l y c o u p l e d a p p r o a c h subproblems. s e t G l o b a l S t a t u s ( 2 ) ; / / a c t i v e a l l sub problems t o g e t h e r subproblems. s o l v e ( d t ) ; / / s o l v e FIM [FLOW, THERMAL, MECHANICS] Listing 2.6: Simplified code example: sub-problems activation depends on the global status. Note there is a unified interface for different coupling strategies.

55 CHAPTER 2. GENERAL IMPLICIT COUPLING FRAMEWORK (GENIC) 38 Subsets Unknowns Properties cell-based S C X C Y C face-based S F Y F (a) Active Subset J = [ RC X C 0 S C S C and S F S F ] [ ] [ R C Y F = 0 RC R F = R R C RC C X Y F = 0 C J = X C Y F J = X C = 0 R C Y F = 0 R 0 0 F Y F = 0 (b) R F Y F ] = Figure 2.12: Subset activation logic for different subsets. On top (a) - definition of subsets and related unknowns X, and properties Y. Face-based subset S F contains only properties Y F (for example, phase velocity v p ) without unknowns. At the bottom (b) - activation logic of subsets affects the Jacobian matrix. Both active subsets form full Jacobian matrix. When subset S C is active only related part of Jacobian is formed. However, when S F is only active subset - Jacobian is empty, because properties Y F depend on unknowns X C which are in non-active state (i.e. fixed and do not have derivatives), as a result R F Y F = 0. A given coupling scheme defines a set of states when a related physics sub-problem needs to be solved, or not (see Eq ). These states form the global status table, which controls the activation and deactivation logic of the physics modules. When the fully implicit strategy is applied, all the physics modules are active, and a full Jacobian matrix is formed. However, if one of the physics modules is inactive, no derivatives will be constructed for this sub-problem. The combination of the sub-problem tree structure and the global status table provides several unique improvements over existing multi-physics simulators. First, the desired coupling strategy can be chosen simply by activating and deactivating specific sub-problems. Second, such approach facilitates an efficient code design: no specific property storage, conditional branches or interfaces are required to couple physics modules that use a sequential strategy; as well as it avoids an excessive copying of data between the physical modules. Finally, we can test hybrid schemes: one can employ a fully coupled strategy for strongly coupled domains of the problem and apply a sequentially coupled strategy for the rest (see, e.g., Tomin and Lunati (2015) for a discussion). Listing 2.6 explicitly shows the usage of a global status table activation logic for solving two sub-problems together or separately.

56 CHAPTER 2. GENERAL IMPLICIT COUPLING FRAMEWORK (GENIC) 39 Figure 2.13, 2.14 and 2.15 represent several examples of different coupling strategies applied for a four-physics problem, namely, flow, thermal, mechanics and chemical reactions. For each example, we specify the associated tree structure and the corresponding global status table. Each terminal node has an assigned global status with a particular linear solver. Figure 2.15 emphasizes that the fully implicit coupling strategy is just another case of the global status table with a single state when all sub-problems are active. Unlike a subset status table where the order of unknowns is flexible (for details refer to Younis (2011)), the order of sub-problems in the global status table is fixed once the corresponding tree is constructed. This guarantees efficient memory allocation for the linear solver and proper update of coupling terms. The framework generates a consistent global status table from the sub-problem tree. There is no need to control it manually unless the nonstandard strategy is tested. One example of such a strategy is a hybrid scheme where we mix fully coupled and sequentially coupled strategies. Listing 2.7 demonstrates an example of a flow-thermal-mechanics problem that is solved in the fully implicit manner for the first 100 days, followed by extended fixed-stress split for the rest of the simulation. Details of the solution algorithm for a general multi-physics problem and convergence are outlined in Section Coupling Terms This section describes the mechanism to support splitting of a sub-problem to treat its terms independently, or to describe the coupling terms of the different physics. For example, wells are coupled with flow and thermal physics by a source/sink term in the corresponding equations. We can rewrite the compositional and energy equations (Eq. 2.1 Eq. 2.6) as follows: R F g F,i + ff,i W = 0, i = 1..n c, (2.29) R T g T + ft,i W = 0, where f W F,i = n p j=1 x i jρ j q j is the well contribution for a multi-component transport, g F,i and g T are Eq. 2.1 and Eq. 2.6 without source/sink input, and f W T,i = n p j=1 h jρ j q j is the well contribution for a thermal part. Each well in the framework is modeled as a separate

57 CHAPTER 2. GENERAL IMPLICIT COUPLING FRAMEWORK (GENIC) 40 d o u b l e tmax = 300; / / t o t a l s i m u l a t i o n t ime i n days d o u b l e t f i m = 100; / / FIM s i m u l a t i o n t ime i n days d o u b l e t = 0, d t = 0. 1 ; / / c u r r e n t t ime and t i m e s t e p / / FIM s t r a t e g y a p p l i e d f i r s t subproblems. s e t G l o b a l S t a t u s ( 2 ) ; w h i l e t < t f i m { subproblems. s o l v e ( d t ) ; / / s o l v e FIM [FLOW, THERMAL, MECHANICS] t = t + d t ; / / u p d a t e time } / / e x t e n d e d f i x e d s t r e s s s p l i t a p p l i e d l a t e r w h i l e t < tmax { / / check c o n v e r g e n c e of r e l a t e d problems w h i l e ( n o t converged ( subproblems ) ) { } subproblems. s e t G l o b a l S t a t u s ( 0 ) ; subproblems. s o l v e ( d t ) ; / / s o l v e FLOW & THERMAL subproblems. s e t G l o b a l S t a t u s ( 1 ) ; subproblems. s o l v e ( d t ) ; / / s o l v e MECHANICS } t = t + d t ; / / u p d a t e time Listing 2.7: Simplified code example demonstrates a hybrid scheme where a flow-thermal-mechanics problem is solved in fully implicit manner for first tfim = 100 days, followed by extended fixed-stress split for the rest of the simulation (tmax = 300 days). Global status table is taken from Listing 2.6, timestep is fixed for simplicity. (a) Global Status FLOW MECHANICS THERMAL REACTIONS status 1 status 2 status 3 status 4 (b) Figure 2.13: Example 1: (a) sub-problem tree and (b) corresponding global status table. The coupling strategy is Ψ = SEQ[FLOW, SEQ[MECHANICS, THERMAL], REACTIONS], where the global sequential approach is used to couple all the sub-problems together (ϒ A = SEQ), and the mechanics and thermal sub-problems are solved sequentially (ϒ B = SEQ).

58 CHAPTER 2. GENERAL IMPLICIT COUPLING FRAMEWORK (GENIC) 41 (a) Global Status FLOW MECHANICS THERMAL REACTIONS status 1 status 2 status 3 (b) Figure 2.14: Example 2: (a) sub-problem tree and (b) corresponding global status table. The coupling strategy is Ψ = SEQ[FLOW, FIM[MECHANICS, THERMAL], REACTIONS], where the global sequential approach is used to couple all the sub-problems together (ϒ A = SEQ), but the geomechanics and thermal sub-problems are solved in a fully implicit manner (ϒ B = FIM). Status 1 activates FLOW, status 2 activates THERMAL and MECHANICS simultaneously, and status 3 activates REACTIONS. sub-problem with corresponding subsets. For example, a simple well has subsets for flow and thermal perforations, while a more complex multi-segment well has additional subsets for its segments. However, a well sub-problem is different from flow, thermal or mechanics sub-problems. While each sub-problem might be coupled with other sub-problems via properties (temperature T in the flow equations for EoS parameters, or pressure p as an input for mechanics equations), a well sub-problem has additional terms to other subproblems such ff,i W for flow and f T,i W for thermal. These additional terms should appear both for active and inactive well sub-problems. For example, Eq can be solved using different coupling strategies such as Ψ 1 = FIM[FLOW,THERMAL, WELLS], Ψ 2 = SEQ[FIM[FLOW,WELLS], THERMAL] or Ψ 3 =SEQ [FLOW, FIM[THERMAL,WELLS]]. A fully-coupled strategy Ψ 1 is solved when all sub-problems are active, while THERMAL and FLOW sub-problems are solved with inactive wells for Ψ 2 and Ψ 3 correspondingly. Well related terms ff,i W and f T,i W should appear in flow and thermal equations both in active and inactive states (well phase inflow q is fixed for the inactive

59 CHAPTER 2. GENERAL IMPLICIT COUPLING FRAMEWORK (GENIC) 42 (a) Global Status FLOW MECHANICS THERMAL REACTIONS status 1 (b) Figure 2.15: Example 3: (a) sub-problem tree and (b) corresponding global status table. The fully coupling strategy is Ψ = FIM[FLOW, MECHANICS, THERMAL, REACTIONS]. state). Additionally, we may employ a coupling strategy Ψ 4 = SEQ[FIM[FLOW,WELLS], FIM[THERMAL,WELLS]], where sub-problems WELLS are always active. The last strategy is allowed in the framework since there are no restrictions of uniqueness in a construction of a global status table. We extend the framework by adding coupling terms. The logic of an active and inactive state for additional coupling terms is identical to the activation logic of subset and is enabled automatically via subsets. However, the appearance of additional terms should be specified for each sub-problem manually. Thus, we construct the residuals of sub-problems in two steps. First, we build residual for each active sub-problem, then we add related coupling terms for all active sub-problems. Using the coupling term mechanism, we can decouple a sub-problem s into series of sub-problems s i (i = 1...n s ), where each s i is solved with fixed terms from other sub-problems. For example, we may solve a flux and an accumulation term in a sequential manner for a flow sub-problem by splitting it into two sub-problems with one additional term each. This splitting is of practical interest since it can improve convergence for some problems as reported by Maes et al. (2016) Coupled Properties In this section, we describe how to treat properties that are shared by different physics equations and modules. For example, porosity φ (see Eq. 2.17) is a function of cell pressure

60 CHAPTER 2. GENERAL IMPLICIT COUPLING FRAMEWORK (GENIC) 43 Figure 2.16: Example of a coupled property. Here we have three sub-problems MECHANICS, FLOW and THERMAL with corresponding mech cell, flow cell and therm cell subsets. Solid lines between subproblems and subsets indicate that properties defined in these subsets can be changed by a corresponding sub-problem, while dashed lines indicate that these properties are fixed for a related sub-problem. There are two coupled properties introduced in this system. Porosity is a function of volumetric stress σ v (mech cell subset) and pressure p (flow cell subset), while mobility depends on pressure p and composition x cp from flow cell subset, and on temperature T from therm cell subset. Notice flow cell subset coupled with mechanics sub-problem via porosity. p (flow subset), volumetric strain ε v, and mean stress σ v (mechanics subsets), while the phase mobility λ j is a function of pressure p, phase saturation S j, compositions x i j (flow subsets), and temperature T or enthalpy h (thermal subset). Figure 2.16 demonstrates the relationship between subsets, sub-problems and coupled properties, such as mobility and porosity. While in this example both coupled properties depend on two subsets there is no such limitation in the framework. A coupled property can be a function of multiple subsets and multiple sub-problems. We can present a coupled property as a sub-problem without its own subsets and governing equations and with only one defined property belonging to other subsets. Both coupled properties and regular (non-coupled) properties relate to subsets; however, there are several major differences between them: (i) A property is a function of several subsets of the same problem, while a coupled property is a function of several subsets from different sub-problems. We link subsets of other sub-problems to the coupled property at the construction stage. This is similar to a sub-problem when it uses other subsets without modifying them. (ii) All subsets and all related properties of the same sub-problem are activated and deactivated simultaneously. When a sub-problem is an inactive state, corresponding property is fixed and does not require updates. For a coupled property that does not

61 CHAPTER 2. GENERAL IMPLICIT COUPLING FRAMEWORK (GENIC) 44 hold: a property is a combination of active and inactive subsets and updates every time when any related active subset is updated. For example, for both coupling strategies Ψ 1 = FIM[FLOW, MECHANICS] and Ψ 2 = SEQ[FLOW,MECHANICS] porosity φ is always calculated and updated. In Ψ 2, porosity φ is a function of fixed mechanics variables (volumetric stress σ v ) and non-fixed flow variables (pressure p) when solving a flow sub-problem. Similarly, when solving a mechanics sub-problem porosity is a function of fixed flow properties and non-fixed mechanics unknowns. (iii) Whenever we apply a global status switch, we calculate all coupled properties first, followed by computation of the properties that are associated with active sub-problems. This guarantees the correctness of derivatives produced by automatic differentiation. In addition, we can update related coupled properties when the corresponding subset is updated. (iv) The order of the sub-problems and their properties is invariant. However, the order of coupled properties depends on the sequential evaluation of corresponding functions used for this property. For example, in thermal-mechanics problems, we may have two coupled properties: porosity φ and enthalpy h, where φ is a function of h. While the order of physics (thermal followed by mechanics, or mechanics followed by thermal) is not critical, the order of coupled properties is strictly defined by the corresponding relations, thus, h should be calculated prior to φ. The flexibility of order of coupled properties can be greatly extended using a concept of dependency properties graph described in Chapter 4. Listing 2.8 demonstrates a coupled property (porosity φ) structure and usage in the framework. Any other coupled property can be defined in a similar fashion. Since a coupled property relates to subsets and not to sub-problems, the relationship towards a sub-problem is established through a connection between a sub-problem and its subsets Mapping Operators While the presented framework is flexible in terms of physics and coupling properties, sometimes it is still limited by the stability of coupling strategies. For flow and transport

62 CHAPTER 2. GENERAL IMPLICIT COUPLING FRAMEWORK (GENIC) 45 c l a s s P o r o s i t y : p u b l i c C o u p l e d P r o p e r t y { P o r o s i t y (... ) l i n k S u b s e t s (... ) F l o w C e l l S u b s e t * f l o w c e l l s u b s e t ; / / p o r o s i t y i s a p a r t of t h i s s u b s e t c o n s t M e c h a n i c s C e l l S u b s e t * m e c h c e l l s u b s e t ; / / l i n k e d f i x e d s u b s e t... } P o r o s i t y : : compute ( i n t i b l o c k ) { f l o w c e l l s u b s e t >p o r o s i t y [ i b l o c k ] = func ( f l o w c e l l s u b s e t, m e c h c e l l s u b s e t ) ; } Listing 2.8: Simplified code example for defining porosity as a coupled property, subsets defined by linksubsets() method. Figure 2.17: Modified sequential implicit (m-sfi) approach for a flow-thermal problem. Color annotation: yellow active domain, gray inactive domain. We iteratively solve full domain for FLOW together with a strongly coupled domain for THERMAL, followed by solving THERMAL in the rest. splitting in compositional flow simulation, one way to improve the convergence properties of a sequential coupling scheme is to apply a modified sequential implicit (m-sfi) algorithm (Moncorgé et al., 2017). The m-sfi approach separates the strongly coupled domain where it solves fully coupled flow and transport, and the weakly coupled domain where it solves sequentially flow and transport. Using this coupling strategy we take advantage of the sequential approach in the regions, taking the advantage of the stability of the fully coupled approach in the rest. In this work, we extend the m-sfi approach for general multi-physics problems. Figure 2.17 demonstrates the m-sfi approach for a flow-thermal problem, where we have a strong coupling between flow and thermal in the two-phase regions around wells and weak coupling in the rest of the reservoir.

63 CHAPTER 2. GENERAL IMPLICIT COUPLING FRAMEWORK (GENIC) 46 Figure 2.18: Schematic representation of m-sfi approach. We apply coupling strategy Ψ = SEQ[FIM[FLOW, THERMAL<M1>], THERMAL<M2>], where M1 and M2 are mapping operators for strongly and weakly coupled physics correspondingly. We extend the flexibility of the framework with mapping operators. A mapping operator specifies the active domain and can dynamically change through a simulation run. It has several practical applications such as m-sfi, domain decomposition, and multi-scale. Figure 2.17 is schematically represented by Figure We specify two mapping operators M1 and M2, where M1 defines a strongly coupled domain, and M2 relates to a weakly coupled part of the reservoir. The criteria for strong coupling may include the number of phases, CFL number, and distance to a well. Using the mapping operator concept we can write the m-sfi approach for flow-thermal problem in a general form as Ψ = SEQ[FIM[FLOW, THERMAL<M1>], THERMAL<M2>], where <> relates to an applied mapping operator for a given sub-problem. Similarly, we can write m-sfi for other physics by applying another mapping operators for different physics, and changing the order as needed. In subsection we introduced one approach to handle domain decomposition by specifying different subsets for each subdomain. While it has a lot of flexibility in terms of the interaction between subsets, this approach is difficult for multi-physics domain decomposition extension. An alternative approach is to specify different mapping operators for each subset. For example, we can specify domain-decomposition approach for a flow problem as coupling strategy Ψ = SEQ[FLOW<DD-1>,...,FLOW<DD-N>], where DD-K is a mapping operator to activate K s sub-domain. The GENIC framework supports mapping operators through the extension of the global status table, where we replace a global status from the activation state (active or inactive) by a corresponding mapping operator. For example, Ψ 1 = SEQ[FLOW,THERMAL] can be

64 CHAPTER 2. GENERAL IMPLICIT COUPLING FRAMEWORK (GENIC) 47 Global Status FLOW THERMAL status 1 status 2 (a) Global Status FLOW THERMAL status 1 FULL NONE status 2 NONE FULL (b) Figure 2.19: Extension of global status table concept: a) Ψ 1 = SEQ[FLOW,THERMAL] and b) Ψ 2 = SEQ[FLOW<FULL>,THERMAL<FULL>]. Coupling strategy Ψ 2 is Ψ 1 written with mapping operators. Here, mapping operator <FULL> specifies a whole applied domain, and <NONE> denotes an inactive domain. Global Status FLOW THERMAL status 1 FULL M1 status 2 NONE M2 Figure 2.20: Example of global status table for m-sfi coupling strategy Ψ = SEQ[FIM[FLOW, THERMAL<M1>], THERMAL<M2>] rewritten explicitly with mapping operators as Ψ 2 = SEQ[FLOW<FULL>, THERMAL<FULL>] (see Figure 2.19). Here, mapping operator <FULL> specifies a whole applied domain, and <NONE> denotes an inactive domain. Figure 2.20 shows a global status table for m-sfi coupling strategy Ψ = SEQ[FIM[FLOW, THERMAL<M1>], THERMAL<M2>]. For optimal efficiency, the mapping operator implementation is a part of the ADETL library. Notice that each mapping operator can be updated after each Newton iteration. Listing 2.9 demonstrates an example of the M1 mapping operator where full physics are required. Mapper M1 is an implementation of interface class Mapper where we only redefine updatemapping() function. We add cells with strong coupling based on CFL criteria and number of phases into the vector of active cells m actives. Any mapping operator can be defined in a similar fashion and will be automatically enabled in the framework Solving Algorithm Typically, different algorithms are used with different coupling strategies (Zienkiewicz et al., 1988; Settari and Walters, 2001; Kim et al., 2013; Castelletto et al., 2015). However, because every solution scheme for a multi-physics problem in the GENIC framework is represented by a tree structure and a global status table, we can apply a general algorithm for any coupling strategy for which a tree structure can be constructed. We start solving

65 CHAPTER 2. GENERAL IMPLICIT COUPLING FRAMEWORK (GENIC) 48 c l a s s Mapper M1 : p u b l i c Mapper { p u b l i c : Mapper MSFI T1 (... ) v i r t u a l void updatemapping ( ) { m a c t i v e s. c l e a r ( ) ; f o r ( s i z e t i b = 0 ; i b < t h e r m c e l l s u b s e t >num blocks ( ) ; ++ i b ) { / / two phase, w e l l or c f l c r i t e r i a f o r s t r o n g c o u p l i n g i f ( i s t w o p h a s e ( i b ) i s w e l l ( i b ) c f l ( i b ) > m i n c f l ) m a c t i v e s. p u s h b a c k ( i b ) ; } u p d a t e i s a c t i v e ( ) ; } ; p r i v a t e : / / used s u b s e t s F l o w C e l l S u b s e t * f l o w c e l l s u b s e t ; T h e r m a l C e l l S u b s e t * t h e r m c e l l s u b s e t ; } ; Listing 2.9: Simplified code example for defining active cells for flow-thermal m-sfi approach. our system at the head node, which always has at least one child node, by applying the SOLVE function outlined in Algorithm 1. Note that the SOLVE function is recursively applied to each node of the tree. In Step 2 of the algorithm, we check the terminal condition, i.e., this node has no children nodes. Each terminal node has a corresponding status in the global status table. We activate the global status at Step 3 (see Algorithm 2 for details) and on Step 4 we solve active sub-problems using a dedicated solver and return the convergence state. Steps 5 9 describe the convergence logic of a local node; it is similar to a global convergence check of a node (Step 18 27) with some key differences. First, the global convergence is often achieved after several sequential iterations (as outlined in Step 29), while the local convergence takes only one sequential iteration. Second, global convergence is achieved when a residual norm is below a threshold prior to solving a related linearized system of equations, i.e. at the first Newton iteration (Step 22). Local convergence, instead, is achieved when the residual norm is below the threshold at the last Newton iteration (Step 7).

66 CHAPTER 2. GENERAL IMPLICIT COUPLING FRAMEWORK (GENIC) 49 Algorithm 1: Node solve function pseudocode. The function is called recursively in the sub-problems tree passing algorithm. Here node is the sub-problem tree node, node.children gives an ordered set of corresponding children elements, k is a number of sequential iterations between node.children tree nodes, and k max is a maximum number of such iterations. 1: function SOLVE(node) 2: if node is terminal then 3: activate global status see Algorithm 2 for details 4: solve active sub-problems S A using a dedicated linear solver 5: for all s S A do check local convergence 6: if R s > ε s then residual norm at the last iteration 7: return node is NOT converged 8: end for 9: update variables related to S A 10: return node is converged 11: else 12: set k = 0 13: do 14: for all i node.children do 15: if SOLVE(node.children[i]) is not converged then 16: return node is NOT converged stop in case of convergence problem 17: end for 18: set conv = true check global convergence 19: for all i node.children do 20: find active sub-problems S A related to node i 21: for all s S A do 22: if Rs 1 > ε s then residual norm at the first iteration 23: conv = f alse 24: end for 25: end for 26: if conv is true then 27: return node is converged 28: set k = k : while k < k max 30: return node is NOT converged 31: end function

67 CHAPTER 2. GENERAL IMPLICIT COUPLING FRAMEWORK (GENIC) 50 We apply a sequential iterative strategy for the children nodes when they exist, i.e., the node is not a terminal node. We solve each child node independently using a recursive procedure SOLVE (Step 15). If any node does not converge, we cannot solve the time step for the coupled problem and a time step cut is required (Step 16). If the node converges, then we check a global convergence criterion (Step 18 27). We repeat Steps until we reach the global convergence for a given tree node or exceed the maximum allowed a number of sequential iterations k max for this node. Algorithm 2 explains the logic of switching to the new global status. For each subproblem, we set its local status to be consistent with the current global status (Step 5). At Step 7 we assemble a set of active sub-problems S A and apply corresponding mapping operators. We need to find coupled terms C related to active sub-problems S A. First, we find which coupled terms C g are related to the global status (Step 3). Then for each sub-problem, we add intersection of its coupled terms and C g into set C (Step 9). Note that we carry out this procedure for all sub-problems both active and inactive. For example, an inactive sub-problem might have coupled terms which appear in equations for active sub-problems; failing to add these coupling terms will lead to inconsistent derivatives. Once a set of active sub-problems S A and related coupled terms C are defined, we reorder unknowns using ADETL capabilities and update coupled terms followed by sub-problems (Step 11 13). Algorithm 1 provides a general approach to solve a multi-physics problem using a given coupling strategy. It can be shown that it is identical to available algorithms (Settari and Walters, 2001; Garipov et al., 2016b) when applied for two or three physics problems. Algorithm 3 and Algorithm 4 are algorithms to solve fully coupled Ψ 1 = FIM[FLOW, MECHANICS] and sequentially coupled Ψ 2 = SEQ[FLOW, MECHANICS] strategies for a flow-mechanics problem correspondingly (adapted from (Garipov et al., 2016b)). The coupling strategy Ψ 1 is represented by a sub-problem tree with one terminal node FIM[FLOW, MECHANICS]. It can be seen that Steps 2 10 from Algorithm 1 coincide with Algorithm 3. The coupling strategy Ψ 2 is represented by a sub-problem tree with two terminal nodes (FLOW and MECHANICS) and with the one parent node (SEQ[FLOW, MECHANICS]). Algorithm 5 is Algorithm 1 applied to Ψ 2 with two coupled terms: volumetric stress σ v and pressure p. The right side of Algorithm 5 explains Steps 4 9 which are identical to

68 CHAPTER 2. GENERAL IMPLICIT COUPLING FRAMEWORK (GENIC) 51 Algorithm 2: Global status activation pseudocode. The procedure should be called prior to solving any system of equations to guarantee correct derivatives. S - set of all sub-problems available from sub-problem tree, gstatus - boolean array of active status for each sub-problem in S (a corresponding line from the global status table). 1: procedure ACTIVATE GLOBAL STATUS(gstatus) 2: set S A =, C = S A set of active sub-problems, C set of coupled problems 3: find set of coupled terms C g related to S A 4: for each sub-problem s S do 5: status(s) = gstatus(s) update local sub-problem status 6: if gstatus(s) is active then 7: S A s 8: find set of couple terms C s related to s 9: C C g C s 10: end for 11: restructure Jacobian 12: update coupled terms C 13: update variables related to S A 14: end procedure Algorithm 3: Fully implicit solution strategy for a flow-mechanics problem (adapted from Garipov et al. (2016b)), x n represents all unknowns on the previous time step. 1: F = (x F,x n ),M = (x M,x n ) 2: do 3: calculate residual and Jacobian R (F,M), J (F,M) 4: solve system of linear equations 5: update R F and R M 6: calculate error norm R 7: while ( R > ε) Steps 5 17 from Algorithm 4. The convergence condition of Algorithm 5 (Step 10) is identical to the convergence condition from Algorithm 4 (Step 11 and Step 18) because residual norms are taken at the first Newton iteration prior to local convergence. The comparison of these strategies makes clear that Algorithm 1 is indeed a general algorithm and consistent with algorithms available from the literature for two and three physics problems. Similar observations can be made for flow-thermal (van Odyck et al., 2009; Trangenstein, 1989) or thermal-mechanics (Aboustit et al., 1985; Armero and Simo, 1992) problems.

69 CHAPTER 2. GENERAL IMPLICIT COUPLING FRAMEWORK (GENIC) 52 Algorithm 4: Sequential implicit solution strategy for a flow-mechanics problem (adapted from Garipov et al. (2016b)). 1: F = (x F,x n ),M = (x M,x n ) 2: set sequential iteration k = 0 3: set F k = F and M k = M 4: do 5: update volumetric stress σv k+1 6: do 7: calculate residual and Jacobian R F (F k+1,m k ) J F (F k+1,m k ) 8: solve system of linear equations 9: update F k+1 10: while (R F > ε F ) 11: calculate error norm for mechanics R M (F k+1,m k ) 12: use fixed F k+1 13: do 14: calculate residual and Jacobian R M (F k+1,m k+1 ) J M (F k+1,m k+1 ) 15: solve system of linear equations 16: update M k+1 17: while (R M > ε M ) 18: calculate error norm for flow R F (F k+1,m k+1 ) 19: set k = k : while ( R F > ε F, R F > ε F ) Algorithm 5: Example of application of Algorithm 1 to a specific coupling strategy Ψ = SEQ[FLOW,MECHANICS]. On the left reduced algorithm, on the right some specific details of Steps 4-9. The produced algorithm is identical to Algorithm 4. 1: F = (x F,x n ),M = (x M,x n ) 2: set k = 0 3: do 4: activate global status FLOW 5: solve FLOW sub-problem 6: update variables related to FLOW 7: activate global status MECHANICS 8: solve MECHANICS sub-problem 9: update variables related to MECHANICS 10: if R 1 F > ε F, R 1 M > ε M then 11: return node is converged 12: set k = k : while k < k max 14: return node is NOT converged Steps 4-6 in details: 4.1: F k+1 - active, M k - inactive (fixed) 4.2: S A = F k+1,c = σv k+1 (fixed stress) 5.1: calculate R F (F k+1,m k ), J F (F k+1,m k ) 5.2: solve system of linear equations 6: update F k+1 Steps 7-9 in details: 7.1: M k+1 - active, F k+1 - inactive (fixed) 7.2: S A = M k+1,c = p k+1 (fixed pressure) 8.1: calculate R M (F k+1,m k+1 ) 8.2: calculate J M (F k+1,m k+1 ) 8.3: solve system of linear equations 9: update M k+1

70 Chapter 3 Numerical Applications The objective of this chapter is to demonstrate the power and flexibility of the GENIC framework in modeling complex multi-physics problems. We consider a series of nonlinear problems that involve several mechanisms of flow and transport, and we show that the GENIC framework allows for investigating the full range of possible coupling strategies. The chapter is organized as follows. First, we apply the framework to solve a flow-thermal problem (i.e., multi-component multiphase flow and transport of mass and energy). Next, we explore existing (constant pressure) and novel coupled properties, such as a constant density and a hybrid scheme for flow-thermal sequential coupling strategies. We investigate the convergence properties of three different sequential coupling schemes. Then, we demonstrate the convergence improvement by extending the modified Sequential Fully Implicit (m-sfi) method proposed recently by Moncorgé et al. (2017) for isothermal compositional problems. Following that, we consider coupled flow and mechanics problems and demonstrate how the proper choice of the coupling strategy depends on the coupling strength between the mass and momentum balance equations. Finally, we apply the GENIC framework to a flow-thermal-mechanics problem for which little guidance is available on which coupling strategy to employ. 53

71 CHAPTER 3. NUMERICAL APPLICATIONS Flow and Thermal Coupling Numerical modeling of multiphase flow with thermal effects (flow-thermal) is quite challenging. The mass conservation equations are highly nonlinear and tightly coupled, and the overall energy conservation is also nonlinear with a wide range of coupling strength to the mass balances both in space and in time. There is a limited investigation of sequential methods to model flow-thermal problems (Trangenstein, 1989; van Odyck et al., 2009). The work of van Odyck et al. (2009) is an extension of a compositional solver for non-isothermal multi-component two-phase flow and transport of mass and energy developed in Trangenstein (1989). It was shown that the system of equations can be split into a parabolic pressure equation and a hyperbolic system of conservation laws for the component density and the enthalpy. The authors successfully demonstrated a sequential method, where pressure equation is solved implicitly, followed by an explicitly solved thermal equation. Unfortunately, available studies have two limitations, which are addressed here. First, due to the explicitness of enthalpy, all the numerical tests were conducted for a CFL < 1. This CFL is relatively low in comparison to the CFL size used for problems of practical interest. We couple flow and thermal sub-problems using a sequential method, which does not have such a strict time step limitation. As a result, we were able to reach CFL 100. Second, there is no comparison between FIM and published sequential methods for a compositional-thermal problem in terms of convergence and performance. We compare both methods using the flexibility provided by the GENIC framework. The result of this preliminary study was first published in Wong et al. (2017). For a coupled flow-thermal problem we introduce a vector of unknowns, x = (x F,x T ), where x F are the flow unknowns, and x T are the thermal unknowns. The corresponding residuals are R F (x F,x T ) and R T (x F,x T ). We use a pressure-enthalpy formulation (Wong et al., 2016), and solve the following system of equations for FIM coupling strategy (x n is all unknowns on the previous time step): R n+1 F R n+1 T ( x n+1 T,x n) = 0, ( x n+1 F,xT n+1,x n) = 0. F,x n+1 (3.1) For the implicit sequential coupling strategy SEQ, the flow and thermal equations are

72 CHAPTER 3. NUMERICAL APPLICATIONS 55 solved separately, and coupled through carefully chosen coupled properties. We use coupled properties c F and c T for flow and thermal equations correspondingly. We model sequential algorithm for the flow-thermal problem using Algorithm 1 applied for a problem with two physics modules as following: Step 1. Solve the flow sub-problem: RF n+1 (xn+1 F, x T (δc n+1 F = 0)) = 0 (3.2) This involves solving the flow equation with a variable δc F = 0. We assume that the variable c F is fixed while we solve the flow equation. The choice of variable for c F affects the convergence and numerical stability of the coupled system. Here, we take x F p and c F h the fluid enthalpy of the system. Once we have the solution xf n+1, we use that as the initial guess to the solution of the thermal equation. Step 2. Solve the thermal sub-problem R n+1 T (x F (δc n+1 T = 0), x n+1 T ) = 0 (3.3) This involves solving the thermal equation using δc T = 0. Similar to c F, the variable c T is held constant when solving the thermal equation. We repeat Steps 1 and 2 until global convergence is achieved. We apply three different strategies for the coupled property c T : 1. First scheme is with constant pressure, where we assume that the pressure is constant (c T p) in the thermal residual equation. We can rewrite Eq. 3.3 as follows: R n+1 T (x F (δp n+1 = 0), x n+1 T ) = 0 This is the same as solving the equation with p/ h = Second scheme is with constant density, where we assume each block has a constant total density (c T ρ). We first compute the density ρ(p,h) after Step 1 (Eq. 3.2),

73 CHAPTER 3. NUMERICAL APPLICATIONS 56 and solve Step 2 (Eq. 3.3) as follows: We calculate p h where F(p,h) = p ρ(p,h). R n+1 T (x F (δρ(p,h) n+1 = 0), x n+1 T ) = 0 using the inverse function: p h (δρ = 0) = F h ( F ) 1 p 3. Third solution strategy is a combination of constant-pressure and constant-density formulations. We refer to this formulation as the hybrid formulation. This mixed formulation depends on the phase state of the block. For this formulation, we chose a constant-pressure coupled property for the single-phase blocks and constantdensity for two-phase blocks. We define the phase-state of the block by comparing the enthalpy h with the saturated enthalpies of the block h w (p) and h s (p). More formally, we define the coupled property as: ρ, h w (p) h h s (p) c T p, otherwise Example 1. Avdonin s Problem We consider a geothermal problem (Avdonin, 1964; Wong, 2015) for which an analytical solution is available. This test case involves a one-dimensional linear steady-state flow and unsteady heat transport of single-phase liquid water. Cold water at T = 423 K is injected with a flow rate of q = 10 m 3 /day into a reservoir with water at T = 473 K. The boundary condition imposed is a pressure of Pa at an outer radius of 1000 m (see Figure 3.1). The model properties are shown in Table 3.1. The analytical solution is given by Avdonin (1964): u(χ,τ) = χ (πτ) 1/2 1 0 exp [ ( ) ] sγ(τ) 1/2 χ 2 ds 2s(τ) 1/2 s 2, (3.4)

74 CHAPTER 3. NUMERICAL APPLICATIONS 57 Figure 3.1: Schematic representation of Avdonin s problem (uniform linear spacing). where χ = 2x /b, τ = 4K mt/c m ρ m b 2, γ = Qc mρ w /4K m, Q is the injection rate, and b is the reservoir thickness. Figure 3.2 shows the temperature distribution at t = 1000 days and the temperature evolution at x = 25 m, using fully implicit Ψ 1 = FIM[FLOW, THERMAL], sequentially implicit SEQ Ψ 2 = SEQ[FLOW, THERMAL] coupling strategies, and an analytical solution. We use constant pressure c T p as a coupled property for the sequential approach. The resulting solutions match within the desired tolerance for all applied coupling strategies. The sequential approach normally requires 3 7 iterations for a tolerance ε = 10 6 to accurately capture the analytical solution (Figure 3.3). We note that the error decreases monotonically with iteration Example 2. Injection Flash Front This numerical model is a one-dimensional, radial model. Cold water at T = 355 K is injected at a constant pressure p in j = 90 bars into a reservoir with water at T = K (Wong et al., 2017). We consider two scenarios: single-phase and two-phase. The singlephase case involves injection in the radial model, where the fluid remains as a single-phase liquid water with a fixed outer boundary reservoir pressure at p res = 45 bars. The two-phase Table 3.1: Parameters for Avdonin s problem. Reservoir length, m 1000 Reservoir thickness, m 100 Rock permeability, md 1000 Rock density, kg/m Rock specific heat capacity, J/(g K) 1.0 Porosity 0.2 Spatial grid with uniform 20 m grid spacing

75 CHAPTER 3. NUMERICAL APPLICATIONS 58 Figure 3.2: Avdonin s problem. Comparison of the analytical and numerical solutions of the temperature distribution at t = 1000 days (left), and the temperature evolution at location x = 25 (right). Figure 3.3: Avdonin s problem. Sequentially implicit method convergence profile after the first time step t = days. case involves the propagation of the injected cold-water as a sharp front with an initial water saturation S w = 0.1 and a fixed outer boundary reservoir pressure at p res = 10 bars (see Figure 3.4). The model properties are shown in Table 3.2. We compare the fully implicit Ψ 1 = FIM[FLOW, THERMAL], and the sequential implicit SEQ Ψ 2 = SEQ[FLOW, THERMAL] coupling strategies. We study three different coupled properties for Ψ 2 : constant pressure,

76 CHAPTER 3. NUMERICAL APPLICATIONS 59 Figure 3.4: Schematic representation of Injection Flash Front problem (uniform radial spacing). Table 3.2: Parameters for Injection Flash Front problem. Reservoir length, m 1000 Rock permeability, md 500 Rock density, kg/m Rock thermal conductivity, kj/(m day K) 1728 Porosity 0.2 Number of radial blocks 10, 50, 150 Spatial grid (radial) uniform grid spacing constant density, and hybrid schemes. We compare the time step size in terms of the maximum CFL number, which is defined as: CFL max = tq V φ (3.5) where t is the maximum time step, Q is the injection rate, V φ is pore volume of the block. Single-Phase scenario. Figure 3.5 shows the pressure and temperature distribution after t = 100 days. The injection front is located around x = 200 m. The temperature front is smeared due to the difference in dynamics between flow and thermal effects, and also due to numerical dispersion. All of the tested coupled properties yield very similar results. There is a small difference due to varying time step for the constant density case, which is not stable for the single-phase scenario. Figure 3.6 shows the pressure and temperature profiles at x = 167 m. The pressure reaches its maximum value (around 80 bars) when the injection front reaches x = 167 m (around 10 days) and subsides after that. The pressure increases slightly after 90 days, and this is caused by a decrease in temperature. We conduct a refinement study using 10, 50, and 150 grid blocks. Figure 3.7 and Figure 3.8 show convergence to the true solution. Table 3.3 summarizes a comparison between sequential coupling schemes with three different coupled properties and the fully coupled approach. In the single-phase case, the

77 CHAPTER 3. NUMERICAL APPLICATIONS 60 Figure 3.5: Single-phase injection flash front. Pressure and temperature distribution after 100 days (50 grid cells). Figure 3.6: Single-phase injection flash front. Pressure and temperature profile at x = 167 m (50 grid cells). constant pressure and hybrid schemes achieve nearly identical nonlinear performance. This is because the entire domain remains in single-phase water through the entire simulated period; that makes pressure a good choice as a coupled property for all the grid blocks. We can reach maximum time step for constant pressure, hybrid and fully implicit method without time step cuts. The constant pressure takes 4-5 sequential iterations to decrease the residual to ε = For the constant density case, the maximum time step was about

78 CHAPTER 3. NUMERICAL APPLICATIONS 61 Figure 3.7: Grid refinement for single-phase radial one-dimensional injection scenario. Pressure and temperature distribution after 100 days. Figure 3.8: Grid refinement for single-phase radial one-dimensional injection scenario. Pressure and temperature profile at x = 167 m. an order of magnitude less than the other three schemes; moreover, the constant-density formulation suffers from a large number of wasted iterations (i.e. makes a lot of time step cuts). Two-Phase scenario. Figure 3.9 shows the pressure and temperature distribution after t = 100 days. The injection front is located around x = 350 m. For this scenario, the constant pressure does not converge regardless of the chosen time step.

79 CHAPTER 3. NUMERICAL APPLICATIONS 62 Table 3.3: Single-phase injection flash front. Comparison between sequential coupling schemes, such as Constant Pressure (CP), Constant Density (CD), Hybrid (H) and fully coupled method (FIM). Number of blocks CP CD H FIM CP CD H FIM CP CD H FIM number of timesteps total Newton iterations total seq iterations timesteps wasted wasted Newton iterations maximum CFL Figure 3.9: Two-phase radial one-dimensional injection scenario. Pressure and temperature distribution after 100 days (50 grid cells). Other tested coupled properties produce results that are quite similar to the fully implicit approach. Figure 3.10 shows the pressure and temperature profile at x = 167 m. Here we observe numerous fluctuations in the pressure profile; such behavior poses significant challenges for the sequential scheme. We conduct a refinement study using 10, 50, and 150 grid blocks. Figure 3.11 and Figure 3.12 show convergence to the true solution. Table 3.4 summarizes a comparison between sequential coupling schemes using two coupled properties with the fully coupled approach. In all the cases studied here, the hybrid scheme outperforms the constant-density formulation allowing for larger time steps. Moreover, the hybrid scheme requires less sequential iterations (9-10 in average versus for constant density) for a given time step. Overall, the hybrid scheme scales much better than the constant density scheme. Nevertheless, the hybrid scheme is not efficient enough to

80 CHAPTER 3. NUMERICAL APPLICATIONS 63 Figure 3.10: Two-phase radial one-dimensional injection scenario. Pressure and temperature profile at x = 167 m (50 grid cells). Figure 3.11: Grid refinement for two-phase radial one-dimensional injection scenario. Pressure and temperature distribution after 100 days. replace the fully implicit approach. While the hybrid scheme produces less wasted time steps for 10 grid blocks, it scales non-linearly with grid size (36 wasted time steps for 10 blocks and 3,121 for 150 blocks), while FIM shows excellent scalability.

81 CHAPTER 3. NUMERICAL APPLICATIONS 64 Figure 3.12: Grid refinement for two-phase radial one-dimensional injection scenario. Pressure and temperature profile at x = 167 m. Table 3.4: Two-phase injection flash front. Comparison between sequential coupling schemes, such as Constant Density (CD), Hybrid (H) and fully coupled method (FIM). Note: Constant Pressure does not converge for two-phase case. Number of blocks CD H FIM CD H FIM CD H FIM number of timesteps total Newton iterations total seq iterations timesteps wasted maximum CFL Summary We demonstrated the applicability of the GENIC framework for flow-thermal problems. First, we validated our implementation for Avdonin s problem which has an analytical solution. Next, we explored existing (constant pressure) and novel coupled properties, such as a constant density and a hybrid scheme for flow-thermal sequential coupling strategies. We investigated the convergence properties for three different sequential coupling schemes. We found that a hybrid method where the coupled property was pressure for single-phase blocks and density for two-phase blocks performed the best. This hybrid method was able

82 CHAPTER 3. NUMERICAL APPLICATIONS 65 to avoid convergence problems in the single-phase (constant density) or two-phase (constant pressure) regions. More numerical tests also showed that the fully implicit method performed significantly better than any sequential scheme we tried. This preliminary study might be extended with mathematical analysis of the different coupling schemes for flowthermal problems. Further mathematical analysis could provide insight as to how to design better nonlinear solvers for the fully implicit solution and test them with the GENIC framework. As we reported in Wong et al. (2017), existing sequential schemes for flow-thermal problems have convergence issues for more complex scenarios. For example, for a twodimensional heterogeneous reservoir, the hybrid approach is times slower than fully implicit approach. We developed modified SFI for flow and thermal physics to overcome this challenge and will present its results in the next section.

83 CHAPTER 3. NUMERICAL APPLICATIONS Modified SFI for Flow-Thermal Problems We demonstrate the use of the GENIC framework for studying the robustness and efficiency of different coupling strategies for thermal-compositional simulation. We focus on the modified SFI (Moncorgé et al., 2017), which was designed as a stable splitting approach for the flow problem (i.e., coupled multi-component multiphase flow with mass transfer). Here, we applied a variant of the m-sfi approach for flow and thermal physics with promising results. In the GENIC framework, the m-sfi approach is assembled by using the mapping operator introduced in Section We solve the flow-thermal problem at hand with the following coupling strategy: Ψ = SEQ[FIM[FLOW, THERMAL<M1>], THERMAL<M2>]. We define M1 as a region with strong coupling effects, which satisfies one of the following conditions for each grid block: two or three phase region, connected to a well perforation, CFL > CFL tol. Similarly, we define M2 as a domain with weak flow-thermal coupling such that it meets one of the following criteria: single-phase, CFL < CFL tol. For m-sfi we use constant pressure as a coupled property given its efficiency for a single phase region. The fully coupled system can be written as R n+1 F R n+1 T ( x n+1 and in linearized matrix form, we can write [ R F/ x F R T/ x F T,x n) = 0, ( x n+1 F,xT n+1,x n) = 0. F,x n+1 R F/ x T R T/ x T ][ ] δxf = δx T [ RF R T ] (3.6). (3.7) To solve the problem, we split the thermal equation R T into two equations R T1 and R T2 written for domains M1 and M2 correspondingly with unknowns x T1 and x T2. Then, we solve the flow-thermal problem in two steps: Step 1: Solve full flow system with a strongly coupled thermal domain M1 and fixed

84 CHAPTER 3. NUMERICAL APPLICATIONS 67 unknowns x T2. [ R F/ x F R T1 / x F R F/ x T1 R T1 / x T1 ][ ] δxf = δx T1 [ RF R T1 ] (3.8) Step 2: Solve the thermal equation for the weakly coupled domain M2 with fixed x F : [ R T2 / x T2 ][δx T2 ] ] = [R T2. (3.9) We repeat Step 1 and Step 2 until we reach convergence for R F, R T1, and R T Example 3. 3D m-sfi Case A three-dimensional problem based on a reservoir model from Stanford Geothermal Program (1980) is used here. The reservoir model consists of a single-phase liquid water with a two-phase zone of immobile steam sandwiched between hot and cold water regions. Production is from a well that is completed below the two-phase zone. The parameters were chosen such that boiling in the well occurs after a certain period of production. Although the parameters are relatively homogeneous, it was used as a prototype for field-wide reservoir development studies. Figure 3.13 depicts a schematic representation of the layer configuration. This problem has both three-dimensional flow with phase transitions and two-phase flow including gravity drainage. The production well was completed in the corner block and perforates layers The water rate at 500 m 3 /day was used as the well control. The simulation time was 10 years. A full discussion on the problem can be found in Stanford Geothermal Program (1980). The model parameters are summarized in Table 3.5. We compare several coupling strategies. First, we apply the fully implicit coupling strategy Ψ 1 = FIM[FLOW, THERMAL]. The second approach is the sequential coupling strategy Ψ 2 = SEQ[FLOW, THERMAL] with the hybrid coupling property discussed previously (Section 3.1). Hybrid coupling approach is effective for single and two-phase regions in 1D or 2D cases; however, coupling strategy Ψ 2 has limited stability and poor performance for heterogeneous two or three-dimensional cases. Third, we apply m-sfi approach with a coupling strategy Ψ 3 = SEQ[FIM[ FLOW, THERMAL<M1>], THERMAL<M2>] over a range of CFL tol parameters (0.1, 1, 10). Fourth, we change M1 domain in m-sfi approach and, in

85 CHAPTER 3. NUMERICAL APPLICATIONS 68 Table 3.5: Parameters for 3D model for m-sfi study. Reservoir dimensions, m Reference porosity in each layer [0.2, 0.25, 0.25, 0.25, 0.2, 0.2] Layer permeability in x and y directions, md [100, 200, 200, 200, 100, 100] Layer permeability in z direction [2, 50, 50, 50, 2, 2] Layer gas saturation [0, 0.15, 0, 0, 0, 0] Depth at layer top, m [150, 450, 750, 1050, 1350, 1650] Temperature at layer top, K [433.15, ,..., ] Pressure at layer top, bar [40, 64, 88, 112, 136, 160] Figure 3.13: Schematic of the three-dimensional model. addition to CFL condition equals to 1.0, we add one layer of neighboring cells for strongly coupled domain M1 (referred as MSFI-N). We use a direct linear solver PARDISO for numerical simulations. Figure 3.14 shows the pressure and temperature profiles for a grid at production well block. Figure 3.15 represents the pressure and temperature distributions at 16th layer after 10 years of production. As a consequence of production, pressure drops in the wellblock, so that horizontal and vertical flow towards the wellblock is initiated. Downward flow from the two-phase zone gives rise to boiling and an increase in the vapor saturation. As the pressure decline spreads to the margins of the field, water recharge is initiated.

86 CHAPTER 3. NUMERICAL APPLICATIONS 69 Figure 3.14: 3D m-sfi case. Pressure and temperature profile for grid at production well block. One consequence of this is the occurrence of several phase transitions to single-phase conditions in the two-phase layer. Thus, large pressure drops occur in the wellblock, as well as in the adjacent grid blocks, causing several phase transitions to two-phase conditions and subsequent boiling. Transition period is accompanied by a decline in temperatures and pressures, as well as a buildup of vapor saturation. Steam-water counterflow occurs as steam rises from the shallow two-phase layer, whereas water flows downward towards the production well. Figure 3.16 depicts relative error map between FIM and MSFI-N approach where the difference is less than 0.01%. Table 3.6 summarizes a comparison between the sequential coupling scheme with the hybrid coupled property, m-sfi, MSFI-N, and the fully coupled approach for blocks. The SEQ scheme, even with hybrid splitting, is unstable and is not a suitable candidate to solve this problem. We omit the results for the SEQ scheme for other grid resolutions where we observe similar poor performance. All the presented m-sfi schemes show better convergence than FIM, have fewer time steps that fail to converge (4 for FIM vs 1 for m-sfi), and converge to the desired tolerance (as shown on Figure 3.16). We observe that tighter CFL tolerance criterion decreases the number of Newton iterations (from 324 for CFL tol = 10 to 306 for CFL tol = 0.1). However, the convergence improvement is relatively small due to an inability of CFL criterion alone to accurately predict for two-phase region. MSFI-N approach employs more aggressive strategy with additional neighboring

87 CHAPTER 3. NUMERICAL APPLICATIONS 70 (a) (b) Figure 3.15: 3D m-sfi case. Pressure (a) and temperature (b) distribution for grid after 10 years of production at 16th layer. (a) (b) Figure 3.16: 3D m-sfi case. Relative error between FIM and MSFI-N approach for pressure (a) and temperature (b) for grid after 10 years at 16th layer. cells. As a result, two-phase region is predicted more accurately, and we observe the drop in the number of Newton iterations (from 306 for m-sfi to 213 for MSFI-N) with the slight increase in the coverage of M1 domain. For this grid resolution, m-sfi schemes simulation takes 3% more CPU time than FIM. Figure 3.17 demonstrates the convergence of the pressure and temperature with grid refinement. The accuracy of the solution increases for a more refined grid. This is due to

88 CHAPTER 3. NUMERICAL APPLICATIONS 71 Table 3.6: m-sfi 3D case for blocks. Comparison between different coupling schemes: FIM, SEQ, m-sfi and MSFI-N. m-sfi FIM SEQ CFL tol = 0.1 CFL tol = 1 CFL tol = 10 MSFI-N number of timesteps total Newton iterations total sequential iterations number of timesteps wasted % of blocks in M maximum CFL the complex nature of steam-water propagation in the system. Table 3.7 summarizes the performance comparison between the applied coupling strategies for a low grid resolution (4 5 6) and a finer grid resolution ( ). In our example, a low grid resolution does not allow us to capture all the complex physics; as a result, all m-sfi coupling strategies perform relatively well compared with FIM. Once we increase our grid resolution, more phase changes occur, and m-sfi strategies based on a CFL criterion struggle to converge when a phase change occurs, leading to numerous time step cuts. MSFI-N does not have this issue due to an additional layer of neighboring cells. As a result, the MSFI-N approach is stable, and it performs as well as FIM on a finer grid. Moreover, the MSFI- N simulation is about 10% faster than FIM. Unfortunately, the GENIC framework linear solver capabilities are not fully developed, and more advanced linear solvers are needed to conclude if MSFI-N can outperform FIM on complex large-scale problems. Table 3.7: m-sfi 3D case for low grid resolution (L) and finer grid resolution (H). Comparison between different coupling schemes: FIM, SEQ, m-sfi and MSFI-N. m-sfi FIM CFL tol = 0.1 CFL tol = 1 CFL tol = 10 MSFI-N L H L H L H L H L H number of timesteps total Newton iterations total sequential iterations number of timesteps wasted % of blocks in M maximum CFL

89 CHAPTER 3. NUMERICAL APPLICATIONS 72 Figure 3.17: Grid refinement for 3D m-sfi case. Pressure and temperature profile at production well block Example 4. m-sfi SPE 10 Our second example for investigating m-sfi based coupling strategies is the SPE 10 test problem (Christie and Blunt, 2001). The original data set, including relative permeability curves, porosity and permeability distributions, and fluid properties, was used in this example. The simulation domain Ω = m is discretized using a regular Cartesian grid into cells. The top 35 layers represent a formation with a permeability field that has the Gaussian distribution, and the bottom 50 layers are characterized by a channelized permeability field with high-contrast features. We limit our domain by using only the top four layers from the original SPE 10 problem. The initial reservoir pressure is 50 bar filled with steam at temperature T = K. We have two water injectors on the left and right side of the field, injecting at T = K and p = 90 bar, and two producers at the inner part of the reservoir at p = 30 bar. All wells operate under BHP constraint. Figure 3.18 represents porosity, permeability and well configuration. The model parameters are summarized in Table 3.8. We performed three levels of grid refinement: fine scale ( ), coarse scale ( ), and very coarse scale ( ). The upscaling of the properties was performed using a simple volume-averaging approach. This leads to a slightly different solution between the different grid resolutions. The resulting problem is challenging even

90 CHAPTER 3. NUMERICAL APPLICATIONS 73 Figure 3.18: Schematic of the SPE 10 setup: permeability (top) and porosity (bottom) maps. Table 3.8: Parameters for SPE 10 model for m-sfi study. Reservoir dimensions, m Initial reservoir pressure, bar 50 Initial reservoir temperature, K BHP, bar 30 for producers, 90 for injector Injection temperature, K Relative phase permeability k r j = S 2 j Rock thermal expansion, 1/K Rock heat capacity, kj/(kg K) Rock thermal conductivity, kj/(m day K) Oil thermal conductivity, kj/(m day K) 11.5 Gas thermal conductivity, kj/(m day K) Water thermal conductivity, kj/(m day K) 53.5 for the FIM method, and it struggles to converge with more aggressive time step and well constraints. We employ similar coupling strategies to those used in Example 3. First, we apply the fully implicit coupling strategy Ψ 1 = FIM[FLOW, THERMAL]. The second approach is the sequential coupling strategy Ψ 2 = SEQ[FLOW, THERMAL] with a hybrid coupling property

91 CHAPTER 3. NUMERICAL APPLICATIONS 74 discussed previously (Section 3.1). Here, hybrid coupling approach proved to be unstable, and failed to converge. Third, we apply m-sfi approach with a coupling strategy Ψ 3 = SEQ[FIM[FLOW, THERMAL<M1>], THERMAL<M2>] with CFL tol equal to 0.1. We observed that larger CFL parameters lead to poor convergence and instability. Fourth, we employ MSFI-N approach combined with CFL criteria, where we took CFL tol equal to 0.1 and 1. Table 3.9 summarizes comparison between discussed coupling strategies for resolution blocks. Here, we observe that m-sfi with CFL tol = 0.1 outperforms MSFI-N with CFL tol = 1, and takes half the time of the last approach. This observation is different from Example 3, where MSFI-N gave the best performance even when small CFL criterion was used for m-sfi method. In our SPE 10 example, we do not only have a complicated phase-behavior of steam and liquid of the previous example, but also injection and production fronts. Therefore, MSFI-N cannot accurately predict the single-phase region which leads to the poor stability and convergence we observe for the sequential flow-thermal method in two-phase region. In this scenario, smaller CFL tolerance allows for better control over single-phase region change. Nevertheless, we can further improve convergence by combining tight CFL tolerance and neighboring cells: MSFI-N with CFL tol = 0.1 takes 2.6 times less run time. Unfortunately, all presented m-sfi methods perform slower than FIM. There are several reasons why. First, inaccurate prediction of single-phase region. As we have seen CFL condition cannot accurately predict phase change. Second, a significantly larger M1 domain which increased from 25% in the previous case to 70% in SPE 10. This alone makes every sequential iteration almost as expensive as FIM approach. Figure 3.19 shows pressure, temperature, and water saturation for grid at central block of top layer. All coupling strategies are in good agreement. The small temperature variation caused by mixing cold injected water with water from reservoir region; Table 3.9: m-sfi SPE 10 upscaled case with top four layers for grid. Comparison between different coupling schemes: FIM, m-sfi and MSFI-N. FIM m-sfi (CFL tol = 0.1) MSFI-N (CFL tol = 1) MSFI-N (CFL tol = 0.1) number of timesteps total Newton iterations total sequential iterations number of timesteps wasted % of blocks in M normalized run time maximum CFL

92 CHAPTER 3. NUMERICAL APPLICATIONS 75 Figure 3.19: SPE 10 m-sfi case. Pressure, temperature, and water saturation profile for grid at central block of top layer. Figure 3.20: SPE 10 m-sfi case. Pressure (left) and temperature (right) distribution for grid after 100 days at top layer. corresponding water saturation shows two-phase region at this block. Figure 3.20 represents pressure and temperature distribution for the same grid at the top layer after 100 days. Here, the right part of the reservoir has a higher permeability and leads to a larger injection region. Meanwhile, pressure and temperature map are not identical since steam rises from the bottom layers to top. Figure 3.21 depicts relative error between FIM and m-sfi which is within 10 3 % for pressure and less than 0.5% for temperature. Notice, errors are larger on the border of two-phase and single-phase regions. Table 3.10 summarizes performance comparison between applied coupling strategies

93 CHAPTER 3. NUMERICAL APPLICATIONS 76 Figure 3.21: SPE 10 m-sfi case. Relative error between FIM and MSFI-N approach for pressure (left) and temperature (right) distribution for grid after 100 days at top layer. Table 3.10: m-sfi SPE 10 case for very coarse grid resolution (L) and original grid resolution (top four layers) (H). Comparison between different coupling schemes: FIM, m-sfi and MSFI-N. Normalized run time is normalized towards simulation time of FIM approach for each grid resolution. FIM m-sfi (CFL tol = 0.1) MSFI-N (CFL tol = 1) MSFI-N (CFL tol = 0.1) L H L H L H L H number of timesteps total Newton iterations total sequential iterations number of timesteps wasted % of blocks in M normalized run time maximum CFL for fine ( ) and very coarse grid ( ) resolutions. In our example, a low grid resolution does not allow us to capture all complex physics. As a result, all m- SFI coupling strategies perform relatively well compared to FIM. Once we increase our grid resolution, we observe a similar trend as with a coarse grid resolution ( ) in Table 3.9. MSFI-N with CFL tol = 0.1 slightly outperforms m-sfi with CFL tol = 0.1, where MSFI-N with CFL tol = 1 performs much worse than on two coarser examples. For presented example, m-sfi approaches are able to converge to the desired tolerance with good convergence properties. However, given large percentage in M1 domain (over 60%), m-sfi approach does not outperform FIM approach.

94 CHAPTER 3. NUMERICAL APPLICATIONS Summary We presented a new approach for coupling flow and thermal physics. We demonstrated m-sfi method on several challenging examples, where standard sequential approach fails, while m-sfi takes a similar number of time steps as FIM approach. In cases where accurate front prediction is possible and the two-phase region is limited, such as strictly injection or production problems, m-sfi approach outperforms FIM. However, when the two-phase process appears in most of the reservoir, the overall performance of the m-sfi method is more expensive than FIM due to the increased cost of sequential iterations. The performance of m-sfi can be improved by considering another coupled property for flow-thermal which does not suffer in two-phase region. Additionally, m-sfi can be used as a preconditioner for challenging flow-thermal problems.

95 CHAPTER 3. NUMERICAL APPLICATIONS Flow and Mechanics Coupling Flow and mechanics coupling has been extensively studied in the last two decades (Prevost, 1997; Settari and Walters, 2001; Tran et al., 2004; Wheeler and Gai, 2007; Gai, 2004). To solve the coupled flow and geomechanics problems, four different operator-splitting strategies have been proposed, namely, fixed-stress, fixed-strain, drained, and undrained splits. Figure 3.22 is a schematic representation of the operator-splitting strategies. The details of these sequential solution schemes and numerical analysis of their stability and convergence properties can be found in Kim (2010). Kim et al. (2011) and Mikelic and Wheeler (2013) showed that both fixed-stress and undrained split are unconditionally stable, and that fixed-stress converges faster than undrained split. In this section, we demonstrate the applicability of the GENIC framework for flowmechanics problems. For a coupled flow-mechanics problem we introduce a vector of unknowns, x = (x F,x M ), where x F are the flow unknowns, and x M are the mechanics unknowns. The corresponding residuals are R F (x F,x M ) and R M (x F,x M ). We solve following system of equations for FIM coupling strategy (x n is all unknowns on the previous time step): R n+1 F R n+1 M ( x n+1 M,xn) = 0, ( x n+1 F,xM n+1,xn) = 0. F,x n+1 (3.10) For the implicit sequential coupling strategy SEQ, the flow and mechanics equations are Figure 3.22: Schematic representation of different operator-splitting strategies for flow-mechanics problems. (adapted from Kim et al. (2011).

96 CHAPTER 3. NUMERICAL APPLICATIONS 79 solved separately, and coupled through carefully chosen coupled properties. We use coupled properties c F and c M for flow and mechanics equations correspondingly. We model sequential algorithm for the flow-mechanics problem using Algorithm 1 applied for a problem with two physics modules as following: Step 1. Solve flow sub-problem: RF n+1 (xn+1 F, x M (δc n+1 F = 0)) = 0 (3.11) This involves solving the flow equation with a variable δc F = 0. We assume that the variable c F is fixed while we solve the flow equation. Here, we take c F ε v (fixed-strain) or c F σ v (fixed-stress). Once we have the solution xf n+1, we use that as the initial guess to the solution of the mechanics equation. Step 2. Solve mechanics sub-problem RM n+1 (x F(δc n+1 M = 0), xn+1 M ) = 0 (3.12) This involves solving the mechanics equation with a variable δc M = 0. Similar to the c F, the variable c M is constant when solving the mechanics equation. For both the fixed-stress and fixed-strain coupled properties, c F p. We repeat Step 1 and Step 2 until the global convergence is achieved. Remark. Kim (2010) defines operator-splitting strategy as solving two subproblems (flow and mechanics) sequentially. In this work, a coupling strategy defines a coupling order of sub-problems (Eq. 2.22), while the exact coupling operator is defined through a coupled property (see section 2.3.4). Both fixed-stress and fixed-strain operator-splitting can be represented by the same coupling strategy Ψ = SEQ[FLOW, MECHANICS] with two different coupled properties: volumetric strain or stress. Here, fixed-stress method is a short notation for a sequential coupling strategy Ψ with stress as a coupled property. We define fixed-strain method similarly.

97 CHAPTER 3. NUMERICAL APPLICATIONS Example 5. Mandel Problem We consider the two-dimensional Mandel consolidation problem (Mandel, 1953). Mandel s problem consists of a homogeneous and isotropic poroelastic slab sandwiched between two rigid, frictionless, impermeable plates. Both plates are loaded by a compressive vertical force as shown in Figure The slab is infinitely long in the z-direction, with a cross-section measuring 2a 2b. The characteristic length to be used in the estimate of the characteristic time is a. The side surfaces (x = ±a) are stress-free, drained, and are kept at ambient pressure. The force has a magnitude 2F (units of force per unit length in the z-direction) and is suddenly applied [ at t = ] 0, producing both an instantaneous overpressure T p 0 and linear displacement u 0 = u x,0,u y,0. We consider a computational domain Ω = (0,a) (0,b) = (0,100m) (0,100m), the geometry represents a quarter of the original Mandel s problem (see Figure 3.23). The domain is fully saturated with compressible single-phase fluid, and the initial fluid pressure is p Ω = Pa. We impose no-flow boundary conditions everywhere, except on the right boundary where a constant pressure is enforced, p x=a = 0. We set zero horizontal displacement on the left boundary (u x x=0 = 0), zero vertical displacement on the bottom boundary (u z z=0 = 2F y y (a,b) 2b 2a x 2F (0,0) x Figure 3.23: Sketch of the setup for Mandel s problem: (left) physical problem, and (right) numerical discretization. Taking advantage of symmetry of geometry and boundary conditions only the gray region corresponding to one-quarter of the domain is discretized. Figure is taken from Castelletto et al. (2015).

98 CHAPTER 3. NUMERICAL APPLICATIONS 81 Table 3.11: Rock and fluid properties used for Mandel s problem. Reference porosity Young s modulus, Pa 10 9 Biot s constant 1.0 Poisson ratio 0.25 Undrained Poisson ratio 0.47 Permeability, md 1 Fluid viscosity, Pa s Fluid compressibility, Pa Reference fluid density, kg/m ), and uniform loading conditions on the top boundary (F z=b = Pa). The vertical displacement on the upper boundary is constrained by the condition, u z x =constant. The undrained Poisson ratio is used to initialize the initial displacement field, and the drained Poisson ratio is used for the transient simulation. The model properties are shown in Table The initial conditions are as follows (Abousleiman et al., 1996): p 0 (x,0) = 1 3a B(1 + ν u)f x Ω (3.13a) u x,0 (x,0) = Fν u 2G x a u y,0 (x,0) = F (1 ν u) 2G y a x Ω x Ω (3.13b) (3.13c) with B = bm/ ( K + b 2 M ) Skempton s coefficient, ν u = [3ν + bb(1 2ν)]/[3 bb(1 2ν)] the undrained Poisson ratio. The analytical solutions for p, u x, and u y read (Abousleiman

99 CHAPTER 3. NUMERICAL APPLICATIONS 82 et al., 1996): p(x,y,t) =2p 0 n=1 [ u x (x,y,t) = u y (x,y,t) = Fν 2Ga Fν u Ga + F G [ sinα ( n cos α ) ( nx α n sinα n cosα n a cosα n exp α2 nc f t a 2 ( ) ] α2 nc f t x+ n=1 F (1 ν) 2Ga n=1 sinα n cosα n α n sinα n cosα n exp cosα n sin α nx α n sinα n cosα n a exp + F (1 ν u) Ga n=1 a 2 ( α2 nc f t a 2 ) sinα n cosα n α n sinα n cosα n exp ) (3.14a) (3.14b) ( ) ] α2 nc f t a 2 y (3.14c) with α n the positive roots that satisfy the characteristic equation: tan α n = 1 ν ν u ν α n. We employ two coupled properties for sequential coupling strategy Ψ = SEQ[FLOW, MECHANICS]: fixed-stress and fixed-strain. Table 3.12 summarizes a comparison between sequential coupling schemes with two different coupled properties and the fully coupled approach. Fixed-strain method is conditionally stable and can solve Mandel s problem only with tiny time steps and loose tolerance (10 3 ); it diverges for a tighter tolerance. Both FIM and fixed-stress are unconditionally stable and converge to the desired tolerance. Figure 3.24 shows the displacement and pressure evolution. Fixed-stress, FIM, and analytical solution are identical, while fixed-strain converges to a different solution. Both the FIM and fixed-stress approaches capture the transient solution behavior accurately, and the sequential approach requires 3 20 iterations with tolerance ε = 10 6 (Figure 3.25) which is consistent with published results (Castelletto et al., 2015). The fixed-strain method is incapable of resolving the solution to the desired tolerance, this is consistent with the published results (Kim, 2010).

100 CHAPTER 3. NUMERICAL APPLICATIONS 83 Table 3.12: Mandel s problem. Convergence between fixed-strain and fixed-stress methods, and FIM. Note, fixed-strain method is conditionally stable and can solve Mandel s problem only with tiny time steps and loose tolerance (10 3 ). fixed-strain fixed-stress FIM tolerance number of timesteps 2, total Newton iterations 26, total seq iterations 14, timesteps wasted 1, maximum CFL Figure 3.24: Mandel s problem. Comparison of the analytical and the numerical solutions for displacement at location (x = 0, z = 100) (left), and pressure at location (x = 2.5, z = 0) (right). Fixed-strain method converges to a different solution Example 6. Depletion of a Box-Shaped Reservoir (Deans Problem) Next, we demonstrate the simulation of a test problem presented by Dean et al. (2006). This example demonstrates the applicability of GENIC to solve a problem where different physics modules are applied in different spatial domains. Here, the reservoir, where both flow and mechanics interact, is fully enclosed within a larger domain that includes the over- and under-burden formations. Mechanical deformation is accounted for in the large domain. The simulation region is a box Ω = m m m and

101 CHAPTER 3. NUMERICAL APPLICATIONS 84 Figure 3.25: Mandel s problem. Sequentially implicit method convergence profile after first time step t = days. Fixed-strain method cannot decrease error less than 10 3, while fixed-stress can reduce it to the desired tolerance. contains three layers: top layer (0 m 3048 m), middle layer (3048 m m), and bottom layer ( m m). The pay zone of the reservoir, Ω r = m m 76.2 m, is located in the middle layer and has the same thickness as the middle layer and is surrounded by a stiff non-pay impermeable medium. Figure 3.26 shows geometrical configuration of the model, and Table 3.13 provides the rock properties. The pay and non-pay zones are fully saturated with a single-phase liquid which has a density of 1000 kg/m 3 at 10 5 Pa, the compressibility of Pa 1, and the viscosity of 10 3 Pa s. The initial fluid pressure is 10 5 Pa at the surface with the hydrostatic gradient of Pa/m; the initial vertical stress is 0 Pa at the surface with the vertical stress gradient of Pa/m, and initial horizontal stresses are equal to the Table 3.13: Rock properties for Dean s model. Property Reservoir Nonpay zones Porosity, % Horizontal permeability, md Vertical permeability, md 10 0 Young s modulus, Pa Biot s constant 1 1 Poisson ratio Rock density, kg/m

102 CHAPTER 3. NUMERICAL APPLICATIONS m 3 /d Well y z x (x,y) plane x (x,z) plane Figure 3.26: Dean s model geometry. half of the vertical stress. The bottom and side boundaries have the zero normal displacement constraint. The wellbore is located at the center of the domain, has the radius of m, and fully penetrates the reservoir. The well operates at a constant production rate of m 3 /day for 4000 days. We employ a fixed-stress coupled property for sequential implicit coupling strategy. We plot the pressure behavior at the top boundary of the reservoir (top right corner of the dark box in Figure 3.26), and subsidence at the top of the reservoir and at the surface. The simulation results have been compared against the published solution (Dean et al., 2006), and show a good match (see Figure 3.27). Due to the Mandel-Crayer effect, the pressure increases in the early period, reaches a maximum value, and then decreases monotonically as shown in Figure 3.27a. Our numerical implementation reproduces Dean s results both for the fully-implicit and the fixed-stress sequentially implicit strategies. A small discrepancy (less than 0.1%) appears due to presumable differences in the initialization procedure.

103 CHAPTER 3. NUMERICAL APPLICATIONS 86 (a) (b) Figure 3.27: Dean s problem solution comparison: a) pressure at cell (6, 11, 6), b) displacement at point (0,0) m (top curve), and point (0, 3048) m (bottom curve). Solid line represents reference solution (Dean et al., 2006), and symbols represent fully-implicit and sequential-implicit methods respectively Example 7. Water Flooding Problem The previous subsection results show that the sequentially implicit approach is able to converge to the desired tolerance, yet requires performing coupling iterations between subproblems. A question then arises: how expensive are these iterations? The main purpose of this subsection is to compare the performance of the fully-implicit and the sequentially implicit methods. To demonstrate the applicability of the proposed framework to solve problems of practical interest, we consider a modified SPE 10 test problem (Christie and Blunt, 2001). The original data set, including relative permeability curves, porosity and permeability distributions, fluid properties, and wells controls, was used in this study. The simulation domain Ω = m is discretized using a regular Cartesian grid into cells. The top 35 layers represent a formation with a permeability field that has the Gaussian distribution, and the bottom 50 layers characterized by a channelized permeability field with high-contrast features. A set of coarsened models, which contains from 24,960 to 399,360 cells, was generated (see Table 3.14). The upscaling of properties was performed using a simple volume-averaging approach. The original SPE 10 test case is a two-phase flow problem and does not involve geomechanics. Our extended model is supplemented by boundary conditions and parameters for

104 CHAPTER 3. NUMERICAL APPLICATIONS 87 Table 3.14: Number of cells and total degrees of freedom (DOF) for the simulation cases. Case Name 25k 50k 100k 200k 400k Number of cells 24,960 49,920 99, , DOF 134, , ,555 1,037,235 2,068,395 a pore-mechanical problem. We set zero normal displacement on the side boundaries, zero vertical displacement on the bottom boundary, and uniform loading conditions on the top boundary, F top = Pa. Figure 3.28: Water Flooding Problem. The Young s modulus distribution (units are Pa). Reservoir domain Ω = [0,250] [0,500] [ 3650, 3800] m. The top part is characterized by smoothly varying rock properties, whereas the bottom part has channelized distributions of the properties with long-correlation and high-contrast features. We take the Biot s coefficient equal to 1.0 and the Poisson ratio equal to 0.2. Three different settings with different coupling strengths are used to study the performance of the sequential scheme. A detailed description of the coupling strength for multiphase problem can be found in Kim et al. (2013) and Castelletto et al. (2015). The base case uses the original rock compressibility value, c r = Pa 1, to estimate the average Young s modulus, E av = 3(1 2ν)/c r. Then, Young s modulus values are distributed on the grid using the following porosity correlation: E 1.0 (x) = E av (a φ(x) + b), (3.15) where φ is the initial porosity, and parameters a and b are chosen to obtain min(e 1.0 ) = E 1.0 (max(φ)) = 0.2E av and max(e 1.0 ) = E 1.0 (min(φ)) = 5E av. Figure 3.28 illustrates the

105 CHAPTER 3. NUMERICAL APPLICATIONS 88 distribution of the Young modulus. The second and third cases use softer and stiffer moduli: E 0.1 = 0.1 E 1.0 and E 10.0 = 10 E 1.0. The former setup corresponds to a stronger coupling and the later setup corresponds to a weaker coupling (as compared with the base case). This set of models allow us to investigate the performance of sequential scheme for 15 different scenarios overall. Usually, the most time-consuming part of the simulation is the solution of the linear system. As has been mentioned above, the FIM approach for coupled geomechanical problems requires a sophisticated linear solver development (White et al., 2016), and the choice of efficient solvers is limited. In the following comparison, we employ our in-house iterative linear solver (Klevtsov et al., 2016), which applies a multi-stage preconditioning strategy combining the generalized Constrained Pressure Residual approach (Zhou et al., 2013) with the fixed-stress block-partitioned algorithm (White et al., 2016). The performance of the solver is validated against the widely used SAMG (Algebraic Multigrid Methods for Systems) software (Gries et al., 2014). We made a comparison for a set of problems of different resolutions ( degrees of freedom) and found that the difference in solution time between the two solvers is comparable, and does not change overall conclusions. This justifies using the in-house linear solver for FIM simulations. For the sequential scheme, we employ the Constrained Pressure Residual (CPR) preconditioned GMRES solver (Zhou et al., 2013) to solve the flow problem, and the SAMG-preconditioned GMRES solver to solve the geomechanical problem. We use the same convergence tolerance values for FIM and SEQ approaches. In addition to the effect of coupling strength between flow and mechanics, we also investigate how the nonlinearities related to each physics module affect the overall performance. The simulation results are summarized in Table In the first setup, to mimic a single-phase poromechanics behavior, we set the initial water saturation equal to one, S w = 1.0. In this case, for the sequential scheme, flow problem converges in one nonlinear iteration because the water compressibility is small, c w = Pa. Since in this case, the mechanical equation is linear and quasi-static, convergence is achieved in one nonlinear iteration. It has been shown before (Castelletto et al., 2015) that the sequential scheme requires larger numbers of iterations when the coupling is stronger. Consequently, the case E 10.0 requires less coupling iterations and has lower computational cost (Figure 3.29). The

106 CHAPTER 3. NUMERICAL APPLICATIONS 89 Figure 3.29: Water Flooding Problem. Comparison of FIM (solid color) and SEQ (dashed) simulation time for single-phase setup. Notice the logarithmic scale for the vertical axis. Table 3.15: Normalized simulation time for Water Flooding Problem. Corresponding FIM simulation time for the smallest problem (marked in bold) is used to normalize simulation time the single-phase setup (S w = 1.0) and the two-phase setup (S w = 0.2). E 0.1 E 1.0 E 10.0 Case FIM SEQ FIM SEQ FIM SEQ S w = 1.0, elasticity 25k k k k k S w = 0.2, elasticity 25k k k k k S w = 0.2, plasticity 25k k k sequential scheme outperforms the fully-implicit scheme for E At the same time, it performs comparably well for E 1.0, and slower for E 0.1 because the fully-implicit scheme

107 CHAPTER 3. NUMERICAL APPLICATIONS 90 is less sensitive to the change of coupling strength. Next, we solve two-phase problems with initial water saturation S w = 0.2. Remember that the porous matrix still has linear-elastic deformation characteristics, while the flow is highly nonlinear because of the saturation transport. However, the fully-implicit approach treats the overall system as nonlinear and multiple Newton steps are required for convergence. Thus, the cost of the fully-implicit scheme increases due to the increased number of nonlinear and linear iterations performed for the full system. On the other hand, the sequential scheme requires approximately the same number of coupling iterations as in the single-phase setup. In this solution scheme, multiple nonlinear iterations are required only for the smaller flow and transport problem, while the mechanical problem still needs only one nonlinear iteration. As a result, the sequential scheme is comparable or faster even in the case of strong coupling strength and becomes a very attractive solution strategy (Figure 3.30). Figure 3.30: Water Flooding Problem. Comparison of FIM (solid color) and SEQ (dashed) simulation time for two-phase setup. Notice the logarithmic scale for the vertical axis. Finally, we consider the two-phase flow problem but include plastic deformation (Drucker and Prager, 1952). Now both sets of equations are nonlinear. The fully-implicit approach requires almost the same number of nonlinear iterations as before and consequently, its performance differs only slightly from the elastic case. However, the performance of the

108 CHAPTER 3. NUMERICAL APPLICATIONS 91 Figure 3.31: Water Flooding Problem. Comparison of FIM (solid color) and SEQ (dashed) simulation time for the two-phase plastic setup. Light part of the bars correspond to the simulation time increase between elastic and plastic models. sequential scheme suffers a lot since two nested nonlinear loops are performed for each coupling step. This is especially evident in the case when the coupling is strong and the sequential approach demands the higher number of coupling iterations. For the moderate and weak coupling, the sequential approach still performs better than the fully-implicit approach (Figure 3.31) Summary We demonstrated the applicability of the GENIC framework for coupled nonlinear flowmechanics problems. First, we validated our implementation on Mandel s problem against the analytical solution. We tested two existing coupled properties: fixed-stress and fixedstrain. The results are consistent with published results (Castelletto et al., 2015; Kim, 2010), where fixed-strain is only conditionally stable, and fixed-stress is unconditionally stable. Next, we validated our framework on Dean s problem against the available numerical solution. Dean s problem demonstrated the framework s ability to model physics on different domains. Finally, we compared the performance of the sequential coupling

109 CHAPTER 3. NUMERICAL APPLICATIONS 92 strategy against the fully implicit approach in the extended SPE 10 problem. For the moderate and weak coupling, the sequential approach performs better than the fully-implicit approach, while it is not efficient for strongly coupled problems where both physics are highly nonlinear.

110 CHAPTER 3. NUMERICAL APPLICATIONS Coupled Flow, Thermal, and Mechanics Example 8. Cold Water Injection Here, we consider problems whereby the interactions involve three physics modules, namely, thermal, compositional, and mechanics. We demonstrate the applicability of the proposed framework to solve problems of practical interest (Rin et al., 2017). A modified version of the SPE 9 model (Killough, 1995) is considered (Fig. 3.32). We extend the description from three-phase black-oil to a 5-component system (H 2 S, C 1, C 5, C 13, H 2 O). The K-values flash algorithm (Rannou et al., 2013; Zaydullin, 2015) is used for the phase equilibrium calculations. There are 25 producers and one injector as in the original SPE9 model. All the wells operate with a BHP control. The model parameters are summarized in Table Here, our starting point and reference is the fully coupled solution obtained using Ψ 1 = FIM[FLOW,THERMAL, MECHANICS]. (3.16) For the sequential simulations, we apply the fixed-stress splitting (Kim, 2015). In the first setup, the flow and thermal problems are solved together, followed by the mechanical overburden SPE9 reservoir underburden Figure 3.32: Cold water injection problem. The domain includes the overburden, under-burden, and the reservoir. The reservoir geometry, porosity, permeability, well locations are from the SPE 9 problem. The permeability field is shown and black blocks represent the well locations.

111 CHAPTER 3. NUMERICAL APPLICATIONS 94 Table 3.16: Main parameters for Cold Water Injection problem. See Killough (1995) for full specifications of the reservoir properties. Grid dimensions Grid block size in X and Y directions, m Reservoir depth range, m Reservoir dip in X direction, deg 10 Reservoir layers thickness, m 6.1, 4.6, 7.9, 4.6, 4.9, 4.3, 2.4, 2.4, 5.5, 3.7, 5.8, 5.5, 6.1, 15.2, 30.5 Young s modulus, Pa Biot s constant 1.0 Poisson s ratio 0.25 Undrained Poisson s ratio 0.47 Rock density, kg/m Rock thermal expansion, 1/K Rock heat capacity, kj/(kg K) Rock thermal conductivity, kj/(m day K) Oil thermal conductivity, kj/(m day K) 11.5 Gas thermal conductivity, kj/(m day K) Water thermal conductivity, kj/(m day K) 53.5 Initial reservoir overall composition 0.049, 0.18, 0.6, 0.157, Initial reservoir temperature, K BHP, Pa 10 7 for producers, for injector Injection temperature, K Relative phase permeability k r j = S 2 j deformation step: Ψ 2 = SEQ[FIM[FLOW, THERMAL], MECHANICS]. (3.17) The method requires 3 10 iterations to converge for a tolerance ε = 10 4 (Figure 3.33). The SEQ simulation results agree with the FIM solution. Figure 3.34 shows the final saturation and temperature distributions together with the differences between FIM and SEQ results. The calculated production and injection rates are also nearly identical (Figure 3.35). Next, we run the sequential algorithm in a nested way: the inner sequential loop is between the flow and thermal problems; the outer loop is between the flow-thermal block and geomechanics: Ψ 3 = SEQ[SEQ[FLOW, THERMAL], MECHANICS]. (3.18)

112 CHAPTER 3. NUMERICAL APPLICATIONS 95 Figure 3.33: Cold Water Injection. Sequentially implicit method convergence profile after first time step t = days. The scheme is able to reproduce the fully coupled solution; however, it experiences substantial stability problems. Specifically, the coupling scheme does not converge for the same time steps as the FIM method (the average time step is about two times smaller). The investigation of stable coupling schemes for multiphase thermal problems is the subject of future research. Table 3.17 summarizes a comparison between fully implicit, sequentially implicit, and nested sequential-implicit strategies. The fully implicit and sequentially implicit methods show stable convergence behavior without time step cuts, and take the same time step size, while the nested sequentially coupled scheme is less stable due to instabilities in resolving the coupling between the flow and thermal physics modules. The SEQ scheme converges in 1-2 sequential iterations per time step. While the SEQ scheme takes more total Newton Table 3.17: Cold Water Injection case. Comparison between FIM and SEQ schemes FIM (Ψ 1 ) SEQ (Ψ 2 ) Ψ 3 number of timesteps total Newton iterations total sequential iterations number of timesteps wasted maximum CFL normalized total time linear factorization time, % linear solution time, % 5 4 4

113 CHAPTER 3. NUMERICAL APPLICATIONS 96 Figure 3.34: Saturation and temperature distributions in the middle of the reservoir at the end of simulation for FIM approach (left) and difference in them between FIM and SEQ simulations (right). Figure 3.35: Oil production and water injection rates. FIM and SEQ approaches produce nearly identical results.

114 CHAPTER 3. NUMERICAL APPLICATIONS 97 iterations than FIM (1170 vs 773), it requires fewer Newton iterations per physics (585 vs 773) taking advantage of weak coupling between the physics modules. As a result, SEQ scheme takes 32% of the FIM simulation time, and is much more efficient for this problem. Moreover, both linear factorization and solution time account for a smaller ratio of the total time compared with the FIM scheme (94% vs 96%). This is due to the smaller system size, and less memory overhead for the SEQ scheme Summary We study flow-thermal-mechanics problems using three different coupling strategies: fully coupled, sequentially coupled, and nested sequentially coupled. This is the first example available in the literature where both fully coupled and sequentially coupled approaches are applied to this type of three-physics problem. We found that when the mechanics problem is linear, and the coupling strength is weak, the sequential coupling strategy performs faster than FIM; however, as the coupling strength involving mechanics increases, sequential coupling strategies become less effective.

115 Chapter 4 Future Extensions of GENIC In this chapter, we elaborate on the details of possible framework extensions, such as automatic coupled properties, parallelization, and multiscale formulations. 4.1 Automatic Coupled Properties There are several limitations of the current design of Coupled Properties (CP), which is expected to become critical for problems with a large number of coupled properties. First, the order of CP calculations requires careful arrangement. For example, in thermal-mechanics problems we may have two coupled properties: porosity φ and enthalpy h, where φ is a function of h. While we may solve thermal-mechanics and mechanics-thermal problems without code modification, the order of the CP is fixed. Here, we must calculate h before φ, otherwise both of them would use inconsistent parameters; therefore, in this case, only one order for the coupled properties guarantees the desired sequence of operations. This limitation is illustrated in Figure 4.1, where we demonstrate CP s order dependency for Ψ = SEQ[MECHANICS, THERMAL] coupling. Second, all CPs are updated whenever a sub-problem is solved to avoid the use of nonupdated CPs when another sub-problem is solved. This guarantees the correctness of the overall solution; however, it might become costly if CP is expensive to calculate when it is not used. This limitation is based on the previous restriction: proper update needs to update all relevant CPs. Figure 4.2 demonstrates an example of a sequentially coupled 98

116 CHAPTER 4. FUTURE EXTENSIONS OF GENIC 99 Figure 4.1: Schematic representation of the solution for Ψ = SEQ[MECHANICS,THERMAL]. In case 1, we update porosity prior to enthalpy. As a result, porosity φ(h) uses old enthalpy leading to a wrong value of porosity. In case 2, we update enthalpy followed by porosity to account dependency between them. Figure 4.2: Update of CPs for Ψ = SEQ[MECHANICS,THERMAL]: a) current design - all properties are updated, b) naive design: only related properties are updated ignoring a dependency between CPs, c) correct design - minimal CP update. mechanics-thermal problem, where the mechanics sub-problem depends on porosity φ(h), and the thermal sub-problem uses enthalpy h. It shows three designs: (a) full, (b) naive, and (c) correct. While a naive design results in a minimal update, it ignores the dependency between coupled properties causing the wrong solution of the mechanics sub-problem. The correct design (c) produces minimal possible CP updates and accounts for dependency between φ and h. A possible way to resolve these framework limitations is by using a dependency graph, which is a directed, acyclic graph describing the functional relationship of each CP in the system of equations (see Figure 4.3). Leaf (terminal) nodes in the dependency are unknowns and properties stored in corresponding sub-problem s subsets (shown in blue).

117 CHAPTER 4. FUTURE EXTENSIONS OF GENIC 100 Figure 4.3: A dependency graph for a flow-thermal-mechanics problem. The unknowns and properties are shown in blue, and coupled properties are shown in green. Arrows shows the relationship between coupled properties and unknowns. All other nodes are coupled properties (shown in green), and directed edges represent the relationships between them. Based on the dependency graph, we can reconstruct a proper order of CP independently of initialization. Moreover, the order automatically changes to align with used properties and physics. From the example in Figure 4.3, we have three coupled properties: mobility µ, enthalpy h, and porosity φ(h). Using a dependency graph it is easy to find the right order: calculate mobility µ and enthalpy h independently from each other, followed by porosity φ update. Additionally, the dependencies between coupled properties allow the recalculation of the CPs only if their dependencies have changed, resulting in automatic coupled properties evaluation. As a consequence, dependency graphs allow for a more flexible and efficient framework for a large number of coupled properties when identification of their correct order may be challenging. 4.2 Parallelization GENIC framework parallelization is implemented via shared memory multi-processing through OpenMP. However, there are serious limitations to solve large-scale problems of practical interest caused by the mostly serial implementation of the ADETL library. We can

118 CHAPTER 4. FUTURE EXTENSIONS OF GENIC 101 Figure 4.4: Scheme of the interaction between simulator and the ADETL library in the GENIC framework. Both mappers and subsets are part of ADETL. achieve much better scalability by utilizing a distributed memory system, such as MPI. We proceed as follows. First, we explain the current issues with the parallelization of GENIC, and we elaborate on how we can resolve them in distributed memory architecture. Figure 4.4 depicts the GENIC framework in terms of the interaction between the ADETL library, and a simplified representation of the simulator. All the properties are stored in AD vectors, which contain both values and derivatives for each unknown. Subsets provide efficient and unified access to AD vectors (and related data) through the use of mappers and a status table. Finally, the main set organizes logic within subsets and performs updates whenever a change of global status is needed causing a global renumeration. As we explained in detail in Chapter 2, GENIC framework heavily relies on a subset concept to provide seamless physics integration and straightforward physics extension. Subsets are designed as a part of ADETL to minimize memory and CPU overheads. The parallelization of GENIC is impossible without parallel access to data and subsets. Consequently, parallelization of ADETL is a required step towards the development of a scalable parallel multi-physics simulator with a flexibility provided by the framework. In addition, solver part should also be modified to account for the distributed memory system. One way to provide such modifications is through the use of Parallel Mapping Operators (PMO). Similar to the mapping operators introduced in Chapter 2, PMO defines a

119 CHAPTER 4. FUTURE EXTENSIONS OF GENIC 102 (a) (b) Figure 4.5: Differences between regular and parallel mapping operators. On the left (a): both M 1,M 2 belong to the same computational node. On the right (b): P 1 and P 2 might belong to different computational nodes, and P 1 = P1 local P ghost 1. Note: dashed line separates two local parts of P 1 and P 2. sub-domain related to the subset. However, it has several differences as shown in Figure 4.5. First, PMO has two parts: local (sub-domain itself) and ghost, which represents cells from neighboring nodes. Ghost part minimizes communications betweens nodes and performs synchronization. Second, PMOs are distributed among different nodes (along with the memory allocated for related parts of AD vectors) and can be solved in parallel. Finally, each PMO has a fixed domain with memory allocation in initialization stage (regular mapping operator allows a dynamic domain change). We employ PMO by replacing a sequential representation of the coupling strategy with a parallel. For example, for a flow-thermal problem we substitute a sequential coupling strategy Ψ 1 = SEQ[FLOW, THERMAL] with Ψ 2 = SEQ[FIM[FLOW<P-i>], FIM [THERMAL<P-i>]], where P-i is short notation for P1,...,Pn, and each P-i is parallel mapping operator. Notice, the number n of used PMOs are computed internally and based on partitioning algorithm on initialization stage. For Ψ 2 we employ an extended status table as shown in Figure 4.6. This strategy preserves a flexibility of the framework in terms of coupling strategies and mapping operators. Next, we need to solve a sub-problem with PMO s in parallel. This requires small modification in Algorithm 1, where we split line 4 (solve active sub-problems using a linear solver) in two steps. First, for each PMO we compute its ghost part by synchronizing with neighboring nodes. This step is skipped if the ghost part is empty (as might be needed for multi-scale). Second, we solve each sub-problem with applied PMO in parallel, and other sub-problems in serial. Since we have parallel access to ADETL there is no need to merge data from different parts of the sub-problems. Notice, Algorithm 1 transforms to its

120 CHAPTER 4. FUTURE EXTENSIONS OF GENIC 103 Global Status F T status 1 status 2 (a) Global Status F 1 F 2... F n T 1 T 2... T n status 1 P 1 P 2... P n... status 2... P 1 P 2... P n (b) Figure 4.6: Parallel implementation through parallel mapping operators for a flow-thermal problem for n computation nodes. For brevity, we use F for FLOW, and T for THERMAL. On the left (a), serial coupling strategy. On the right (b), similar yet parallel coupling strategy, where P 1,...,P n are PMOs, and F 1,...,F n and T 1,...,T n are related parts of flow and thermal domains correspondingly. original when no parallel mapping operators are used. We briefly described the GENIC framework modifications for distributed memory environment. There are other challenges in implementation, such as an efficient parallel access to AD Vector through subsets, the optimization of global status change routine in the main set (which restructures a global Jacobian), and the updates of parallel coupled properties. In addition, hiding this new level of complexity on the general level is another challenge. 4.3 Multi-Scale Framework Multi-scale methods in general and particularly the Multi-scale Finite Volume (MsFV) method represent an active area of research in the context of reservoir simulation (Jenny et al., 2003; Lunati and Jenny, 2006; Hajibeygi et al., 2008; Zhou, 2010; Cusini et al., 2015; Myner, 2016; Kozlova et al., 2015). Recently the MsFV approach was extended for multiphysics applications (Tomin and Lunati, 2013, 2015, 2016), where pore-scale and Darcyscale physics were combined in a general conservation-based local-global framework with flexible spatiotemporal adaptive strategies. Nevertheless, to the best of author s knowledge, available commercial solutions (DeBaun et al., 2005; Cusini et al., 2015; Myner, 2016; Kozlova et al., 2015) and research prototypes, are limited in terms of physics to flow-transport problems at Darcy scale; a flow-thermal-mechanics multi-scale simulator does not yet exist. This limitation can be resolved by extending the GENIC framework to support multi-scale methods. The flexibility of GENIC allows coupling multi-scale and standard methods within the same simulation. In addition, presented ideas could be applied for smart upscaling approaches including adaptive local-global upscaling (Chen and

121 CHAPTER 4. FUTURE EXTENSIONS OF GENIC 104 Durlofsky, 2006; Li and Durlofsky, 2016). Algorithm 6 presents the basic steps of the MsFV approach (see, e.g., Tomin and Lunati (2015) for details). First a grid partitioning (coarse and dual grids) has to be provided (step 1). Next, interpolators that include basis (and correction functions) are computed locally on the dual grid cells (step 2) and a global problem is constructed (step 3). After the global problem is solved, boundary conditions for the local sub-problems are constructed (step 4) and the full system of equations is solved locally on the coarse grid cells (step 5). Steps 2 5 are repeated till the end of the simulation. Algorithm 6: Basic steps of the MsFV algorithm. 1: generate MsFV grids (dual and coarse) 2: compute basis functions 3: construct and solve coarse system 4: construct boundary condition for coarse cells by prolongating the coarse solution back to the fine scale 5: solve local fine systems with prescribed boundary conditions Based on Algorithm 6, we treat each scale in MsFV as a separate sub-problem. We denote the coarse system as FLOW COARSE, dual grid computations as FLOW BASIS, and the local fine systems as FLOW. We introduce parallel mapping operators DUAL and FINE for dual and fine systems correspondingly. Since no global synchronization is required between applied PMO, ghost part of each PMO is empty. Figure 4.7 shows a schematic representation of the multi-scale sub-problems and the corresponding parallel mapping operators. We couple the fine and coarse scales with basis functions computed on the dual grid. Thereby, the basis functions can be considered as coupled properties. The interaction between them and sub-problems is shown on Figure 4.8. Finally, we write the MsFV algorithm in GENIC framework as Ψ = SEQ[FIM[FLOW BASIS<DUAL-i>], FLOW COARSE, FIM[FLOW<FINE-i>]] It must be noted that FLOW<FINE-i> differs from a standard flow problem by using basis functions to reconstruct required quantities (boundary conditions). When basis functions are not computed (i.e. in other non-msfv methods), standard computations are called. Therefore, the MsFV approach requires only the two new sub-problems (FLOW BASIS and

122 CHAPTER 4. FUTURE EXTENSIONS OF GENIC 105 Figure 4.7: Schematic representation of multi-scale sub-problems and corresponding sub-domains on grid. On the left (a) - four flow sub-problem on FINE-isub-domain (shown in blue), in the center (b) - one coarse sub-problem, and on the right (c) - dual grid with 9 sub-domains (DUAL-i) for computation of basis functions (shown in green). Figure 4.8: Interaction between multi-scale sub-problems and their coupled properties - basis functions ϕ i. FLOW COARSE), construction of MsFV grids at initialization stage, and one coupled property (basis functions). There is no need to modify any general algorithm for MsFV methods. We can couple multi-scale and standard methods within one coupling strategy by adding MsFV logic into it. For example, we might employ three coupling strategies: Ψ 1 = SEQ[SEQ[FIM[FLOW BASIS<DUAL-i>], FLOW COARSE, FIM[FLOW<FINE-i>], THERMAL], Ψ 2 = SEQ[FLOW, SEQ[FIM[THERMAL BASIS<DUAL-i>], THERMAL COARSE, FIM[THERMAL <FINE-i>]]], Ψ 3 = SEQ[FIM[FLOW BASIS<DUAL-i>, THERMAL BASIS<DUAL-i>], FIM[FLOW COARSE, THERMAL COARSE],... FIM[FLOW<FINE-i>,THERMAL[FLOW<FINE-i>]]]. In Ψ 1, we couple an MsFV flow problem followed by a thermal problem. In Ψ 2, we couple a flow problem with an MsFV thermal problem. Finally, in Ψ 3, we solve a flowthermal problem using MsFV for both physics. The flexibility in coupling MsFV and standard methods within the same framework can identify new methods of coupling multi-physics within MsFV framework, or test new

123 CHAPTER 4. FUTURE EXTENSIONS OF GENIC 106 multi-scale preconditioners. In addition, both MsFV and non-msfv methods co-exist within the same framework this enables an accurate comparison between methods.

124 Chapter 5 Conclusions and Future Work In this work, we developed a unique general implicit coupling framework for multi-physics problems (GENIC). The framework is implemented as the next generation of the Automatic- Differentiation General-Purpose Research Simulator (AD-GPRS) and represents a flexible and efficient reservoir simulation research laboratory in which students and researchers can easily practice their ideas. We applied GENIC to solve challenging two- and three-physics problems with novel coupled properties and coupling strategies. Here, we summarize the findings from this work followed by recommendations for future research efforts. 5.1 Conclusions We designed and implemented a multi-physics framework to address the challenges in current multi-physics simulator development such as a code extensibility to new physics and capabilities, and flexibility in the choice of coupling strategies for various physical processes. The exploration of different coupling strategies facilitates the study of new multi-physics preconditioners. The newly developed framework supports any coupling strategy, while the modular design with transparent subset-based interfaces greatly improves code extensibility to new physics and complex features. This level of flexibility is achieved with a sub-problem tree structure, global status table, and its integration with subsets through an automatic differentiation layer. In addition, we employed a unique mapping operators concept, which enables advanced 107

125 CHAPTER 5. CONCLUSIONS AND FUTURE WORK 108 coupling strategies including (parallel) domain-decomposition and multi-scale approaches. As a result, we have designed, developed, implemented and tested a novel general implicit coupling framework for multi-physics problems (GENIC) that allows rapid customizations. We explored existing (constant pressure) and novel coupled properties, such as a constant density and a hybrid scheme for flow-thermal sequential coupling strategies. We investigated the convergence properties for three different sequential coupling schemes. We found that a hybrid method where the coupled property was pressure for single-phase blocks and density for two-phase blocks performed the best. This hybrid method was able to avoid convergence problems in the single-phase (constant density) or two-phase (constant pressure) regions. We presented a new approach for coupling flow and thermal physics modified Sequentially Fully Implicit (m-sfi) method. We demonstrated its performance for several challenging examples, where the standard sequential approach fails, while m- SFI takes a similar number of time steps as the fully coupled approach (FIM). In cases where accurate front prediction is possible and the two-phase region is limited, such as strictly injection or production problems, m-sfi approach outperforms FIM. However, when the two-phase process appears in most of the reservoir, the overall performance of the m-sfi method is slower than FIM due to the increased cost of sequential iterations. We demonstrated the applicability of the GENIC framework for coupled nonlinear flow-mechanics problems. First, we validated our implementation on Mandel s problem (Mandel, 1953) against the analytical solution. We tested two existing coupled properties: fixed-stress and fixed-strain. The results are consistent with published works (Kim, 2010; Castelletto et al., 2015), where the fixed-strain is only conditionally stable, and the fixed-stress is unconditionally stable. Next, we validated our framework on Dean s problem (Dean et al., 2006) against the available numerical solution. Dean s example demonstrated the applicability of GENIC to solve a problem where different physics modules are applied in different spatial domains. Finally, we compared the performance of the sequential coupling strategy against the fully

126 CHAPTER 5. CONCLUSIONS AND FUTURE WORK 109 implicit approach for an extended SPE 10 problem. For the moderate and weak coupling, the sequential approach performs better than the fully-implicit approach, while it is not efficient for strongly coupled problems where both physics are highly nonlinear. We investigated flow-thermal-mechanics problems using three different coupling strategies: fully coupled, sequentially coupled, and nested sequentially coupled. This is the first example available in the literature where both fully coupled and sequentially coupled approaches are applied and analyzed to this type of three-physics problem. We found that when the mechanics problem is linear, and the coupling strength is weak, the sequential coupling strategy performs faster than FIM; however, as the coupling strength increases, sequential strategies become less effective. 5.2 Recommendations for Future Work In addition to automatic coupled properties, multi-scale and parallelization discussed in Chapter 4, future research efforts in the areas addressed in this work may include the following: Extend the framework to new physics and new coupled properties. For example, the next research objective might include a study of multi-physics processes with chemical reactions and electrical resistivity phenomena. Both physics are already implemented in the framework, however, the choice of coupled properties poses some challenges for a sequential scheme. Apply an adaptive coupling strategy for a given problem. In this work, we showed that the choice of optimal (in terms of performance) coupling strategy depends on the coupling strength between physics. In addition, the overall performance and convergence are affected by the choice of coupled property, coupling terms, and advanced coupling strategies. Presently, the framework allows switching of a coupling strategy between timesteps and should be extended to support various coupled properties and coupling terms for different timesteps. We suggest applying machine learning for

127 CHAPTER 5. CONCLUSIONS AND FUTURE WORK 110 a large set of realistic problems to establish reliable criteria for adaptively changing between coupling strategies. Develop a technique to generate an optimized specialized simulator based on the GENIC framework. The GENIC framework is a very flexible platform, however, this flexibility comes with some performance and memory overhead. It is possible to automatically create a low-level optimized code based on the used physics, coupled properties, and coupling strategy. This challenging task can be accomplished with the advancement of automatic code generation in computer science. Extend the framework to support different time discretization for different sub-problems. It might be beneficial to use smaller timesteps for faster processes such as chemical reactions or wellbore modeling. Apply the framework to study sequential coupling for a facility network which has significant convergence issues in the simulation of large oil fields with hundreds and thousands of wells even with the fully implicit method. Often facility network shows slow convergence comparing to a reservoir compositional part. We might improve convergence of such problems by solving them in a sequential fashion. Use the mapping operator concept from the GENIC framework to construct a better preconditioner for coupled systems. For example, instead of solving FIM[FLOW,THERMAL], we might solve SEQ[THERMAL<M1>, FIM[FLOW,THERMAL]]. The later coupling strategy mimics a preconditioner for a thermal part of the coupled flow-thermal problem and allows rapid exploration in this area which traditionally requires major changes in linear solver part. Incorporate new linear solvers and preconditioners to solve large-scale problems. Optimize the GENIC framework in terms of memory consumption and performance (through efficient cache usage).

128 Nomenclature α α j α φ u v j εv p ε v,o ε v κ C F Skeleton linear thermal expansion/dilation coefficient Holdup, or in-situ phase fraction of phase j Volumetric thermal dilation coefficient related to the porosity Displacement vector Velocity of phase j Volumetric plastic strain Reference volumetric strain Volumetric total strain Thermal-conductivity coefficient Fourth order elasticity moduli tensor Yield surface µ j Viscosity of phase j q j φ φ p φ o Inflow of phase j to the well Porosity Irreversible porosity Reference porosity 111

129 CHAPTER 5. CONCLUSIONS AND FUTURE WORK 112 Ψ ψ ρ ρ f ρ j ρ m j ρ s σ ε e b a θ A Coupling strategy Internal friction angle Overal-mass density Average fluid density Mass density of phase j Molar density of phase j Density of solid skeleton Total stress tensor Second order elasticity strain tensor Second order tensor of Biot s coefficients Thermal expansion tensor Inclination angle of the segment from vertical Set of given physics Cross-sectional area of an multi-segment well segment A, B Material parameters in Drucker Prager model c Cohesion c F,c T,c M Coupled property for flow, thermal, and mechanics d f i, j h j I 1 Distance between cell centers Fugacity of component i in phase j Enthalpy of phase j Volumetric invariants of the effective stress

130 CHAPTER 5. CONCLUSIONS AND FUTURE WORK 113 J 2 k K d k r j m n c n p P p P c p j P o q loss r w Deviatoric invariants of the effective stress Absolute permeability Local drainage bulk modulus Relative permeability of phase j Permeability exponent Number of components Number of phases Average fluid pressure Pressure Capillary pressure Pressure of phase j Reference pressure Head loss from wellbore fluid to surroundings Wellbore radius R F,R T,R M Flow, thermal, and mechanics residuals S j T T w u j u r u to Saturation of phase j Temperature Temperature of wellbore fluid Internal energy of phase j Rock internal-energy Overall heat transfer coefficient

131 CHAPTER 5. CONCLUSIONS AND FUTURE WORK 114 V in, j V s, j Interstitial velocity of phase j Superficial phase velocity of phase j x F,x T,x M Flow, thermal, and mechanics unknowns x i, j Mass fraction of component i in phase j

132 Bibliography Aavatsmark, I., Barkve, T., and Mannseth, T. (1998). Control-volume discretization methods for 3D quadrilateral grids in inhomogeneous, anisotropic reservoirs. SPE Journal, 3(2): Aavatsmark, I., Eigestad, G., Heimsund, B., Mallison.B., Nordbotten.J., and /0ian E. (2010). Control-volume discretization methods for 3D quadrilateral grids in inhomogeneous, anisotropic reservoirs. SPE Journal, 15(3): Abousleiman, Y., Cheng, A.-D., Cui, L., Detournay, E., and Roegiers, J.-C. (1996). Mandel s problem revisited. Géotechnique, 46(2): Aboustit, B., Advani, S., and Lee, J. (1985). Variational principles and finite element simulations for thermo-elastic consolidation. International Journal for Numerical and Analytical Methods in Geomechanics, 9(1): Abrajano, T. A. and Bates, J. K. (1987). Transport and reaction kinetics at the glass solution interface region: results of repository-oriented leaching experiments. In: Bates, J.K., Seefeldt, W.B. (Eds.), pages Scientific Basis for Nuclear Waste Management X. Mater. Res. Soc, Pittsburgh, PA, USA. Agullo, E., Demmel, J., Dongarra, J., Hadri, B., Kurzak, J., Langou, J., Ltaief, H., Luszczek, P., and Tomov, S. (2009). Numerical linear algebra on emerging architectures: The PLASMA and MAGMA projects. Journal of Physics: Conference Series, 180(1): Ames, L. and Rai, D. (1978). Radionuclide interactions with soil and rock media: processes influencing radionuclide mobility and retention, element chemistry and geochemistry, 115

133 BIBLIOGRAPHY 116 conclusions and evaluation,. vol. 1. Final Report., page 330. Battelle Pacific Northwest Labs., Richland, WA, USA. Armero, F. and Simo, J. (1992). A new unconditionally stable fractional step method for non-linear coupled thermomechanical problems. International Journal for Numerical Methods in Engineering, 35(4): Avdonin, N. (1964). Some formulas for calculating the temperature field of a stratum subject to thermal injection. Neft i Gaz, 3: Aziz, K., Durlofsky, L., and Tchelepi, H. (2005). Notes for petroleum reservoir simulation. Department of Energy Resource Engineering, Stanford, CA, page 186. Aziz, K. and Settari, A. (1979). Petroleum Reservoir Simulation. Applied Science Publishers LTD. Aziz, K. and Wong, T. (1989). Considerations in the development of multipurpose reservoir simulation models. In 1st and 2nd International Forum on Reservoir Simulation. Bendtsen, C. and Stauning, O. (1996). FADBAD, a flexible C++package for automatic differentiation. Technical report, IMM REP , Department of Mathematical Modelling, Technical University of Denmark, Lyngby, Denmark. Berzins, M., Meng, Q., Schmidt, J. A., and Sutherland, J. C. (2011). DAG-based software frameworks for PDEs. In Euro-Par Workshops (1), pages Bevillon, D. and Masson, R. (2000). Stability and convergence analysis of partially coupled schemes for geomechanical reservoir simulations. In The European Conference on the Mathematics of Oil Recovery. Bischof, C. H., Bücker, H. M., Hovland, P., Naumann, U., and Utke, J. (2008). Advances in Automatic Differentiation. Springer Publishing Company. Borja, R. I. (2013). Plasticity: Modeling & Computation. Springer Science & Business Media.

134 BIBLIOGRAPHY 117 Butler, R. (1985). New approach to the modelling of steam-assisted gravity drainage. Journal of Canadian Petroleum Technology, 24(3): Byer, T. (2000). Preconditioned Newton methods for Simulation of Reservoir with Surface Facilities. PhD thesis, Stanford University. Cao, H. (2002). Development of Techniques for General Purpose Simulators. PhD thesis, Stanford University. Carey, J., McSaveney, M., and Petley, D. (2016). Dynamic liquefaction of shear zones in intact loess during simulated earthquake loading. Landslides, 3(14): Castelletto, N., White, J. A., and Tchelepi, H. A. (2015). Accuracy and convergence properties of the fixed-stress iterative solution of two-way coupled poromechanics. International Journal for Numerical and Analytical Methods in Geomechanics, 39(14): Chaussard, E., Amelung, F., Abidin, H., and Hong, S.-H. (2013). Sinking cities in Indonesia: ALOS PALSAR detects rapid subsidence due to groundwater and gas extraction. Remote Sensing of Environment, 128: Chen, Y. and Durlofsky, L. J. (2006). Adaptive local global upscaling for general flow scenarios in heterogeneous formations. Transport in porous Media, 62(2): Chien, M., Lee, S., and Chen, W. (1985). A new fully implicit compositional simulator. In SPE Reservoir Simulation Symposium. Christie, M. and Blunt, M. (2001). Tenth SPE comparative solution project: A comparison of upscaling techniques. In SPE Reservoir Simulation Symposium. Coats, K. H. (1976). Simulation of steamflooding with distillation and solution gas. SPE Journal, 16(5): Coats, K. H. (1980). An equation of state compositional model. SPE Journal, 20(05):

135 BIBLIOGRAPHY 118 Coats, K. H. (2000). A note on IMPES and some IMPES-based simulation models. SPE Journal, 5(3): Coats, K. H. (2003). Compositional and black oil reservoir simulation. SPE Journal, 8(2): Coats, K. H., George, W. D., and Marcum, B. E. (1974). Tree-dimensional simulation of streamflooding. Trans, SPE of AIME, 257: Coon, E. T., Moulton, J. D., and Painter, S. L. (2016). Managing complexity in simulations of land surface and near-surface processes. Environmental Modelling & Software, 78: Coussy, O. (2004). Poromechanics. John Wiley and Sons Ltd. Craig, A. P., Vertenstein, M., and Jacob, R. (2012). A new flexible coupler for earth system modeling developed for CCSM4 and CESM1. The International Journal of High Performance Computing Applications, 26(1): Craig, T. (2010). CPL7 User s Guide. cpl7/cpl7_doc/ug.pdfs. [Online; accessed 03-July-2017]. Cusini, M., Lukyanov, A. A., Natvig, J., and Hajibeygi, H. (2015). Constrained pressure residual multiscale (CPR-MS) method for fully implicit simulation of multiphase flow in porous media. Journal of Computational Physics, 299: Danilov, D. L., Konovalov, A. N., and Yakuba, S. I. (1968). Equations and boundary value problems in the theory of two phase flows through porous media. Dokl. Akad. Nauk. USSR, 183(2): David, C., Wong, T., Zhu, W., and Zhang, J. (1994). Laboratory measurement of compaction-induced permeability change in porous rocks: Implications for the generation and maintenance of pore pressure excess in the crust. Pure and Applied Geophysics, 143(1-3):

136 BIBLIOGRAPHY 119 De Loubens, R., Riaz, A., and Tchelepi, H. A. (2009). Error analysis of an adaptive implicit scheme for hyperbolic conservation laws. SIAM Journal on Scientific Computing, 31(4): Dean, R., Gai, X., Stone, C., and Minkoff, S. (2006). A comparison of techniques for coupling porous flow and geomechanics. SPE Journal, 11(1): DeBaun, D., Byer, T., Childs, P., Chen, J., Saaf, F., Wells, M., Liu, J.,, Cao, H., Pianelo, L., Tilakraj, V., Crumpton, P., Walsh, D., Yardumian, H., Zorzynski, R., Lim, K.-T., Schrader, M., Zapata, V., Nolen, J., and Tchelepi, H. (2005). An extensible architecture for next generation scalable parallel reservoir simulation. In SPE Reservoir Simulation Symposium. Devanathan, R., Gao, F., Sun, X., and Khaleel, M. (2010). Fundamental science-based simulation of nuclear waste forms. Fuel Cycle Research & Development. Prepared for U.S. Department of Energy Fuel Cycle R&D Program. Pacific Northwest National Laboratory. Richland, WA. FCR&D-MDSM Drucker, D. and Prager, W. (1952). Soil mechanics and plastic analysis or limit design. Quarterly of Applied Mathematics, 10: Dunlap, R., Rugaber, S., and Mark, L. (2013). A feature model of coupling technologies for Earth System Models. Computers & Geosciences, 53: Edwards, M. and Rogers, C. (1998). Finite volume discretizations with imposed flux continuity for the general tensor pressure equation. Computational Geosciences, 2(4): Ehlers, W. and Ellsiepen, P. (1998). PANDAS: Ein FE-System zur simulation von sonderproblemen der bodenmechanik. Finite Elemente in der Baupraxis: Modellierung, Berechnung und Konstruktion, Beiträge zur Tagung FEM, 98: Elnashai, A. S. and Di Sarno, L. (2008). Fundamentals of Earthquake Engineering. Wiley New York.

137 BIBLIOGRAPHY 120 Fan, Y. (2010). Chemical Reaction Modeling in a Subsurface Flow Simulator with Application to In-Situ Upgrading and CO 2 Mineralization. PhD thesis, Stanford University. Farshidi, S. (2016). Compositional Reservoir Simulation-Based Reactive-Transport Formulations, with Application to CO 2 Storage in Sandstone and Ultramafic Formations. PhD thesis, Stanford University. Flemisch, B., Darcis, M., Erbertseder, K., Faigle, B., Lauser, A., Mosthaf, K., Mthing, S., Nuske, P., Tatomir, A., Wolff, M., and Helmig, R. (2011). DuMux: DUNE for multi- {phase, component, scale, physics,...} flow and transport in porous media. Advances in Water Resources, 34(9): Fussell, L. and Fussell, D. (1979). An iterative technique for compositional reservoir models. SPE Journal, 19(4): Gai, X. (2004). A Coupled Geomechanics and Reservoir Flow Model on Parallel Computers. PhD thesis, The University of Texas at Austin. Garipov, T., Karimi-Fard, M., and Tchelepi, H. (2016a). Discrete fracture model for coupled flow and geomechanics. Computational Geosciences, 20(1):149. Garipov, T., Voskov, D., and Tchelepi, H. (2015). Rigorous coupling of geomechanics and thermal-compositional flow for SAGD and ES-SAGD operations. In SPE Canada Heavy Oil Technical Conference. Garipov, T., White, J., Lapene, A., and Tchelepi, H. (2016b). Thermo-hydro-mechanical model for source rock thermal maturation. In 50th US Rock Mechanics / Geomechanics Symposium. Gaucher, E., Robelin, C., Matray, J., Ngrel, G., Gros, Y., Heitz, J., Vinsot, A., Rebours, H., Cassagnabre, A., and Bouchet, A. (2004). ANDRA underground research laboratory: interpretation of the mineralogical and geochemical data acquired in the Callovian- Oxfordian formation by investigative drilling. Physics and Chemistry of the Earth, Parts A/B/C, 29(1):55 77.

138 BIBLIOGRAPHY 121 Grambow, B., Landesman, C., and Ribet, S. (2014). Nuclear waste disposal: I. laboratory simulation of repository properties. Applied Geochemistry, 49: Gries, S., Stüben, K., Brown, G. L., Chen, D., and Collins, D. A. (2014). Preconditioning for efficiently applying algebraic multigrid in fully implicit reservoir simulations. SPE Journal, 19(04): Griewank, A., Juedes, D., and Utke, J. (1996). Algorithm 755: ADOL-C: a package for the automatic differentiation of algorithms written in C/C++. ACM Transactions on Mathematical Software (TOMS), 22(2): Hajibeygi, H., Bonfigli, G., Hesse, M. A., and Jenny, P. (2008). Iterative multiscale finitevolume method. Journal of Computational Physics, 227(19): Hansen, K., Prats, M., and Chan, C. (1995). Modeling of reservoir compaction and surface subsidence at south belridge. SPE Production & Facilities, 10(03): Hermansen, H., Thomas, L., Sylte, J., and Aasboe, B. (1997). Twenty five years of Ekofisk reservoir management. In SPE Annual Technical Conference and Exhibition. Society of Petroleum Engineers. Hu, L., Winterfeld, P., Fakcharoenphol, P., and Wu, Y. (2013). A novel fully-coupled flow and geomechanics model in enhanced geothermal reservoirs. Journal of Petroleum Science and Engineering, 107:1 11. Iskhakov, R. (2013). High-Resolution Numerical Simulation of CO 2 Sequestration in Saline Aquifers. Master s thesis. Stanford University. Jaeger, J. C., Cook, N. G., and Zimmerman, R. (2009). Fundamentals of Rock Mechanics. John Wiley & Sons. Jenny, P., Lee, S., and Tchelepi, H. (2003). Multi-scale finite-volume method for elliptic problems in subsurface flow simulation. Journal of Computational Physics, 187(1):47 67.

139 BIBLIOGRAPHY 122 Jenny, P., Lee, S. H., and Tchelepi, H. (2005). Adaptive multiscale finite volume method for multi-phase flow and transport. Multiscale Modeling & Simulation, 3: Jenny, P., Lee, S. H., and Tchelepi, H. (2006). Adaptive fully implicit multi-scale finitevolume method for multi-phase flow and transport in heterogeneous porous media. Journal of Computational Physics, 127(2): Jha, B. and Juanes, R. (2007). A locally conservative finite element framework for the simulation of coupled flow and reservoir geomechanics. Acta Geotechnica, 2: Jiang, Y. (2007). Techniques for Modeling Complex Reservoirs and Advanced Wells. PhD thesis, Stanford University. Jiang, Y. and Tchelepi, H. A. (2009). Scalable multistage linear solver for coupled systems of multisegment wells and unstructured reservoir models. In SPE Reservoir Simulation Symposium. Kale, L. V., Bohm, E., Mendes, C. L., Wilmarth, T., and Zheng, G. (2007). Programming petascale applications with Charm++ and AMPI. Petascale Computing: Algorithms and Applications, 1: Kay, J., Deser, C., Phillips, A., Mai, A., Hannay, C., Strand, G., Arblaster, J., Bates, S., Danabasoglu, G., and Edwards, J. (2015). The Community Earth System Model (CESM) large ensemble project: A community resource for studying climate change in the presence of internal climate variability. Bulletin of the American Meteorological Society, 96(8): Keilegavlen, E. and Nordbotten, J. M. (2017). Finite volume methods for elasticity with weak symmetry. International Journal for Numerical Methods in Engineering. Published online in Wiley Online Library. Killough, J. (1995). Ninth SPE comparative solution project: a reexamination of black-oil simulation. In SPE Reservoir Simulation Symposium.

140 BIBLIOGRAPHY 123 Killough, J. and Wheeler, M. (1987). Parallel iterative linear equation solvers: An investigation of domain decomposition algorithms for reservoir simulation. In SPE Reservoir Simulation Symposium. Kim, J. (2010). Sequential Methods for Coupled Geomechanics and Multiphase Flow. PhD thesis, Stanford University. Kim, J. (2015). Unconditionally stable sequential schemes for thermoporomechanics: Undrained-adiabatic and extended fixed-stress splits. In SPE Reservoir Simulation Symposium. Kim, J., Tchelepi, H., and Juanes, R. (2013). Rigorous coupling of geomechanics and multiphase flow with strong capillarity. SPE Journal, 18(06): Kim, J., Tchelepi, H. A., and Juanes, R. (2011). Stability and convergence of sequential methods for coupled flow and geomechanics: Fixed-stress and fixed-strain splits. Computer Methods in Applied Mechanics and Engineering, 200(1316): Klevtsov, S., Castelletto, N., White, J., and Tchelepi, H. (2016). Block-preconditioned Krylov methods for coupled multiphase reservoir flow and geomechanics. In ECMOR XIV-15th European Conference on the Mathematics of Oil Recovery. Kolditz, O., Bauer, S., Bilke, L., Bottcher, N., Delfs, J., Fischer, T., Gorke, U., Kalbacher, T., Kosakowski, G., Mcdermott, C., Park, C., Radu, F., Rink, K., Shao, H., Shao, H., Sun, F., Sun, Y., Singh, A., Taron, J., Walther, M., Wang, W., Watanabe, N., Wu, Y., Xie, M., Xu, W., and Zehner, B. (2012). OpenGeoSys: an open-source initiative for numerical simulation of thermo-hydro-mechanical/chemical (THM/C) processes in porous media. Environmental Earth Sciences, 67(2): Kozlova, A., Li, Z., Natvig, J. R., Watanabe, S., Zhou, Y., Bratvedt, K., and Lee, S. H. (2015). A real-field multiscale black-oil reservoir simulator. In SPE Reservoir Simulation Symposium. Larson, J., Jacob, R., and Ong, E. (2005). The model coupling toolkit: a new Fortran90 toolkit for building multiphysics parallel coupled models. The International Journal of High Performance Computing Applications, 19(3):

141 BIBLIOGRAPHY 124 Le Potier, C. (2005). Schéma volumes finis monotone pour des opérateurs de diffusion fortement anisotropes sur des maillages de triangles non structurés. Comptes Rendus Mathematique, 341(12): Li, H. and Durlofsky, L. J. (2016). Local global upscaling for compositional subsurface flow simulation. Transport in Porous Media, 111(3): Li, P. and Chalaturnyk, R. (2007). Gas-over-bitumen geometry and its SAGD performance analysis with coupled reservoir geomechanical simulation. Journal of Canadian Petroleum Technology, 46(1): Lie, K.-A. (2014). An introduction to reservoir simulation using matlab: user guide for the Matlab Reservoir Simulation Toolbox (MRST). SINTEF ICT. Lim, K.-T. (1995). A new approach for residual and Jacobian arrays construction in reservoir simulators. SPE Computer Applications, 7(04): Lubick, N. (2002). Modeling complex, multiphase porous media systems. SIAM News, 5(3). Lunati, I. and Jenny, P. (2006). Multiscale finite-volume method for compressible multiphase flow in porous media. Journal of Computational Physics, 216(2): MacDonald, R. and Coats, K. (1970). Methods for numerical simulation of water and gas coning. SPE Journal, 10(04): Maes, J., Moncorgé, A., and Tchelepi, H. (2013). Thermal adaptive implicit method: Time step selection. Journal of Petroleum Science and Engineering, 106: Maes, J., Muggeridge, A., Jackson, M., Quintard, M., and Lapene, A. (2016). Modelling insitu upgrading of heavy oil using operator splitting method. Computational Geosciences, 20(3): Mainguy, M. and Longuemare, P. (2002). Coupling fluid flow and rock mechanics: formulations of the partial coupling between reservoir and geomechanical simulators. Oil & Gas Science and Technology, 57(4):

142 BIBLIOGRAPHY 125 Mandel, J. (1953). Consolidation des sols (étude mathématique). Geotechnique, 3(7): Markert, B., Heider, Y., and Ehlers, W. (2010). Comparison of monolithic and splitting solution schemes for dynamic porous media problems. International Journal for Numerical Methods in Engineering, 82(11): Matringe, S. (2008). Mixed Finite Element Methods for Discretization and Streamline Tracing. PhD thesis, Stanford University. McKenna, F. (2011). OpenSees: a framework for earthquake engineering simulation. Computing in Science & Engineering, 13(4): Mikelic, A. and Wheeler, M. (2013). Convergence of iterative coupling for coupled flow and geomechanics. Computational Geosciences, 17(3): Minkoff, S., Stone, C., Bryant, S., Peszynska, M., and Wheeler, M. (2003a). Coupled fluid flow and geomechanical deformation modeling. Journal of Petroleum Science and Engineering, 38(1-2): Minkoff, S. E., Stone, C., Bryant, S., Peszynska, M., and Wheeler, M. F. (2003b). Coupled fluid flow and geomechanical deformation modeling. Journal of Petroleum Science and Engineering, 38(1-2): Moncorgé, A., Tchelepi, H. A., and Jenny, P. (2017). Modified sequential fully implicit scheme for compositional flow simulation. Journal of Computational Physics, 337: Moridis, G. J., Kim, J., Reagan, M. T., and Kim, S.-J. (2013). Feasibility of gas production from a gas hydrate accumulation at the UBGH2-6 site of the Ulleung basin in the Korean East Sea. Journal of Petroleum Science and Engineering, 108: Myner, O. (2016). Next Generation Multiscale Methods for Reservoir Simulation. PhD thesis, Norwegian University of Science and Technology. Nicholls, R. J. (1995). Coastal megacities and climate change. GeoJournal, 37(3):

143 BIBLIOGRAPHY 126 Nikitin, K., Terekhov, K., and Vassilevski, Y. (2014). A monotone nonlinear finite volume method for diffusion equations and multiphase flows. Computational Geosciences, 18(3): Nikolaevskij, V. N. (1990). Mechanics of Porous and Fractured Media, volume 8. World Scientific. Nogaret, C. (1996). Implementation of a Network-Based Approach in an Object Oriented Reservoir Simulator. Master s thesis. Stanford University. Notz, P. K., Pawlowski, R. P., and Sutherland, J. C. (2012). Graph-based software design for managing complexity and enabling concurrency in multiphysics PDE software. ACM Transactions on Mathematical Software (TOMS), 39(1):1 21. Ottosen, N. and Ristinmaa, M. (2005). The Mechanics of Constitutive Modeling. Elsevier. Pan, H. and Tchelepi, H. A. (2010). Reduced variable method for general-purpose compositional reservoir simulation. In International Oil and Gas Conference and Exhibition in China. Pan, H. and Tchelepi, H. A. (2011). Compositional flow simulation using reduced-variables and stability-analysis bypassing. In SPE Reservoir Simulation Symposium. Park, K. (1983). Stabilization of partitioned solution procedure for pore fluid-soil interaction analysis. International Journal for Numerical Methods in Engineering, 19(11): Prevost, J. H. (1997). Partitioned solution procedure for simultaneous integration of coupled-field problems. Communications in Numerical Methods in Engineering, 13(4): Rannou, G., Voskov, D., and Tchelepi, H. (2013). Tie-line-based K-value method for compositional simulation. SPE Journal, 18(06): Redler, R., Valcke, S., and Ritzdorf, H. (2010). OASIS4-a coupling software for next generation earth system modeling. Geoscientific Model Development, 3(1):

144 BIBLIOGRAPHY 127 Rin, R., Tomin, P., Garipov, T., Voskov, D., and Tchelepi, H. (2017). General implicit coupling framework for multi-physics problems. In SPE Reservoir Simulation Conference. Russell, T. (1989). Stability analysis and switching criteria for adaptive implicit methods based on the CFL condition. In SPE Reservoir Simulation Symposium. Rutqvist, J. (2011). Status of the TOUGH-FLAC simulator and recent applications related to coupled fluid flow and crustal deformations. Computers & Geosciences, 37(6): Rutqvist, J. and Moridis, G. J. (2009). Numerical studies on the geomechanical stability of hydrate-bearing sediments. SPE Journal, 14(2): Rutqvist, J. and Tsang, C.-F. (2003). Analysis of thermal hydrologic mechanical behavior near an emplacement drift at Yucca Mountain. Journal of Contaminant Hydrology, 62: Samier, P., Onaisi, A., and de Gennaro, S. (2008). A practical iterative scheme for coupling geomechanics with reservoir simulation. SPE Reservoir Evaluation and Engineering, 11(5): Settari, A. and Walters, D. (2001). Advances in coupled geomechanical and reservoir modeling with applications to reservoir compaction. SPE Journal, 6(03): Settari, A., Walters, D. A., Stright, D. H. J., and Aziz, K. (2008). Numerical techniques used for predicting subsidence due to gas extraction in the North Adriatic Sea. Petroleum Science and Technology, 26(10-11): Sheldon, J. W., Zondek, B., and Cardwell, W. T. (1959). One-dimensional, incompressible, non-capillary, two-phase fluid flow in a porous medium. Trans. SPE of AIME, 216: Shiriaev, D. and Griewank, A. (1996). ADOL-F: Automatic differentiation of Fortran codes. Computational Differentiation: Techniques, Applications, and Tools, pages

145 BIBLIOGRAPHY 128 Slattery, S., Wilson, P., and Pawlowski, R. (2013). The data transfer kit: a geometric rendezvous-based tool for multiphysics data transfer. In International conference on mathematics & computational methods applied to nuclear science & engineering (M&C 2013). Spillete, A., Hillestad, J., and Stone, H. (1973). A high-stability sequential solution approach to reservoir simulation. In 48th Annual Fall Meeting of SPE of AIME. Stanford Geothermal Program (1980). Proceedings of the special panel on geothermal model study, report SGP-TR-42. In Sixth Annual Workshop on Geothermal Reservoir Engineering. Stone, H. and Garder, A. (1961). Analysis of gas-cap or dissolved-gas drive reservoirs. SPE Journal, 1: Terekhov, K. M., Mallison, B. T., and Tchelepi, H. A. (2017). Cell-centered nonlinear finite-volume methods for the heterogeneous anisotropic diffusion problem. Journal of Computational Physics, 330: Thomas, G. and Thurnau, D. (1983). Reservoir simulation using an adaptive implicit method. SPE Journal, 23(05): Thomas, L., Chin, L., Pierson, R., and Sylte, J. (2003). Coupled geomechanics and reservoir simulation. SPE Journal, 8: Tomin, P. and Lunati, I. (2013). Hybrid multiscale finite volume method for two-phase flow in porous media. Journal of Computational Physics, 250: Tomin, P. and Lunati, I. (2015). Local global splitting for spatiotemporal-adaptive multiscale methods. Journal of Computational Physics, 280: Tomin, P. and Lunati, I. (2016). Spatiotemporal adaptive multiphysics simulations of drainage-imbibition cycles. Computational Geosciences, 20(3): Tran, D., Settari, A., and Nghiem, L. (2004). New iterative coupling between a reservoir simulator and a geomechanics module. SPE Journal, 9(3):

146 BIBLIOGRAPHY 129 Trangenstein, J. A. (1989). Analysis of a model and sequential numerical method for thermal reservoir simulation. In ECMOR I-1st European Conference on the Mathematics of Oil Recovery. van Odyck, D., Bell, J., Monmont, F., and Nikiforakis, N. (2009). The mathematical structure of multiphase thermal models of flow in porous media. Proceedings of the Royal Society of London A: Mathematical, Physical and Engineering Sciences, 465(2102): Verma, A. K., Gautam, P., Singh, T. N., and Bajpai, R. K. (2015). Numerical simulation of high level radioactive waste for disposal in deep underground tunnel. Engineering Geology for Society and Territory - Volume 1: Climate Change and Engineering Geology, pages Verma, S. (1996). Flexible Grids for Reservoir Simulation. PhD thesis, Stanford University. Voskov, D. and Tchelepi, H. (2012). Comparison of nonlinear formulations for two-phase multi-component EoS based simulation. Journal of Petroleum Science and Engineering, 82: Voskov, D. V. and Tchelepi, H. A. (2009a). Compositional space parameterization: Multicontact miscible displacements and extension to multiple phases. SPE Journal, 14(03): Voskov, D. V. and Tchelepi, H. A. (2009b). Compositional space parameterization: theory and application for immiscible displacements. SPE Journal, 14(03): Voskov, D. V. and Volkov, O. (2013). Advanced strategies of forward simulation for adjointbased optimization. In SPE Reservoir Simulation Symposium. Watts, J. W. (1986). A compositional formulation of the pressure and saturation equations. SPE Reservoir Engineering, 1: Wheeler, M. F., Arbogast, T., Bryant, S., Eaton, J., Lu, Q., Peszynska, M., and Yotov, I. (1999). A parallel multiblock/multidomain approach for reservoir simulation. In SPE Reservoir Simulation Symposium.

147 BIBLIOGRAPHY 130 Wheeler, M. F. and Gai, X. (2007). Iteratively coupled mixed and Galerkin finite element methods for poro-elasticity. Numerical Methods for Partial Differential Equations, 23(4): White, J. and Borja, R. (2011). Block-preconditioned Newton Krylov solvers for fully coupled flow and geomechanics. Computational Geosciences, 15(4): White, J. A., Castelletto, N., and Tchelepi, H. A. (2016). Block-partitioned solvers for coupled poromechanics: A unified framework. Computer Methods in Applied Mechanics and Engineering, 303: White, M. and Oostrom, M. (2006). STOMP: Subsurface Transport Over Multiple Phases. Version 4.0, User s Guide. Richland: Pacific Northwest National Laboratory. White, W. A. and Morton, R. A. (1997). Wetland losses related to fault movement and hydrocarbon production, southeastern Texas coast. Journal of Coastal Research, 13: Wong, T. (1988). Compositional and Black Oil Modeling with Emphasis on the Volume Balance Method. Master s thesis. Stanford University. Wong, Z. (2015). A Geothermal Reservoir Simulator in AD-GPRS. Master s thesis. Stanford University. Wong, Z., Horne, R., and Voskov, D. (2016). Comparison of nonlinear formulations for geothermal reservoir simulations. In 41st workshop on Geothermal Reservoir Engineering. Wong, Z., Rin, R., Tchelepi, H. A., and Horne, R. (2017). Comparison of a fully implicit and sequential implicit formulation for geothermal reservoir simulations. In 42nd workshop on Geothermal Reservoir Engineering. Wronkiewicz, D. J., Bates, J. K., Wolf, S. F., and Buck, E. C. (1996). Ten-year results from unsaturated drip tests with UO 2 at 90 C: implications for the corrosion of spent nuclear fuel. Journal of Nuclear Materials, 238(1):78 95.

148 BIBLIOGRAPHY 131 Yang, D., Moridis, G. J., and Blasingame, T. A. (2014). A fully coupled multiphase flow and geomechanics solver for highly heterogeneous porous media. Journal of Computational and Applied Mathematics, 270: Young, L. and Stephenson, R. (1983). A generalized compositional approach for reservoir simulation. SPE Journal, 23(05): Younis, R. (2011). Modern Advances in Software and Solution Algorithms for Reservoir Simulation. PhD thesis, Stanford University. Zaydullin, R. (2015). Compositional Space Parameterization Methods for Thermal- Compositional Simulation. PhD thesis, Stanford University. Zaydullin, R., Voskov, D., James, S., Henley, H., and Lucia, A. (2014). Fully compositional and thermal reservoir simulation. Computers and Chemical Engineering, 63: Zaydullin, R., Voskov, D., and Tchelepi, H. (2013). Nonlinear formulation based on an equation-of-state free method for compositional flow simulation. SPE Journal, 18(2): Zhou, H. (2010). Algebraic Multiscale Finite-Volume Methods for Reservoir Simulation. PhD thesis, Stanford University. Zhou, Y. (2009). Multistage Preconditioner for Well Groups and Automatic Differentiation for Next Generation GPRS. Master s thesis. Stanford University. Zhou, Y. (2012). Parallel General-Purpose Reservoir Simulation with Coupled Reservoir Models and Multi-Segment Wells. PhD thesis, Stanford University. Zhou, Y., Jiang, Y., and Tchelepi, H. (2013). A scalable multistage linear solver for reservoir models with multisegment wells. Computational Geosciences, 17(2): Zienkiewicz, O., Paul, D., and Chan, A. (1988). Unconditionally stable staggered solution procedure for soil-pore fluid interaction problems. International Journal for Numerical Methods in Engineering, 26(5):

149 BIBLIOGRAPHY 132 Zienkiewicz, O. C., Chan, A., Pastor, M., Schrefler, B., and Shiomi, T. (1999). Computational Geomechanics. Wiley Chichester.

150 Appendix A Programming Model of AD-GPRS In this appendix, the programming model of AD-GPRS is described. The primary design objective of the GENIC framework is to create a platform where different coupling strategies for the simulation of complex multi-physical processes can be employed with a great flexibility. We adopt a modular object-oriented design. The extensibility is achieved through the usage of a subset concept presented in Chapter 2. All of the code is written in standard C++11. Presently, AD-GPRS code contains over 94k lines and over 400 files. The GENIC AD-GPRS is very different from the previous version of AD-GPRS which was built as a compositional simulator and later extended to some multi-physics functionality. Instead, GENIC AD-GPRS was developed as multi-physics simulator from the beginning. Many classes are heavily refactored or completely rewritten to meet a general multiphysics paradigm. For example, the functionality of allstatus, totalblocks, and full set is fully incorporated into subsets, and many redundant structures are removed. Moreover, the core classes of the previous version such as Reservoir and Facilities are evolved into Flow- Problem/ThermalProblem and Well, which are on top level abstracted by the general class of SubProblem. Another two new key classes to mention are Subset and CoupledProperties. This entails significant changes in the whole simulator. The logic of input-output, the configuration of parameters, convergence criteria, and discretization have been also revised. We highly recommend to study in detail Two-Phase Prototype (see Appendix B) prior to any code development in AD-GPRS. 133

151 APPENDIX A. PROGRAMMING MODEL OF AD-GPRS 134 A.1 The Structure of AD-GPRS This section is organized similarly to Appendix A in Zhou (2012). The structure of AD- GPRS is discussed level by level, from top to bottom, followed by a flow sequence of the simulation framework. Here we use italic shape to denote class names, slanted shape to denote variable names, and boldface to denote function names. We explain the structure of AD-GPRS with multi-layer class diagrams. Figure A.1 shows the overall structure of the entire simulator. Simulator is the topmost class that is instantiated in the main function and is an entry point for the whole simulation. We identify four groups of class-members of Simulator: 1. The first group is the core of the simulator, and it contains adx, globalstatustable, subproblemtree, and vsubproblems. The global variable set (adx) stores all the simulation variables with both values and gradients, as well as the global backup data, local status tables, and mapping operators. The global status table (globalstatustable) specifies which sub-problems are active based on mapping operators and is used to define the sub-problems calculation order. The sub-problem tree (sub- ProblemTree) represents tree structure of a given coupling strategy based on the COUPLING keyword. Note that subproblemtree is accessible indirectly via simdata. The vector of sub-problems (vsubproblems) is a list of all unique sub-problems for Figure A.1: Overall structure of the entire simulator.

152 APPENDIX A. PROGRAMMING MODEL OF AD-GPRS 135 a given coupling. 2. The second group represents additional members needed for coupling such as vcouplingsubproblems and the list of applied coupled properties (vcoupledproperties). The set of sub-problems with coupling terms stored in vcouplingsubproblems. This includes wells and any other sub-problem which contributes to other physics despite being non-active at the moment. The first and second group complete the core of the GENIC framework as discussed in Chapter The third group specifies the subset of sub-problems (vactivesubproblems) which are active for a given sub-problem tree node (see Algorithm 1 for details), corresponding linear solver (pactivelinearsystem), Jacobian, and residual. The later two represented by ADVector residual (see ADETL manual for details). 4. The last group is responsible for input-output (pio) and data infrastructure (sim- Data). simdata enables change of parameters with time such as well controls, time stepping and some parameters for physics. The input-output class IO is general for every subset newly introduced sub-problems and subsets will automatically appear in the generated output. Figure A.2 shows the structure of SubProblemTree and its relationship with the Simulator. SubProblemTree builds the global status table which is used by Simulator. The elements of sub-problem tree represented by SubProblemNode. Each sub-problem node contains the list of children nodes (children) again of SubProblemNode type, and this list Figure A.2: Structure of SubProblemTree class.

153 APPENDIX A. PROGRAMMING MODEL OF AD-GPRS 136 is empty when the corresponding node is a terminal node. Each terminal node has a corresponding global status in the global status table (globalstatus), current and maximum number of iterations for a given node for SEQ coupling (niter and nitermax). In addition, SubProblemNode stores the list of given problems in the node (problems), their linear solver (plinearsystem), and linear solver parameters (linearsolverparams). Sub-problem is another core concept of GENIC AD-GPRS. We represent any physical process as a sub-problem which is implemented as a derived class of the abstract Sub- Problem class. Figure A.3 shows the structure of SubProblem class. Every sub-problem has name and type; the type is used to properly link the subsets between different subproblems (function bindsubsets) and to apply a linear solver. We can add a new subset to a given sub-problem using the function addsubset. It allocates memory in the global variable set (adx), and adds the pointer to the new subset into vsubsets. Notice, while every sub-problem calls the constructor for each subset, its destructor is called in the destructor of the global variable set (adx); therefore, calling the destruction of a subset in the sub-problem would lead to a memory corruption. We write discretized governing equations into the global residual (residual) with the function addtoresidual. The convergence is checked with an object of type ResidualChecker, presidualchecker is the pointer to that object. All possible mappers for a given sub-problem are identified at the input stage and stored in the corresponding vector of mappers mmappers. The set setmappedsubsets contains the list of subsets affected by mappers. The change of global status leads to the change of the applied mapper (changestatusfrommapper) which is specified via the parameter mapperstatus. Figure A.3: Structure of SubProblem class.

154 APPENDIX A. PROGRAMMING MODEL OF AD-GPRS 137 Presently, AD-GPRS supports the following sub-problems: FlowProblem compositional flow and transport, ThermalProblem energy balance, GeomechProblem momentum balance (geomechanics), Well abstract parent class for standard (StandardWell) and multi-segment (MSWell) wells, ReactionsProblem chemical reactions (currently only kinetic reactions), TimeOfFlightProblem time of flight equation, TracerProblem tracer transport equation, ResistivityProblem Poisson equation for electric potential field. The governing equations for FlowProblem, ThermalProblem, GeomechProblem, MSWell, and StandardWell are given in Chapter 2. The governing equations for other sub-problems are given in the AD-GPRS keywords manual. Table A.1 represents owned and binded subsets for each sub-problem. For example, binded subsets for StandardWell and MSWell classes are identical to the parent class Well. Both WellFlowSubsetPerf and WellFlowSubsetSC are of type WellFlowSubset and represent properties and unknowns defined at perforations and standard conditions. Similarly, WellThermalSubsetPerf and WellThermalSubsetSC are of type WellThermalSubset. Segments subsets for MSWells SegFlowSubset and SegThermalSubset are of types Flow- CellSubset and ThermalCellSubset correspondingly. Figure A.4 shows the class diagram for all available subsets. The topmost class AD IndependentSubSet defining all methods needed for a subset belongs to ADETL library. Derivative classes specify properties and unknowns; no additional methods are required to utilize the subset. The details of subset methods and members are given in the ADETL manual. Example of the subset creation and usage can be found in Appendix A.2. The domain of active elements for a given subset is defined through class Mapper (part of ADETL). The activation logic of class Mapper is different from ACTNUM (active cells)

155 APPENDIX A. PROGRAMMING MODEL OF AD-GPRS 138 Table A.1: Sub-problems, subsets and properties. Sub-problem Owned subsets Binded subsets FlowProblem FlowCellSubset ThermCellSubset FlowConnectionSubset ReactionsSubset ThermalProblem ThermCellSubset FlowCellSubset FlowConnectionSubset GeomechProblem GeomechSubset FlowCellSubset ThermalCellSubset WellFlowSubsetPerf FlowCellSubset Well WellFlowSubsetSC ThermalCellSubset WellThermalSubsetPerf TOFSubset WellThermalSubsetSC TracerSubset StandardWell: WellBhpSubset MSWell: SegFlowSubset SegThermSubset ReactionsProblem ReactionsSubset FlowCellSubset ThermalCellSubset TimeOfFlightProblem TOFSubset FlowCellSubset FlowConnectionSubset TracerProblem TracerSubset FlowCellSubset FlowConnectionSubset FlowCellSubset ResistivityProblem ResistivityCellSubset ThermalCellSubset TracerSubset concept. The later disables inactive cells prior to a simulation (and all relevant memory allocations); therefore, these cells can no longer be activated later on. In addition, ACTNUM is limited to reservoir cells only. Mapper allows activation and deactivation at any moment (time step or Newton iteration) and can be applied to any subset. The new mapper can be derived from the base class Mapper, and only requires a redefinition of the function updatemapping, which defines which cells are active and which are not. We specify the choice and order of unknowns for each sub-problem by using the Formulation class. The usage of the formulation is specific to the type of the sub-problem, unlike

156 APPENDIX A. PROGRAMMING MODEL OF AD-GPRS 139 Figure A.4: Class diagram for subsets. the previous version of AD-GPRS where all formulations are derived from the base class NonlinearFormulation. Such class design provides a higher level of flexibility, improves the code readability and reduces the memory overhead. The usage of the formulation is not mandatory for a subproblem, and not all sub-problems have a corresponding formulation. For a sub-problem that requires a formulation, there is a corresponding base class SubproblemNameFormulation, and a specific formulation for this sub-problem is derived from the base class. For example, class FlowProblem has the base formulation class FlowFormulation. Both natural (NaturalFormulation), and molar (MolarFormulation) formulations are derived classes from the FlowFormulation class. By constructing new inherited classes of a corresponding sub-problem base formulation with the same interfaces but possible different realizations, we are able to introduce new formulations. Figure A.5 is a class diagram for all available formulations. Notice, that class WellProblem does not have a formulation; instead, WellProblem uses the formulation of FlowProblem. The modeling of a wellbore without reservoir would require its own formulation. The formulations of different sub-problems vary greatly. In general, all formulations

157 APPENDIX A. PROGRAMMING MODEL OF AD-GPRS 140 Figure A.5: Class diagram for formulations. have three common functions: constructor: initialization of parameters and subsets, buildstatustable: create the table of local statuses, for example, a flow-related formulations it is a table of existing phases associated with each possible phase status, bindsubsets: link subsets used by the formulation. In addition, each sub-problem s formulation defines a set of additional functions. For example, flow formulation contains member functions such as enforceconstraints (enforce certain local constraints), changestatus (check the phase status and change it if necessary), calcfluidpropertiesincell (compute all fluid properties), makesecondaryterm (compute secondary terms such as fugacity), and makeaccumulation (compute accumulation term). The linear system of equations in class Simulator is of type LinearSystemBase. The major functions of LinearSystemBase are setup: extraction of Jacobian matrix from the AD-residual vector, solve: solve linear system, getsolution: return the pointer to the linear solution).

158 APPENDIX A. PROGRAMMING MODEL OF AD-GPRS 141 Figure A.6: Class diagram for LinearSystemBase. The structure of LinearSystemBase class in shown on Figure A.6. Presently, LinearSystemBase has only one derived class CSRLinearSystem which works with the Compressed Sparse Row (CSR) matrix format. Note that the BlockLinearSystem (available in the previous version of AD-GPRS), which works with a much more complicated matrix format, is currently not supported due to limited extensibility for multi-physics problems. The integration of the block matrix storage and manipulation is the subject of future research and development. Each CSRLinearSystem contains corresponding CSR matrix (CSR Matrix), linear solver (LibrarySolver), and its parameters (ParametersForLibrarySolvers). Generally, any solver working with CSR format can be supported. Presently, LibrarySolver includes direct linear solvers such as PARDISO and SuperLU, and iterative solvers such as HYPRE, INMOST, and SAMG. A.2 Flow Sequence The essential steps of a forward simulation are as follows: 1. Read the input (problem definition) from the given simulation deck. 2. Build the sub-problem tree and the global status table based on the input coupling strategy.

159 APPENDIX A. PROGRAMMING MODEL OF AD-GPRS Initialization (allocate memory, create objects, link subsets, and apply initial conditions). 4. While there is still a time interval that has not been simulated, do the following: (a) Update the simulation parameters and corresponding data for the current input time interval if it is not the first one. (b) While the current input time interval has not been finished, do the following: i. Calculate the timestep size. ii. Initialize the timestep: if the nonlinear solution succeeded in the last timestep, use this converged solution as the initial guess and save independent variables into a backup set; otherwise, restore from the backup set and compute all dependent variables. iii. Call the recursive function solvenode that solves the coupled multi-physics problem starting from the head of the sub-problem tree (see the details of this function in Algorithm 1). iv. If solvenode returned that the solution is accepted, report statistics for the converged timestep and dump the solution if necessary (e.g., at a report step); otherwise, update the statistics for wasted Newton and timestep iterations. v. Go to step 4b for the next timestep. (c) Go to step 4 for the next input time interval. 5. Report the overall statistics and perform post-processing if necessary. 6. Deallocate everything and end the simulation.

160 Appendix B Two-Phase Prototype In addition to multi-physics AD-GPRS GENIC presented in this work, we developed a Two- Phase Prototype. The Prototype is based on the key principles of the GENIC framework and serves as an introductory step to the multi-physics AD-GPRS GENIC simulator. It supports two different models: a water-oil (dead-oil) system with gravity and an oil-gas system without gravity. Both are isothermal problems with single-perforation wells. The data for physical properties and the reservoir was taken from Stanford Reservoir Simulation class (ERE 223). While the Prototype has the simplified physics, it can be potentially extended to handle multi-physics problems. The usage of the two key components of GENIC framework, namely sub-problems and subsets, is demonstrated in the Prototype, while the full framework with the sub-problem tree, global status table, coupling terms, and properties can be found in the full AD-GPRS GENIC. Only one coupling strategy Ψ = FIM[FLOW,WELLS] is available in the Prototype. The entire Prototype source code is located in the master branch of AD-GPRS simulator in ExtraModules/ProtoTwoPhase directory. There are four types of sub-problems in the Prototype: FlowProblem for dead-oil system, FlowGasProblem for oil-gas flow and transport, WellProblem for a well, 143

161 APPENDIX B. TWO-PHASE PROTOTYPE 144 Figure B.1: Class diagram for subproblems and their interaction with subsets in Two-Phase Prototype. Figure B.2: Class diagram for subsets in Two-Phase Prototype. WellGasProblem specifically for gas producers. For simplicity, we specify dead-oil and oil-gas formulations directly in FlowProblem and FlowGasProblem classes correspondingly. We use the three subsets: FlowCellSubset for unknowns and properties defined at reservoir cell centers FlowFaceSubset for phase fluxes, WellCellSubset for bottom hole pressure and rates values for each well. Figure B.1 represents the class diagram for subproblems and their interactions with corresponding subsets, while Figure B.2 depicts the class diagram for subsets. In the following, we go through the building block for a simple flow and transport simulator. For brevity, we limit our description to dead-oil formulation only.