Software Science: How Far Could Mathematics and Rigor Take Us? New Proposal Dr. Lan Lin (llin4@bsu.edu) Ball State University November 8, 2017 1
Some well-known software failures } Compiled by Gang Tan, Penn State University Knight Capital s $40 million loss (8/1/2012) Microsoft Zune s New Year crash (12/312008) Air traffic control system in LA airport (9/14/2004) Northeast blackout (8/14/2003) NASA Mars climate orbiter (9/23/1999) Denver airport baggage-handling system (11/1993 6/1994) Therac-25 (6/1985 1/1987) USS Yorktown incident (9/1997) Ariane 5 Explosion (9/1997) } Cost of correcting an error $1 in requirements analysis $2 in preliminary design $4 in detailed design $8 in code and test $16 in integration test $32 in the field 2
Searching for software silver bullets } Industry focus } Short development cycles } No failures in the field } Our goal: the economical production of high quality software } Societal need: safe software, dependable software, designed software } Designed systems vs. accidental systems } Correct software by design } Choices are limited. 3
Software silver bullets } Precise specifications } Code generation from specifications } Automated program verification } Automated statistical certification } Design for testability } Design for conceptual integrity } Software development as an engineering process with mathematical foundations 4
Goals of the proposed research } To touch the surface of the fundamental problem of software engineering } To seek an economical means to introduce and apply more rigorous methods } To define a systematic process with a workflow and tool chain for such integration 5
Mathematical nature of SW development } Every software program implements the mapping rule of a mathematical function [Mills 75]. } Sequence-based software specification [Prowell and Poore] } High-quality software design in essence relies on conceptual integrity [Brooks 95, 10]. } Linear software models and the modularity matrix [Exman] } Software testing is a statistical experiment; development should be placed under statistical quality control [Mills 87]. } Statistical testing based on a Markov chain usage model [Whittaker and Poore] 6
Sequence-based software specification } Systematically derive mathematically rigorous specification from requirements Source: Poore et al., From requirements to automated statistical testing, 2007 Source: Prowell and Swain, Sequence-based specification of critical software systems, 2004 7
Usage-based statistical testing } Comprehensive application of statistical science to the testing of software } Demonstrate highly likely paths do not fail Population (All Uses) What to test: a statistically appropriate sample Sample (Tests) How much to test: a statistically valid inference Source: Swain and UTK SQRL, Model-based statistical testing, 2013 8
Combining them together Source: Poore et al., From requirements to automated statistical testing, 2007 Source: Poore and Eschbach, Sequencebased specification and statistical testing for embedded systems, 2007 9
Specification to implementation } Require a high level architecture } Define an implementation for } Stimulus gathering } Response generation } State data items } State box entries requirements } Map state data to architectural components } Map state box entry implementation to architecture S* R BB: S* --> R S x T T x R S x T T x R SB: S x T --> T x R SBS SBS SBS + architecture CB: S x T --> T x R Source: Swain and UTK SQRL, Sequence-based specification, 2013 10
Strengthening the linkage } Use Exman s linear software models and the modularity matrix to check the design after state box clear box refinement } A formal theory of modularity based on plain linear algebra } Enable formal treatment of informal design notions and principles } Make conceptual integrity quantitative calculations Source: Exman, Linear software models: An algebraic theory of software composition, keynote at SEKE 2016 11
Strengthening the linkage } Incorporate combinatorial testing techniques to achieve good combinatorial coverage of the generated test suite } One more sampling option enabling high fault detection rate besides reliability estimation } Sequence covering arrays Source: Rick Kuhn (NIST), Combinatorial methods in software testing, keynote at SCSE 2015 12
The proposed workflow } Each step based on mathematically-based methods that have been proven to be sound and promising Sequence-Based Specification Requirements Delivered Software Software Certification statistics function + automaton linear algebra Formal Specification Testing Combinatorial Testing Statistical Testing combinatorics statistics Linear Software Models and Modularity Matrix Modular Design Code/ Implementation theory practice theory 13
Deliverables } A theory that connects rigorous specification with conceptual software design } A theory that augments statistical testing with combinatorial testing } A process that flows from original requirements to modular design to testing all based on rigorous methods } A real-world case study to demonstrate its feasibility and practicality } Publications in suitable venues 14
15