Distributed Knowledge and Information Flow Security

Size: px
Start display at page:

Download "Distributed Knowledge and Information Flow Security"

Transcription

1 Distributed Knowledge and Information Flow Security School of Computer Science and Engineering, University of New South Wales Sydney, Australia meyden September 24, 2008

2 Overiew Starting Point: An intuition concerning distributed knowledge in causally structured systems: An agent s knowledge about other agents must be distributed knowledge to the other agents that can causally affect it. (ESORICS-07) An accepted definition of causality from the security literature does not satisfy the intuition. New definitions of causality that better fit the security literature theory.

3 (KR-08) Exactly how well do the new definitions support the intuition? Outcomes: New definitions of distributed knowledge A refined understanding of information about other agents. an exact characterization of the new definitions of causality in terms of distributed knowledge of information about other agents.

4 Background Multi-level secure systems for military/intelligence applications: Top Secret, Secret, Unclassified Concern with Trojan Horse attacks Covert Channels: e.g. locks on shared resources, timing of process execution. definitions of security wth an information theoretic flavour.

5 Noninterference policies (Goguen and Meseguer 1982) Let Agents be a set of security domains/components/agents. A noninterference policy is a reflexive relation Agents Agents u v means actions of u are permitted to interfere with v, or actions of u are permitted to have effects observable to v, or information is permitted to flow from u to v

6 Policy Examples Downgrading: Channel Control: H D L (header) BYPASS RED BLACK (body) CRYPTO

7 Deterministic System Model Machines have the form M = S, s 0, Actions,agent,step, O where S is a set of states,

8 Deterministic System Model Machines have the form M = S, s 0, Actions,agent,step, O where S is a set of states, s 0 S is the initial state,

9 Deterministic System Model Machines have the form M = S, s 0, Actions,agent,step, O where S is a set of states, s 0 S is the initial state, Actions is a set of actions,

10 Deterministic System Model Machines have the form M = S, s 0, Actions,agent,step, O where S is a set of states, s 0 S is the initial state, Actions is a set of actions, agent : Actions Agents associates each action to an agent in Agents,

11 Deterministic System Model Machines have the form M = S, s 0, Actions,agent,step, O where S is a set of states, s 0 S is the initial state, Actions is a set of actions, agent : Actions Agents associates each action to an agent in Agents, step : S Actions S is a deterministic transition function, and

12 Deterministic System Model Machines have the form M = S, s 0, Actions,agent,step, O where S is a set of states, s 0 S is the initial state, Actions is a set of actions, agent : Actions Agents associates each action to an agent in Agents, step : S Actions S is a deterministic transition function, and O u : S Obs represents each agent u s observations

13 Deterministic System Model Machines have the form M = S, s 0, Actions,agent,step, O where S is a set of states, s 0 S is the initial state, Actions is a set of actions, agent : Actions Agents associates each action to an agent in Agents, step : S Actions S is a deterministic transition function, and O u : S Obs represents each agent u s observations Notation: s α for the state reached by performing the sequence of actions α Actions from state s.

14 Haigh and Young s Purge Given a sequence of actions α = a 1...a n Actions and agent u, the intransitive purge ipurge u (α) is the subsequence of all actions a i such that there exists with i = i 1 < i 2 <... < i k agent(a i1 ) agent(a i2 )... agent(a ik ) u

15 Example: A B D E C E a b a c a d a c b ipurge E a b c d

16 Haigh and Young s definition: IP-security A system M is IP-secure with respect to a (possibly intransitive) policy if for all u D and all sequences α, α A with ipurge u (α) = ipurge u (α ), we have O u (s 0 α) = O u (s 0 α ).

17 Knowledge in Asynchronous Systems Define the view of agent u with respect to a sequence α Actions to be the sequence view u (α) of all actions and observations of that agent, with stuttering of observations eliminated (to model asynchrony). E.g. if α = hhlh generates (Low observations only): O 1 ho 1 ho 1 lo 2 ho 2 then view Low (α) = O 1 lo 2

18 Let π : Prop P(Actions ), α Actions M, π, α = p if α π(p) M, π, α = K u φ if M, π, α = p for all α Actions with view u (α) = view u (α ) M, π, α = D G φ if M, π, α = p for all α Actions with u G view u(α) = view u (α )

19 The Intuition Given a policy and an agent u, define I u = {v v u and v u} the set of agents that may causally affect/interfere with u.

20 The Intuition Given a policy and an agent u, define I u = {v v u and v u} the set of agents that may causally affect/interfere with u. Intuition: if M is secure with respect to then for all agents u, if π interprets p as being about agents other than u, then M, π = K u p D Iu p

21 π interprets p as about agents other than u if For α, α Actions, if α (Agents \ {u}) = α (Agents \ {u}) then α π(p) iff α π(p). where α G is the subsequence of α of all actions a such that agent(a) G.

22 IP-security does not satisfy the intuition (van der Meyden ESORICS 2007) H D 1 1 L H 2 D 2 Define the system M with O u (s 0 α) = ipurge u (α) This is obviously IP-Secure.

23 Let α 1 = h 1 h 2 d 1 d 2 Then O L (α 1 ) = ipurge u (α 1 ) = α 1

24 Let α 1 = h 1 h 2 d 1 d 2 Then O L (α 1 ) = ipurge u (α 1 ) = α 1 So, M, π, α 1 = K L ( h 1 before h 2 )

25 Let α 1 = h 1 h 2 d 1 d 2 Then O L (α 1 ) = ipurge u (α 1 ) = α 1 So, M, π, α 1 = K L ( h 1 before h 2 ) But, if α 2 = h 2 h 1 d 1 d 2 then view D1 (α 1 ) = view D1 (α 2 ) and view D2 (α 1 ) = view D2 (α 2 )

26 Let α 1 = h 1 h 2 d 1 d 2 Then O L (α 1 ) = ipurge u (α 1 ) = α 1 So, M, π, α 1 = K L ( h 1 before h 2 ) But, if α 2 = h 2 h 1 d 1 d 2 then view D1 (α 1 ) = view D1 (α 2 ) and view D2 (α 1 ) = view D2 (α 2 ) M, π, α 1 = K L ( h 1 before h 2 ) D {D1,D 2 }( h 1 before h 2 )

27 An alternative definition - TA security Given a policy, define, for each agent u D, the function ta u, with domain Actions, inductively by ta u (ǫ) = ǫ, and, for α Actions and a Actions, { tau (α) if agent(a) u ta u (αa) = (ta u (α),ta agent(a) (α), a) if agent(a) u

28 An alternative definition - TA security Given a policy, define, for each agent u D, the function ta u, with domain Actions, inductively by ta u (ǫ) = ǫ, and, for α Actions and a Actions, { tau (α) if agent(a) u ta u (αa) = (ta u (α),ta agent(a) (α), a) if agent(a) u Define a system M to be TA-secure with respect to a policy if for all agents u and all α, α Actions such that ta u (α) = ta u (α ), we have O u (s 0 α) = O u (s 0 α ).

29 TA-security gives the right answer in the example: insecure. Theorem: TA-security implies IP-security

30 Theorem: The following are equivalent M is TA-secure with respect to there exist equivalence relations u for u D on the states of uf(m) satisfying Rushby s unwinding conditions for intransitive noninterference uf(m) admits a weak access control interpretation compatible with.

31 Unfolding a system: b a,b a x y uf x y x y y a b a x y a b b x y a b x y a b y y a b y y a b... uf(m) is bisimilar to M (in the expected sense)

32 Rushby s Unwinding Conditions for IP-security Suppose we have for each agent u an equivalence relation u on the states of M. Theorem (Rushby): If these conditions are satisfied and then M is IP-secure for.

33 Rushby s Unwinding Conditions for IP-security Suppose we have for each agent u an equivalence relation u on the states of M. OC: If s u t then O u (s) = O u (t). Theorem (Rushby): If these conditions are satisfied and then M is IP-secure for.

34 Rushby s Unwinding Conditions for IP-security Suppose we have for each agent u an equivalence relation u on the states of M. OC: If s u t then O u (s) = O u (t). WSC: If s u t and s agent(a) t then s a u t a. Theorem (Rushby): If these conditions are satisfied and then M is IP-secure for.

35 Rushby s Unwinding Conditions for IP-security Suppose we have for each agent u an equivalence relation u on the states of M. OC: If s u t then O u (s) = O u (t). WSC: If s u t and s agent(a) t then s a u t a. LR: If agent(a) u then s u s a. Theorem (Rushby): If these conditions are satisfied and then M is IP-secure for.

36 Access Control (Rushby s semantics for Bell-La Padula) A system with structured state is a machine S, s 0, Actions,step, O,agent together with a set N of names, a set V of values, and functions contents : S N V, with contents(s, n) interpreted as the value of object n in state s, observe : Agents P(N), with observe(u) interpreted as the set of objects that agent u can observe, and alter : D P(N), with alter(u) interpreted as the set of objects whose values agent u is permitted to alter.

37 A refinement of Rushby s Reference Monitor Assumptions A system with structured states is a weak access control system if where s oc u t if contents(s, n) = contents(t, n) for all n observe(u)

38 A refinement of Rushby s Reference Monitor Assumptions A system with structured states is a weak access control system if RM1 If s oc u t then O u(s) = O u (t). where s oc u t if contents(s, n) = contents(t, n) for all n observe(u)

39 A refinement of Rushby s Reference Monitor Assumptions A system with structured states is a weak access control system if RM1 If s oc u t then O u(s) = O u (t). RM2 For all actions a states s, t and names n alter(agent(a)), if s oc agent(a) t and contents(s, n) = contents(t, n) we have contents(s a, n) = contents(t a, n). where s oc u t if contents(s, n) = contents(t, n) for all n observe(u)

40 A refinement of Rushby s Reference Monitor Assumptions A system with structured states is a weak access control system if RM1 If s oc u t then O u(s) = O u (t). RM2 For all actions a states s, t and names n alter(agent(a)), if s oc agent(a) t and contents(s, n) = contents(t, n) we have contents(s a, n) = contents(t a, n). RM3 If contents(s a, n) contents(s, n) then n alter(agent(a)). where s oc u t if contents(s, n) = contents(t, n) for all n observe(u)

41 Consistency of an access control system with a policy: AOI. If alter(u) observe(v) then u v.

42 d h h,d. h.. d H D L A TA-secure system for H D L

43 TO-security Given a policy, define the functions to u with domain Actions by to u (ǫ) = ǫ and to u (αa) = to u (α) when agent(a) u and to u (αa) = (to u (α),view agent(a) (α), a) otherwise.

44 TO-security Given a policy, define the functions to u with domain Actions by to u (ǫ) = ǫ and to u (αa) = to u (α) when agent(a) u and to u (αa) = (to u (α),view agent(a) (α), a) otherwise. Define M to be TO-secure with respect to if for all agents u and all α, α Actions with to u (α) = to u (α ), we have O u (s 0 α) = O u (s 0 α ).

45 d h h,d. h.. d H D L A TA-secure system for H D L, not TO-secure

46 d h h,d. h.. d H D L TO-secure for H D L

47 d h h,d. h.. d H D L not TO-secure for H D L

48 How these definitions are related TO-security TA-security IP-security. If is transitive then all are equivalent.

49 Strengthening Distributed Knowledge (KR 08 paper) Consider D 1 L, D 2 L Observation: L may observe/know the relative order of actions of D 1, D 2, but this is not distributed knowledge to D 1, D 2. Define M, π, α = D p G φ if M, π, α = φ for all α such that α G = α G and u G view u(α) = view u (α ).

50 Theorem: Suppose that is acyclic, M is TO-secure with respect to and that π interprets q as depending only on Agents \ {u}. Then M, π = K u q D p I u q

51 Theorem: Suppose that is acyclic, M is TO-secure with respect to and that π interprets q as depending only on Agents \ {u}. Then M, π = K u q D p I u q Question: is the converse true?

52 Theorem: Suppose that is acyclic, M is TO-secure with respect to and that π interprets q as depending only on Agents \ {u}. Then M, π = K u q D p I u q Question: is the converse true? Answer: No, for several reasons (counter-examples in the paper)

53 A first refinement: Given α Actions, define m u (α) to be the prefix of α up to but excluding the rightmost action a with agent(a) = u.

54 A first refinement: Given α Actions, define m u (α) to be the prefix of α up to but excluding the rightmost action a with agent(a) = u. Intuition: if u v, then view u (m u (α)) is the latest information u could have passed to v in α.

55 A first refinement: Given α Actions, define m u (α) to be the prefix of α up to but excluding the rightmost action a with agent(a) = u. Intuition: if u v, then view u (m u (α)) is the latest information u could have passed to v in α. Define M, π, α = D m G φ if M, π, α = φ for all α such that α G = α G and u G view u(m u (α)) = view u (m u (α )).

56 A strengthened version of the theorem... Theorem: Suppose that is acyclic, M is TO-secure with respect to and that π interprets q as depending only on Agents \ {u}. Then M, π = K u q D m I u q

57 A strengthened version of the theorem... Theorem: Suppose that is acyclic, M is TO-secure with respect to and that π interprets q as depending only on Agents \ {u}. Then M, π = K u q D m I u q But this is still not enough to yield the converse...

58 A refined notion of proposition about other agents: Given a sequence α A, an agent u, and a set of agents G, define α u G to be the subsequence of α consisting of all actions of agents in G {u}, but with u s actions replaced by u.

59 A refined notion of proposition about other agents: Given a sequence α A, an agent u, and a set of agents G, define α u G to be the subsequence of α consisting of all actions of agents in G {u}, but with u s actions replaced by u. h d 1 l 2 h d 2 l 2 h h L {D} = d 1 L d 2 L

60 A refined notion of proposition about other agents: Given a sequence α A, an agent u, and a set of agents G, define α u G to be the subsequence of α consisting of all actions of agents in G {u}, but with u s actions replaced by u. h d 1 l 2 h d 2 l 2 h h L {D} = d 1 L d 2 L Say π interprets q as depending only on G relative to u if for all α, α A, if α u G = α u G, then α π(p) iff α π(p).

61 And a relativized notion of distributed knowledge... Define M, π, α = D m G,u φ if M, π, α = φ for all α such that α u G = α u G and u G view u(m u (α)) = view u (m u (α )).

62 Now we get the converse: Theorem: Suppose that is acyclic. Then M is TO-secure with respect to iff for all u Agents, and for all π that interpret q as depending only on Agents \ {u} relative to u, we have M, π = K u q D m I u,u q

63 Now we get the converse: Theorem: Suppose that is acyclic. Then M is TO-secure with respect to iff for all u Agents, and for all π that interpret q as depending only on Agents \ {u} relative to u, we have M, π = K u q D m I u,u q See the paper for a more general presentation that yields a similar result for other definitions of security

64 Summary An intuition concerning causality and distributed knowledge intuition: An agent s knowledge about other agents must be distributed knowledge to the other agents that can causally affect it. An accepted definition of causality from the security literature does not satisfy the intuition. New definitions of causality that better fit the security literature theory.

65 Refined definitions of distributed knowledge a proposition being about other agents that enable a theorem: causality in a system is consistent with iff the system satisfies the intuition with respect to.

66 Open Questions Extensions of the definitions/results to nondeterministic systems synchronous systems probabilistic systems cyclic policies Applications! Paper at FAST-08 on information flow in transactional memory

Using Architecture to Reason about Information Security

Using Architecture to Reason about Information Security Using Architecture to Reason about Information Security Ron van der Meyden (University of New South Wales Sydney, Australia) Joint work with Stephen Chong (Harvard) December 3, 2012 Overview Motivation:

More information

A comparison of semantic models for intransitive noninterference

A comparison of semantic models for intransitive noninterference A comparison of semantic models for intransitive noninterference Ron van der Meyden School of Computer Science and Engineering, University of New South Wales meyden@cse.unsw.edu.au Abstract. Noninterference

More information

The Complexity of Intransitive Noninterference

The Complexity of Intransitive Noninterference The Complexity of Intransitive Noninterference Sebastian Eggert, Ron van der Meyden, Henning Schnoor, Thomas Wilke Institut für Informatik, Kiel University School of Computer Science and Engineering, University

More information

Noninterference, Transitivity, and Channel-Control Security Policies 1

Noninterference, Transitivity, and Channel-Control Security Policies 1 May 5, 2005 Noninterference, Transitivity, and Channel-Control Security Policies 1 John Rushby Computer Science Laboratory SRI International Computer Science Laboratory 333 Ravenswood Ave. Menlo Park,

More information

Using Architecture to Reason About Information Security

Using Architecture to Reason About Information Security Using Architecture to Reason About Information Security Stephen Chong Harvard University Ron van der Meyden The University of New South Wales ABSTRACT We demonstrate, by a number of examples, that informationflow

More information

Non-interference. Christoph Sprenger and Sebastian Mödersheim. FMSEC Module 11, v.2 November 30, Department of Computer Science ETH Zurich

Non-interference. Christoph Sprenger and Sebastian Mödersheim. FMSEC Module 11, v.2 November 30, Department of Computer Science ETH Zurich Non-interference Christoph Sprenger and Sebastian Mödersheim Department of Computer Science ETH Zurich FMSEC Module 11, v.2 November 30, 2009 Outline 1 Basic Non-interference Unwinding Access Control Interpretations

More information

Deriving Epistemic Conclusions from Agent Architecture

Deriving Epistemic Conclusions from Agent Architecture Deriving Epistemic Conclusions from Agent Architecture Stephen Chong School of Engineering and Applied Sciences Harvard University Ron van der Meyden School of Computer Science and Engineering University

More information

Modeling Information Routing with Noninterference

Modeling Information Routing with Noninterference Modeling Information Routing with Noninterference Ruud Koolen and Julien Schmaltz Eindhoven University of Technology {r.p.j.koolen, j.schmaltz}@tue.nl ABSTRACT To achieve the highest levels of assurance,

More information

Angelo Troina. Joint work with: Ruggero Lanotte (University of Insubria at Como) Andrea Maggiolo Schettini (University of Pisa)

Angelo Troina. Joint work with: Ruggero Lanotte (University of Insubria at Como) Andrea Maggiolo Schettini (University of Pisa) Angelo Troina Dipartimento di Informatica, Università di Pisa, Italy Probabilistic Joint work with: Ruggero Lanotte (University of Insubria at Como) Andrea Maggiolo Schettini (University of Pisa) 1/23

More information

What is intransitive noninterference?

What is intransitive noninterference? What is intransitive noninterference? A.W. Roscoe Oxford University Computing Laboratory Wolfson Building, Parks Road Oxford OX1 3QD, UK M.H. Goldsmith Formal Systems (Europe) Ltd Keble Court, 26 Temple

More information

Noninterference for Operating System Kernels

Noninterference for Operating System Kernels Noninterference for Operating System Kernels Toby Murray 1,2, Daniel Matichuk 1, Matthew Brassil 1, Peter Gammie 1 and Gerwin Klein 1,2 1 NICTA, Sydney, Australia 2 School of Computer Science and Engineering,

More information

Expressing Security Properties Using Selective Interleaving Functions

Expressing Security Properties Using Selective Interleaving Functions Expressing Security Properties Using Selective Interleaving Functions Joseph Halpern and Sabina Petride August 8, 2008 Abstract McLean s notion of Selective Interleaving Functions (SIFs) is perhaps the

More information

Information Security Theory vs. Reality

Information Security Theory vs. Reality Information Security Theory vs. Reality 0368-4474-01, Winter 2011 Lecture 7: Information flow control Eran Tromer 1 Slides credit: Max Krohn, MIT Ian Goldberg and Urs Hengartner, University of Waterloo

More information

Language-based Information Security. CS252r Spring 2012

Language-based Information Security. CS252r Spring 2012 Language-based Information Security CS252r Spring 2012 This course Survey of key concepts and hot topics in language-based information security The use of programming language abstractions and techniques

More information

Hyperproperties. Department of Computer Science Cornell University

Hyperproperties. Department of Computer Science Cornell University Hyperproperties Michael R. Clarkson Fred B. Schneider {clarkson,fbs}@cs.cornell.edu Department of Computer Science Cornell University Computing and Information Science Technical Report http://hdl.handle.net/1813/9480

More information

Security Models and Information Flow

Security Models and Information Flow Security Models and Information Flow John McLean Center for High Assurance Computer Systems Naval Research Laboratory Washington, D.C. 20375 We develop a theory of information flow that differs from Nondeducibility

More information

Intransitive Non-Interference for Cryptographic Purposes

Intransitive Non-Interference for Cryptographic Purposes Intransitive Non-Interference for Cryptographic Purposes Michael Backes IBM Zurich Research Laboratory Rüschlikon, Switzerland mbc@zurich.ibm.com Birgit Pfitzmann IBM Zurich Research Laboratory Rüschlikon,

More information

Undecidable Cases of Model Checking Probabilistic Temporal-Epistemic Logic (Extended Abstract)

Undecidable Cases of Model Checking Probabilistic Temporal-Epistemic Logic (Extended Abstract) Undecidable Cases of Model Checking Probabilistic Temporal-Epistemic Logic (Extended Abstract) Ron van der Meyden School of Computer Science and Engineering UNSW Australia meyden@cse.unsw.edu.au Manas

More information

Mandatory Access Control (MAC)

Mandatory Access Control (MAC) CS 5323 Mandatory Access Control (MAC) Prof. Ravi Sandhu Executive Director and Endowed Chair Lecture 3 ravi.utsa@gmail.com www.profsandhu.com Ravi Sandhu 1 CS 5323 Lattice-Based Access Control (LBAC)

More information

Towards a Practical Secure Concurrent Language

Towards a Practical Secure Concurrent Language Towards a Practical Secure Concurrent Language Stefan Muller and Stephen Chong TR-05-12 Computer Science Group Harvard University Cambridge, Massachusetts Towards a Practical Secure Concurrent Language

More information

A logical approach to multilevel security of probabilistic systems

A logical approach to multilevel security of probabilistic systems Distrib. Comput. (1998) 11: 73 90 c Springer-Verlag 1998 A logical approach to multilevel security of probabilistic systems James W. Gray, III,1, Paul F. Syverson,2 1 Department of Computer Science, Hong

More information

Dynamic Noninterference Analysis Using Context Sensitive Static Analyses. Gurvan Le Guernic July 14, 2007

Dynamic Noninterference Analysis Using Context Sensitive Static Analyses. Gurvan Le Guernic July 14, 2007 Dynamic Noninterference Analysis Using Context Sensitive Static Analyses Gurvan Le Guernic July 14, 2007 1 Abstract This report proposes a dynamic noninterference analysis for sequential programs. This

More information

Secrecy in Multiagent Systems

Secrecy in Multiagent Systems Secrecy in Multiagent Systems Joseph Y. Halpern & Kevin R. O Neill Department of Computer Science Cornell University halpern@cs.cornell.edu; kroneill@gmail.com We introduce a general framework for reasoning

More information

On the Complexity of Input/Output Logic

On the Complexity of Input/Output Logic On the Complexity of Input/Output Logic Xin Sun 1 and Diego Agustín Ambrossio 12 1 Faculty of Science, Technology and Communication, University of Luxembourg, Luxembourg xin.sun@uni.lu 2 Interdisciplinary

More information

Abstract Non-Interference - An Abstract Interpretation-based approach to Secure Information Flow

Abstract Non-Interference - An Abstract Interpretation-based approach to Secure Information Flow Isabella Mastroeni Abstract Non-Interference - An Abstract Interpretation-based approach to Secure Information Flow Ph.D. Thesis 31 Marzo 2005 Università degli Studi di Verona Dipartimento di Informatica

More information

DISTRIBUTION A: Distribution approved for public release.

DISTRIBUTION A: Distribution approved for public release. AFRL-AFOSR-JP-TR-27-7 Independence-based Optimization of Epistemic Model Checking Ron van der Meyden UNIVERSITY OF NEW SOUTH WALES 2/22/27 Final Report DISTRIBUTION A: Distribution approved for public

More information

Communication and Concurrency: CCS

Communication and Concurrency: CCS Communication and Concurrency: CCS R. Milner, A Calculus of Communicating Systems, 1980 cours SSDE Master 1 Why calculi? Prove properties on programs and languages Principle: tiny syntax, small semantics,

More information

Analyzing Isochronic Forks with Potential Causality

Analyzing Isochronic Forks with Potential Causality Analyzing Isochronic Forks with Potential Causality Rajit Manohar Cornell NYC Tech New York, NY 10011, USA rajit@csl.cornell.edu Yoram Moses Technion-Israel Institute of Technology Haifa 32000, Israel

More information

Properties of Knowledge Forgetting

Properties of Knowledge Forgetting Properties of Knowledge Forgetting Yan Zhang and Yi Zhou Intelligent Systems Laboratory University of Western Sydney, Australia E-mail: {yan,yzhou}@scm.uws.edu.au Abstract In this paper we propose a formal

More information

Pseudorandom Generators

Pseudorandom Generators Outlines Saint Petersburg State University, Mathematics and Mechanics 2nd April 2005 Outlines Part I: Main Approach Part II: Blum-Blum-Shub Generator Part III: General Concepts of Pseudorandom Generator

More information

A Cut Principle for Information Flow

A Cut Principle for Information Flow A Cut Principle for Information Flow Joshua D. Guttman and Paul D. Rowe The MITRE Corporation {guttman,prowe}@mitre.org Abstract. We view a distributed system as a graph of active locations with unidirectional

More information

D21.4 Formal Framework for MILS Integration

D21.4 Formal Framework for MILS Integration This project has received funding from the European Unions Seventh Framework Programme for research, technological development and demonstration under grant agreement no 318353 D21.4 Formal Framework for

More information

DERIVING AND PROVING ABSTRACT NON-INTERFERENCE

DERIVING AND PROVING ABSTRACT NON-INTERFERENCE DERIVING AND PROVING ABSTRACT NON-INTERFERENCE Roberto Giacobazzi and Isabella Mastroeni Dipartimento di Informatica Università di Verona Italy Paris, February 20th, 2004 Deriving and Proving Abstract

More information

A Unified Framework for Representing Logic Program Updates

A Unified Framework for Representing Logic Program Updates A Unified Framework for Representing Logic Program Updates Yan Zhang School of Computing & Information Technology University of Western Sydney NSW 1797 Australia E-mail: yan@cit.uws.edu.au Norman Foo School

More information

An Introduction to Probabilistic Encryption

An Introduction to Probabilistic Encryption Osječki matematički list 6(2006), 37 44 37 An Introduction to Probabilistic Encryption Georg J. Fuchsbauer Abstract. An introduction to probabilistic encryption is given, presenting the first probabilistic

More information

The Bell-LaPadula Model

The Bell-LaPadula Model The Bell-LaPadula Model CSM27 Computer Security Dr Hans Georg Schaathun University of Surrey Autumn 2007 Dr Hans Georg Schaathun The Bell-LaPadula Model Autumn 2007 1 / 25 The session Session objectives

More information

arxiv: v1 [cs.lo] 8 Sep 2014

arxiv: v1 [cs.lo] 8 Sep 2014 An Epistemic Strategy Logic Xiaowei Huang Ron van der Meyden arxiv:1409.2193v1 [cs.lo] 8 Sep 2014 The University of New South Wales Abstract The paper presents an extension of temporal epistemic logic

More information

MOST OF the published research on control of discreteevent

MOST OF the published research on control of discreteevent IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 43, NO. 1, JANUARY 1998 3 Discrete-Event Control of Nondeterministic Systems Michael Heymann and Feng Lin, Member, IEEE Abstract Nondeterminism in discrete-event

More information

Space-Time Viewpoints for Concurrent Processes Represented by Relational Structures

Space-Time Viewpoints for Concurrent Processes Represented by Relational Structures Space-Time Viewpoints for Concurrent Processes Represented by Relational Structures Irina Virbitskaite 1,2, Elena Bozhenkova 1,2, Evgeny Erofeev 3 1 A.P. Ershov Institute of Informatics Systems, SB RAS

More information

Equivalence of Regular Expressions and FSMs

Equivalence of Regular Expressions and FSMs Equivalence of Regular Expressions and FSMs Greg Plaxton Theory in Programming Practice, Spring 2005 Department of Computer Science University of Texas at Austin Regular Language Recall that a language

More information

Can an Operation Both Update the State and Return a Meaningful Value in the Asynchronous PRAM Model?

Can an Operation Both Update the State and Return a Meaningful Value in the Asynchronous PRAM Model? Can an Operation Both Update the State and Return a Meaningful Value in the Asynchronous PRAM Model? Jaap-Henk Hoepman Department of Computer Science, University of Twente, the Netherlands hoepman@cs.utwente.nl

More information

Beyond Determinism in Measurement-based Quantum Computation

Beyond Determinism in Measurement-based Quantum Computation Beyond Determinism in Measurement-based Quantum Computation Simon Perdrix CNRS, Laboratoire d Informatique de Grenoble Joint work with Mehdi Mhalla, Mio Murao, Masato Someya, Peter Turner The Eight International

More information

EAHyper: Satisfiability, Implication, and Equivalence Checking of Hyperproperties

EAHyper: Satisfiability, Implication, and Equivalence Checking of Hyperproperties EAHyper: Satisfiability, Implication, and Equivalence Checking of Hyperproperties Bernd Finkbeiner, Christopher Hahn, and Marvin Stenger Saarland Informatics Campus, Saarland University, Saarbrücken, Germany

More information

Towards information flow control. Chaire Informatique et sciences numériques Collège de France, cours du 30 mars 2011

Towards information flow control. Chaire Informatique et sciences numériques Collège de France, cours du 30 mars 2011 Towards information flow control Chaire Informatique et sciences numériques Collège de France, cours du 30 mars 2011 Mandatory access controls and security levels DAC vs. MAC Discretionary access control

More information

Security of Quantum Key Distribution with Imperfect Devices

Security of Quantum Key Distribution with Imperfect Devices Security of Quantum Key Distribution with Imperfect Devices Hoi-Kwong Lo Dept. of Electrical & Comp. Engineering (ECE); & Dept. of Physics University of Toronto Email:hklo@comm.utoronto.ca URL: http://www.comm.utoronto.ca/~hklo

More information

MAKING THE UNOBSERVABLE, UNOBSERVABLE.

MAKING THE UNOBSERVABLE, UNOBSERVABLE. MAKING THE UNOBSERVABLE, UNOBSERVABLE. 3 PAPERS FROM THE LAST 365 DAYS AVAILABLE TO READ NOW ON YOUR COMPUTER PAWEL SOBOCINSKI AND JULIAN RATHKE GO TO www.ecs.soton.ac.uk/~ps/publications.php Plan of the

More information

Duality in Probabilistic Automata

Duality in Probabilistic Automata Duality in Probabilistic Automata Chris Hundt Prakash Panangaden Joelle Pineau Doina Precup Gavin Seal McGill University MFPS May 2006 Genoa p.1/40 Overview We have discovered an - apparently - new kind

More information

Non-interference in partial order models

Non-interference in partial order models Non-interference in partial order models Béatrice Bérard, Loïc Hélouët, John Mullins To cite this version: Béatrice Bérard, Loïc Hélouët, John Mullins. Non-interference in partial order models. ACSD 15-15th

More information

Section 6 Fault-Tolerant Consensus

Section 6 Fault-Tolerant Consensus Section 6 Fault-Tolerant Consensus CS586 - Panagiota Fatourou 1 Description of the Problem Consensus Each process starts with an individual input from a particular value set V. Processes may fail by crashing.

More information

Review of The π-calculus: A Theory of Mobile Processes

Review of The π-calculus: A Theory of Mobile Processes Review of The π-calculus: A Theory of Mobile Processes Riccardo Pucella Department of Computer Science Cornell University July 8, 2001 Introduction With the rise of computer networks in the past decades,

More information

Sémantique des jeux asynchrones et réécriture 2-dimensionnelle

Sémantique des jeux asynchrones et réécriture 2-dimensionnelle Sémantique des jeux asynchrones et réécriture 2-dimensionnelle Soutenance de thèse de doctorat Samuel Mimram Laboratoire PPS (CNRS Université Paris Diderot) 1 er décembre 2008 1 / 64 A program is a text

More information

Notes on BAN Logic CSG 399. March 7, 2006

Notes on BAN Logic CSG 399. March 7, 2006 Notes on BAN Logic CSG 399 March 7, 2006 The wide-mouthed frog protocol, in a slightly different form, with only the first two messages, and time stamps: A S : A, {T a, B, K ab } Kas S B : {T s, A, K ab

More information

Modal Logics with a Linear Hierarchy of Local Propositional Quantifiers

Modal Logics with a Linear Hierarchy of Local Propositional Quantifiers Draft for submission to Advances in Modal Logic 2002 Modal Logics with a Linear Hierarchy of Local Propositional Quantifiers Kai Engelhardt Ron van der Meyden Kaile Su May 20, 2002 Local propositions arise

More information

Iit Istituto di Informatica e Telematica

Iit Istituto di Informatica e Telematica C Consiglio Nazionale delle Ricerche Specification and Analysis of Information Flow Properties for Distributed Systems R. Gorreri, F. Martinelli, I. Matteucci IIT TR-12/2010 Technical report Febbraio 2010

More information

Communication and Concurrency: CCS. R. Milner, A Calculus of Communicating Systems, 1980

Communication and Concurrency: CCS. R. Milner, A Calculus of Communicating Systems, 1980 Communication and Concurrency: CCS R. Milner, A Calculus of Communicating Systems, 1980 Why calculi? Prove properties on programs and languages Principle: tiny syntax, small semantics, to be handled on

More information

Knowledge Based Obligations RUC-ILLC Workshop on Deontic Logic

Knowledge Based Obligations RUC-ILLC Workshop on Deontic Logic Knowledge Based Obligations RUC-ILLC Workshop on Deontic Logic Eric Pacuit Stanford University November 9, 2007 Eric Pacuit: Knowledge Based Obligations, RUC-ILLC Workshop on Deontic Logic 1 The Kitty

More information

Petri nets. s 1 s 2. s 3 s 4. directed arcs.

Petri nets. s 1 s 2. s 3 s 4. directed arcs. Petri nets Petri nets Petri nets are a basic model of parallel and distributed systems (named after Carl Adam Petri). The basic idea is to describe state changes in a system with transitions. @ @R s 1

More information

Communicating Parallel Processes. Stephen Brookes

Communicating Parallel Processes. Stephen Brookes Communicating Parallel Processes Stephen Brookes Carnegie Mellon University Deconstructing CSP 1 CSP sequential processes input and output as primitives named parallel composition synchronized communication

More information

Embedded Systems 5. Synchronous Composition. Lee/Seshia Section 6.2

Embedded Systems 5. Synchronous Composition. Lee/Seshia Section 6.2 Embedded Systems 5-1 - Synchronous Composition Lee/Seshia Section 6.2 Important semantic model for concurrent composition Here: composition of actors Foundation of Statecharts, Simulink, synchronous programming

More information

Models of Concurrency

Models of Concurrency Models of Concurrency GERARDO SCHNEIDER UPPSALA UNIVERSITY DEPARTMENT OF INFORMATION TECHNOLOGY UPPSALA, SWEDEN Thanks to Frank Valencia Models of Concurrency p.1/57 Concurrency is Everywhere Concurrent

More information

A Goal-Oriented Algorithm for Unification in EL w.r.t. Cycle-Restricted TBoxes

A Goal-Oriented Algorithm for Unification in EL w.r.t. Cycle-Restricted TBoxes A Goal-Oriented Algorithm for Unification in EL w.r.t. Cycle-Restricted TBoxes Franz Baader, Stefan Borgwardt, and Barbara Morawska {baader,stefborg,morawska}@tcs.inf.tu-dresden.de Theoretical Computer

More information

Complexity of Checking Freshness of Cryptographic Protocols

Complexity of Checking Freshness of Cryptographic Protocols Complexity of Checking Freshness of Cryptographic Protocols Zhiyao Liang Rakesh M Verma Computer Science Department, University of Houston, Houston TX 77204-3010, USA Email: zliang@cs.uh.edu, rmverma@cs.uh.edu

More information

Quantitative Information Flow. Lecture 7

Quantitative Information Flow. Lecture 7 Quantitative Information Flow Lecture 7 1 The basic model: Systems = Information-Theoretic channels Secret Information Observables s1 o1... System... sm on Input Output 2 Probabilistic systems are noisy

More information

Differential Privacy for Probabilistic Systems. Michael Carl Tschantz Anupam Datta Dilsun Kaynar. May 14, 2009 CMU-CyLab

Differential Privacy for Probabilistic Systems. Michael Carl Tschantz Anupam Datta Dilsun Kaynar. May 14, 2009 CMU-CyLab Differential Privacy for Probabilistic Systems Michael Carl Tschantz Anupam Datta Dilsun Kaynar May 14, 2009 CMU-CyLab-09-008 CyLab Carnegie Mellon University Pittsburgh, PA 15213 Differential Privacy

More information

Reasoning by Regression: Pre- and Postdiction Procedures for Logics of Action and Change with Nondeterminism*

Reasoning by Regression: Pre- and Postdiction Procedures for Logics of Action and Change with Nondeterminism* Reasoning by Regression: Pre- and Postdiction Procedures for Logics of Action and Change with Nondeterminism* Marcus Bjareland and Lars Karlsson Department of Computer and Information Science Linkopings

More information

Plan diagnosis with agents

Plan diagnosis with agents Plan diagnosis with agents Nico Roos a Cees Witteveen b c a IKAT, Universiteit Maastricht, P.O. Box 616, NL-6200 MD Maastricht, e-mail: roos@cs.unimaas.nl b Faculty EEMCS Delft University of Technology,

More information

Alternating-Time Temporal Logic

Alternating-Time Temporal Logic Alternating-Time Temporal Logic R.Alur, T.Henzinger, O.Kupferman Rafael H. Bordini School of Informatics PUCRS R.Bordini@pucrs.br Logic Club 5th of September, 2013 ATL All the material in this presentation

More information

Propositional Dynamic Logic

Propositional Dynamic Logic Propositional Dynamic Logic Contents 1 Introduction 1 2 Syntax and Semantics 2 2.1 Syntax................................. 2 2.2 Semantics............................... 2 3 Hilbert-style axiom system

More information

Typing Noninterference for Reactive Programs

Typing Noninterference for Reactive Programs Typing Noninterference for Reactive Programs Ana Almeida Matos, Gérard Boudol and Ilaria Castellani June 7, 2004 Abstract We propose a type system to enforce the security property of noninterference in

More information

An Independence Relation for Sets of Secrets

An Independence Relation for Sets of Secrets Sara Miner More Pavel Naumov An Independence Relation for Sets of Secrets Abstract. A relation between two secrets, known in the literature as nondeducibility, was originally introduced by Sutherland.

More information

Wojciech Penczek. Polish Academy of Sciences, Warsaw, Poland. and. Institute of Informatics, Siedlce, Poland.

Wojciech Penczek. Polish Academy of Sciences, Warsaw, Poland. and. Institute of Informatics, Siedlce, Poland. A local approach to modal logic for multi-agent systems? Wojciech Penczek 1 Institute of Computer Science Polish Academy of Sciences, Warsaw, Poland and 2 Akademia Podlaska Institute of Informatics, Siedlce,

More information

arxiv: v1 [cs.lo] 19 Mar 2019

arxiv: v1 [cs.lo] 19 Mar 2019 Turing-Completeness of Dynamics in Abstract Persuasion Argumentation Ryuta Arisaka arxiv:1903.07837v1 [cs.lo] 19 Mar 2019 ryutaarisaka@gmail.com Abstract. Abstract Persuasion Argumentation (APA) is a dynamic

More information

Complete Axiomatizations for Reasoning about Knowledge and Branching Time

Complete Axiomatizations for Reasoning about Knowledge and Branching Time Ron van der Meyden Ka-shu Wong Complete Axiomatizations for Reasoning about Knowledge and Branching Time Abstract Sound and complete axiomatizations are provided for a number of different logics involving

More information

A Cryptographic Decentralized Label Model

A Cryptographic Decentralized Label Model A Cryptographic Decentralized Label Model Jeffrey A. Vaughan and Steve Zdancewic Department of Computer and Information Science University of Pennsylvania IEEE Security and Privacy May 22, 2007 Information

More information

Synchronous cooperation for explicit multi-threading

Synchronous cooperation for explicit multi-threading Acta Informatica (2007) 44:525 569 DOI 10.1007/s00236-007-0057-9 ORIGINAL ARTICLE Synchronous cooperation for explicit multi-threading J. A. Bergstra C. A. Middelburg Received: 24 October 2006 / Accepted:

More information

CS 395T. Probabilistic Polynomial-Time Calculus

CS 395T. Probabilistic Polynomial-Time Calculus CS 395T Probabilistic Polynomial-Time Calculus Security as Equivalence Intuition: encryption scheme is secure if ciphertext is indistinguishable from random noise Intuition: protocol is secure if it is

More information

Using Counterfactuals in Knowledge-Based Programming

Using Counterfactuals in Knowledge-Based Programming Using Counterfactuals in Knowledge-Based Programming Joseph Y. Halpern Cornell University Dept. of Computer Science Ithaca, NY 14853 halpern@cs.cornell.edu http://www.cs.cornell.edu/home/halpern Yoram

More information

Symmetric Encryption

Symmetric Encryption 1 Symmetric Encryption Mike Reiter Based on Chapter 5 of Bellare and Rogaway, Introduction to Modern Cryptography. Symmetric Encryption 2 A symmetric encryption scheme is a triple SE = K, E, D of efficiently

More information

Model Theory for PRS-Like Agents: Modelling Belief Update and Action Attempts

Model Theory for PRS-Like Agents: Modelling Belief Update and Action Attempts Model Theory for PRS-Like Agents: Modelling Belief Update and Action Attempts Wayne Wobcke School of Computer Science and Engineering University of New South Wales Sydney NSW 2052, Australia wobcke@cse.unsw.edu.au

More information

Property Checking of Safety- Critical Systems Mathematical Foundations and Concrete Algorithms

Property Checking of Safety- Critical Systems Mathematical Foundations and Concrete Algorithms Property Checking of Safety- Critical Systems Mathematical Foundations and Concrete Algorithms Wen-ling Huang and Jan Peleska University of Bremen {huang,jp}@cs.uni-bremen.de MBT-Paradigm Model Is a partial

More information

COVERT CHANNEL CAPACITY

COVERT CHANNEL CAPACITY COVERT CHANNEL CAPACITY Jonathan K. Millen The MITRE Corporation Bedford, MA 01730 Techniques for detecting covert channels are based on information flow models. This paper establishes a connection between

More information

The size of decision table can be understood in terms of both cardinality of A, denoted by card (A), and the number of equivalence classes of IND (A),

The size of decision table can be understood in terms of both cardinality of A, denoted by card (A), and the number of equivalence classes of IND (A), Attribute Set Decomposition of Decision Tables Dominik Slezak Warsaw University Banacha 2, 02-097 Warsaw Phone: +48 (22) 658-34-49 Fax: +48 (22) 658-34-48 Email: slezak@alfa.mimuw.edu.pl ABSTRACT: Approach

More information

09 Modal Logic II. CS 3234: Logic and Formal Systems. October 14, Martin Henz and Aquinas Hobor

09 Modal Logic II. CS 3234: Logic and Formal Systems. October 14, Martin Henz and Aquinas Hobor Martin Henz and Aquinas Hobor October 14, 2010 Generated on Thursday 14 th October, 2010, 11:40 1 Review of Modal Logic 2 3 4 Motivation Syntax and Semantics Valid Formulas wrt Modalities Correspondence

More information

An Epistemic Characterization of Zero Knowledge

An Epistemic Characterization of Zero Knowledge An Epistemic Characterization of Zero Knowledge Joseph Y. Halpern, Rafael Pass, and Vasumathi Raman Computer Science Department Cornell University Ithaca, NY, 14853, U.S.A. e-mail: {halpern, rafael, vraman}@cs.cornell.edu

More information

Cryptographic Protocols Notes 2

Cryptographic Protocols Notes 2 ETH Zurich, Department of Computer Science SS 2018 Prof. Ueli Maurer Dr. Martin Hirt Chen-Da Liu Zhang Cryptographic Protocols Notes 2 Scribe: Sandro Coretti (modified by Chen-Da Liu Zhang) About the notes:

More information

Stream ciphers. Pawel Wocjan. Department of Electrical Engineering & Computer Science University of Central Florida

Stream ciphers. Pawel Wocjan. Department of Electrical Engineering & Computer Science University of Central Florida Stream ciphers Pawel Wocjan Department of Electrical Engineering & Computer Science University of Central Florida wocjan@eecs.ucf.edu Definition of block ciphers Block ciphers: crypto work horse n bits

More information

Introduction to Kleene Algebras

Introduction to Kleene Algebras Introduction to Kleene Algebras Riccardo Pucella Basic Notions Seminar December 1, 2005 Introduction to Kleene Algebras p.1 Idempotent Semirings An idempotent semiring is a structure S = (S, +,, 1, 0)

More information

Temporal & Modal Logic. Acronyms. Contents. Temporal Logic Overview Classification PLTL Syntax Semantics Identities. Concurrency Model Checking

Temporal & Modal Logic. Acronyms. Contents. Temporal Logic Overview Classification PLTL Syntax Semantics Identities. Concurrency Model Checking Temporal & Modal Logic E. Allen Emerson Presenter: Aly Farahat 2/12/2009 CS5090 1 Acronyms TL: Temporal Logic BTL: Branching-time Logic LTL: Linear-Time Logic CTL: Computation Tree Logic PLTL: Propositional

More information

Learning to Act: Qualitative Learning of Deterministic Action Models

Learning to Act: Qualitative Learning of Deterministic Action Models Learning to Act: Qualitative Learning of Deterministic Action Models Thomas Bolander Nina Gierasimczuk October 8, 2017 Abstract In this paper we study learnability of fully observable, universally applicable

More information

arxiv: v1 [cs.dc] 22 Oct 2018

arxiv: v1 [cs.dc] 22 Oct 2018 FANTOM: A SCALABLE FRAMEWORK FOR ASYNCHRONOUS DISTRIBUTED SYSTEMS A PREPRINT Sang-Min Choi, Jiho Park, Quan Nguyen, and Andre Cronje arxiv:1810.10360v1 [cs.dc] 22 Oct 2018 FANTOM Lab FANTOM Foundation

More information

Temporal Logics for Hyperproperties

Temporal Logics for Hyperproperties Temporal Logics for Hyperproperties Michael R. Clarkson 1, Bernd Finkbeiner 2, Masoud Koleini 1, Kristopher K. Micinski 3, Markus N. Rabe 2, and César Sánchez 4 1 George Washington University; 2 Universität

More information

Probabilistic Bisimilarity as Testing Equivalence

Probabilistic Bisimilarity as Testing Equivalence Probabilistic Bisimilarity as Testing Equivalence Yuxin Deng a,, Yuan Feng b a Shanghai Key Laboratory of Trustworthy Computing, MOE International Joint Lab of Trustworthy Software, and International Research

More information

CS 6260 Applied Cryptography

CS 6260 Applied Cryptography CS 6260 Applied Cryptography Symmetric encryption schemes A scheme is specified by a key generation algorithm K, an encryption algorithm E, and a decryption algorithm D. K K =(K,E,D) MsgSp-message space

More information

Quantitative Information Flow in Interactive Systems

Quantitative Information Flow in Interactive Systems Quantitative Information Flow in Interactive Systems Mário Alvim, Miguel Andrés, Catuscia Palamidessi To cite this version: Mário Alvim, Miguel Andrés, Catuscia Palamidessi. Quantitative Information Flow

More information

Lecture 9 - Symmetric Encryption

Lecture 9 - Symmetric Encryption 0368.4162: Introduction to Cryptography Ran Canetti Lecture 9 - Symmetric Encryption 29 December 2008 Fall 2008 Scribes: R. Levi, M. Rosen 1 Introduction Encryption, or guaranteeing secrecy of information,

More information

Verifiable Security of Boneh-Franklin Identity-Based Encryption. Federico Olmedo Gilles Barthe Santiago Zanella Béguelin

Verifiable Security of Boneh-Franklin Identity-Based Encryption. Federico Olmedo Gilles Barthe Santiago Zanella Béguelin Verifiable Security of Boneh-Franklin Identity-Based Encryption Federico Olmedo Gilles Barthe Santiago Zanella Béguelin IMDEA Software Institute, Madrid, Spain 5 th International Conference on Provable

More information

Quantitative Information Leakage. Lecture 9

Quantitative Information Leakage. Lecture 9 Quantitative Information Leakage Lecture 9 1 The baic model: Sytem = Information-Theoretic channel Secret Information Obervable 1 o1... Sytem... m on Input Output 2 Toward a quantitative notion of leakage

More information

The Discrete EVent System specification (DEVS) formalism

The Discrete EVent System specification (DEVS) formalism The Discrete EVent System specification (DEVS) formalism Hans Vangheluwe The DEVS formalism was conceived by Zeigler [Zei84a, Zei84b] to provide a rigourous common basis for discrete-event modelling and

More information

Finite State Automata

Finite State Automata Trento 2005 p. 1/4 Finite State Automata Automata: Theory and Practice Paritosh K. Pandya (TIFR, Mumbai, India) Unversity of Trento 10-24 May 2005 Trento 2005 p. 2/4 Finite Word Langauges Alphabet Σ is

More information

Recovering Private Keys Generated With Weak PRNGs

Recovering Private Keys Generated With Weak PRNGs Recovering Private Keys Generated With Weak PRNGs Pierre-Alain Fouque (Univ. Rennes 1) Mehdi Tibouchi (NTT Secure Platform Lab.) Jean-Christophe Zapalowicz (Inria) Journées C2 2014 Jean-Christophe Zapalowicz

More information