Analytical Enhancements and Practical Insights for MPCP with Self-Suspensions. Pratyush Patel, Iljoo Baek, Hyoseung Kim*, Raj Rajkumar

Transcription

1 Analytical Enhancements and Practical Insights for MPCP with Self-Suspensions Pratyush Patel, Iljoo Baek, Hyoseung Kim*, Raj Rajkumar *

2 High Computational Demand of Safety-Critical Systems Sense Compute Actuate Repeat Sensors Perception Deadline 33 ms Path Planning Deadline 66 ms Brake Control Deadline 100 ms Actuators Computing System Long execution time Difficult to meet deadlines Need Computational Accelerators * 2/26 * GP-GPUs (General-Purpose GPUs) Digital Signal Processor (DSP)

3 Problems with Hardware Accelerators They do not support preemption o Due to high context switching overhead* They handle multiple resource requests in any order o Concurrent execution on GPU may result in unpredictable delays 3 identical CUDA kernels on NVIDIA GTX % slowdown on two kernels Unpredictable which kernel gets delayed They do not respect task priorities or scheduling policies o May result in unbounded priority inversion * I. Tanasicet al. Enabling preemptive multiprogramming on GPUs. In International Symposium on Computer Architecture (ISCA), Some recent GPU architectures support preemption - NVIDIA Volta Architecture 3/26

4 Existing Solution: Synchronization- Based Approaches * Lock Critical Section Unlock CPU GPU Benefits of synchronization-based approaches Do not require any change in accelerator device drivers Existing schedulability analyses can be directly re-used * G. Elliott and J. Anderson. Globally scheduled real-time multiprocessor systems with GPUs. Real-Time Syst., 48(1):34 74, G. Elliott and J. Anderson. An optimal k-exclusion real-time locking protocol motivated by multi-gpu systems. Real-Time Syst., 49(2): , G. Elliott et al. GPUSync: A framework for real-time GPU management. In IEEE Real-Time Systems Symposium (RTSS), /26

5 Limitations Busy waiting o Critical sections are executed entirely on the CPU Common assumption of most RT synch. protocols, e.g., MPCP*, FMLP, OMLP Lock Critical Section Unlock CPU Busy wait GPU Analytical pessimism o Traditional recursion-based analysis # o Can lead to expensive over-provisioning * R. Rajkumar, L. Sha, and J. P. Lehoczky. Real-time synchronization protocols for multiprocessors. In IEEE Real-Time Systems Symposium (RTSS), A. Block et al. A flexible real-time locking protocol for multiprocessors. In IEEE Embedded and Real-Time Comp. Systems and Apps., (RTCSA), B. Brandenburg and J. Anderson. The OMLP family of optimal multiprocessor real-time locking protocols. Design Automation for Embedded Systems, 2013 # K. Lakshmanan, D. de Niz, and R. Rajkumar. Coordinated task scheduling, allocation and synchronization on multiprocessors. In IEEE Real-Time Systems Symposium (RTSS), /26

6 Our Contributions Analytical enhancements for the Multiprocessor Priority Ceiling Protocol (MPCP) Tighter bounds for task response times Allow suspensions when executing critical sections Extensive schedulability experiments for a variety of task set parameters Prototype implementation and evaluation on Nvidia TX2 running Linux Extensions can be used with multiple types of computational accelerators, such as a digital signal processor (DSP) and General-Purpose GPU (GP-GPU) 6/26

7 Outline Motivation & Introduction Suspension-based MPCP System model Comparison with busy-waiting approach Task response time analysis Evaluation Conclusions 7/26

8 Example of GPU Execution Non-critical section Non-critical section Non-critical section Critical section Critical section CPU GPU GPU execution GPU execution Task arrival GPU request CPU execution GPU execution Misc. operation 8/26

9 System Model Sporadic tasks with constrained deadlines o Task τ ii = CC ii, GG ii, TT ii, η ii CC ii : Sum of the WCET* of all non-critical sections GG ii : Sum of the WCET* of all critical sections TT ii : Period (Deadline = Period) η ii : Maximum number of critical sections ζ ii,jj : Maximum number of suspensions in the j th critical section o Critical segment GG ii,jj = (GG ee ii,jj, GG ii,jj mm ) Critical section Critical section Each hardware accelerator is modeled as a distinct shared resource Use partitioned fixed-priority preemptive scheduling * Worst-Case Execution Time (WCET) 9/26

10 Example under Busy-Waiting MPCP Response time of ττ h : 9 τ h Busy-wait CPU Core 1 τ mm Global priority ceiling Busy-wait Global priority ceiling CPU Core 2 τ ll Busy-wait Global priority ceiling GPU Time Task arrival CPU execution GPU execution Blocked segment Preempted segment GPU request Busy wait Misc. operation 10/26

11 Example under Suspension-based MPCP Response time of ττ h : 6 + overhead CPU Core 1 τ h τ mm Suspend Priority boosting Suspend Priority boosting CPU Core 2 τ ll Suspend Priority boosting GPU Time Task arrival GPU request CPU execution GPU execution Blocked segment Preempted segment Self-Suspension Misc. operation Self-suspension overhead 11/26

12 Task Response Time Analysis Worst-Case Response Time (WW ii )* i = task number Self-suspension by higher-priority tasks WW ii nn+1 = CC ii + GG ii + BB ii + Blocking delay Our contribution τ h hpppp(τii ) WW ii nn + WW h CC h TT h Preemption delay by Higher-priority tasks CC h * N. Audsley, A. Burns, M. Richardson, K. Tindell, and A. Wellings. Applying new scheduling theory to static priority pre-emptive scheduling. Software Engineering Journal, 8(5): , J.-J. Chen et al. Many suspensions, many problems: A review of self-suspending tasks in real-time systems. Technical Report 854, Department of Computer Science, TU Dortmund, /26

13 Total Blocking Time Analysis For each analyzed task, Request-driven (RD) Approach* o Consider the sum of the worst-case blocking times for each lockacquisition request issued by the analyzed task Job-driven (JD) Approach o Consider the maximum number of lock-acquisition requests issued by other tasks during the execution of the analyzed task Hybrid Approach o Upper-bound the maximum lock-acquisition requests possible in RD analysis by using JD analysis obtain the best of both approaches o Different from RTAS'14 which simply takes the minimum of RD and JD for the blocking delay * K. Lakshmanan, D. de Niz, and R. Rajkumar. Coordinated task scheduling, allocation and synchronization on multiprocessors. In IEEE Real-Time Systems Symposium (RTSS), H. Kim, D. de Niz, B. Andersson, M. Klein, O. Mutlu, and R. Rajkumar. Bounding memory interference delay in COTS-based multi-core systems. In IEEE Real-Time Technology and Applications Symposium (RTAS), /26

14 Request-driven* Blocking Time Analysis Task C i G i ηη ii G i,1 G i,2 TT ii CPU ττ ττ ττ ττ 11 s period ττ 33 s period GPU execution Blocked segment Job arrival ττ 11 ττ 22 ττ 33 For each request made by ττ 33, its blocking time is given by BB 3 = BB 3,1 + BB 3,2 Time = = 204 * K. Lakshmanan, D. de Niz, and R. Rajkumar. Coordinated task scheduling, allocation and synchronization on multiprocessors. In IEEE Real-Time Systems Symposium (RTSS), /26

15 Job-driven Blocking Time Analysis Task C i G i ηη ii G i,1 G i,2 TT ii CPU ττ ττ ττ ττ 11 s period ττ 33 s period GPU execution Blocked segment Job arrival ττ 11 ττ 22 ττ 33 Time Request-Driven BB The max requests (= jobs) made by other tasks 3 = 204 # of req. by ττ 11 = 12 # of req. by ττ 22 = 1 BB 3 = 12x1 + 1x100 = /26

16 Hybrid Blocking Time Analysis Task C i G i ηη ii G i,1 G i,2 TT ii CPU ττ ττ ττ ττ 11 s period ττ 33 s period GPU execution Blocked segment Job arrival ττ 11 ττ 22 ττ 33 Time Both each req. made by ττ 33 and max req. made by other tasks BB 3 = BB 3,1 + BB 3,2 = (4x1) + (1x100) = /26 Job-Driven BB 3 = 112 Request-Driven BB 3 = 204

17 Outline Motivation & Introduction Suspension-based MPCP Evaluation Case Study Schedulability Experiment Conclusions 17/26

18 Case Study Motivated by the software system of CMU s self-driving car* o Lane-Change Detector Read Image GPU Execution Color Conversion GPU Execution Feature Extraction Classification o Workzone Detector Read Image Preprocessing GPU Execution Blob Detection Classification o Matrix Calculation Arithmetic Calculation GPU Execution Matrix Calculation Arithmetic Calculation * J. Wei et al. Towards a viable autonomous driving research platform. In IEEE Intelligent Vehicles Symposium (IV), J. Lee et al. Kernel-based traffic sign tracking to improve highway workzone recognition for reliable autonomous driving. In IEEE International Conference on Intelligent Transportation Systems (ITSC), /26

19 Experimental Setup NVIDIA TX2 6 CPU Cores 1 GPU (256 Cores, Pascal Arch.) CPU core 1 CPU core 2 Lane Change Detector Workzone Detector Two Matrix Calculations One Matrix Calculation Task priorities are assigned based on the rate-monotonic policy 19/26

20 DEMO Deadline : 33 ms Deadline : 33 ms 20/26

21 Suspension-based vs. Busy-waiting Worst-Case Response Time (ms) MPCP Suspension-based MPCP Busy-waiting MPCP 0 LaneChange Workzone Matrix Cal. 1 Matrix Cal. 2 Matrix Cal. 3 Performs better in practice, especially for lower-priority tasks Allows other tasks to use the CPU while a task is using the GPU 21/26

22 Effect of Suspension Overhead Test result w.r.t. the number of co-scheduled tasks w/ small GPU segments Worst-Case Response time (ms) μs Workzone (Busy-wait) Workzone (Self-suspension) Lane Change (Busy-wait) Lane Change (Self-suspension) Number of co-scheduled Matrix calculation tasks per core The overhead of self-suspension implementation negatively affects task response times when the tasks have small GPU segments 22/26

23 Schedulability Experiments Purpose: To explore the impact of the different approaches on task schedulability 10,000 randomly-generated tasksets Parameters Values Number of CPUs (m) 4 Number of shared resources (g) [1, 3] Number of tasks per CPU [3, 6] Percentage of tasks with critical sections [10, 40] % Task period and deadline (TT ii = DD ii ) MPCP Our Analysis FMLP+ LP-based Analysis* [30, 500] ms Utilization per CPU [40, 60] % Ratio of crit. Sec. len. To non-crit. Sec. len. (GG ii / CC ii ) [10, 30] % Number of critical sections per task (η ii ) [1, 3] Number of suspensions in a critical section (ζ ii,jj ) [1, 2] 23/26 * The Schedulability Test Collection and Toolkit (SchedCAT).

24 Schedulability w.r.t. the Percentage of GPU-using Tasks Period range [30, 500] ms Hybrid MPCP outperforms both the original MPCP and LP-based FMLP+ 24/26

25 Schedulability w.r.t. the Percentage of GPU-using Tasks Period range [300, 500] ms Neither hybrid MPCP nor LP-based FMLP+ dominates the other Hybrid MPCP analysis is over 100x faster than LP-based FMLP+ 25/26

26 Conclusions Suspension-based MPCP Motivated by the limitations of the busy-waiting synchronization-based approach Implementation on a real-world embedded platform Significant improvement over the busy-waiting approach Very competitive with and often outperforms LP-based FMLP+ 100x better runtime performance compared to LP-based analysis Future directions A detailed study of the suspension overhead trade-offs on modern platforms with accelerators Comparison with other synchronization protocols 26/26

27 Thank You Analytical Enhancements and Practical Insights for MPCP with Self-Suspensions Pratyush Patel *, Iljoo Baek *, Hyoseung Kim, Raj Rajkumar * ibaek@andrew.cmu.edu * Carnegie Mellon University University of California, Riverside

28 BACKUP SLIDES

29 Self-Suspension Implementation Lock Increase Priority Request GPU Wait GPU response CPU GPU Copy Data GPU Execution Global Lock POSIX pthread_mutex() Shared memory Priority Ceiling sched_setscheduler() Wake up Request GPU Wait GPU response Wake up Unlock Decrease Priority Callback 1 Copy Data Callback 2 CPU Suspension POSIX pthread_cond() GPU Execution Asynchronous functions ex) cudamemcpyasync() Stream and Callback 29/26

30 Experimental Setup NVIDIA TX2-4 ARM Cortex-A9 at 2GHz - 2 Denver at 2GHz - 1 GPU (256 Cores, Pascal Arch.) - Ubuntu CPU 6. 1 GPU 1 LC WZ AM1~5 CC ii GG ii TT ii = DD ii CPU Test LC WZ AM AM AM AM AM /26 * All times are in milliseconds (ms) 1 General Use Case 2 GPU Overload 3 Overhead Test

31 Suspension Overhead CPU-side overhead Suspension Context Switching CPU GPU Lock Busy-wait GPU req. Unlock GPU-side overhead Asynchronous Calls Callback functions CPU Lock (1) Suspend (3) (2) Unlock GPU req. GPU 200 μs GPU access time < the suspension overhead the busy-wait approach is better 31/26

32 Self-Suspension Implementation Increase Priority CPU GPU Global Lock Shared memory pthread_mutex() Lock Request GPU Copy Data Suspension a POSIX conditional variable Priority Ceiling Wait GPU response Wake up Request GPU Wait GPU response GPU Execution Callback 1 Copy Data CUDA-related Asynchronous CUDA functions ex) cudamemcpyasync() Stream and Callback Priority Ceiling sched_setscheduler() Wake up Unlock Decrease Priority Callback 2 32/26 new_pri = highest_pri + (cur_pri lowest_pri) + 1 Ex) Task 1: 80 (cpu) Task 2: 54 (gpu) -> 85 = 80 + (54-50) + 1 Task 3: 53 (gpu) -> 84 = 80 + (53-50) + 1 Task 4: 52 (gpu) -> 83 = 80 + (52-50) + 1 Task 6: 50 (gpu) -> 81 = 80 + (50-50) + 1

33 Task Response Time with Suspension Worst-Case Response Time (WW ii ) Time span between a request and the end of the request WW ii nn+1 = CC ii + GG ii + BB ii + Blocking Delay τ h hpppp(τ ii ) WW ii nn + WW h CC h TT h CC h CPU Core 1 CPU Core 2 τ h τ mm GPU req. τ Suspend ll GPU req. Suspend Preemption Delay by Higher priority task WW ii Deadline (DD ii ) = Schedulable GPU Task arrival CPU execution GPU execution Blocked segment Preempted segment 33/30

34 Schedulability Experiments Purpose: To explore the impact of the different approaches on task schedulability Schedulability: How many taskets are schedulable? CC ii, GG ii, DD ii, TT ii, η ii Taskset 1 = {τ 1, τ 2.τ xx }.. Taskset n = {τ 1, τ 2. τ xx } Schedulability (%) 34/30 100% 0% Parameter Variation % of GPU-using Tasks % of CPU Utilization

35 Blocking Definition A task τ i is said to be blocked If a local task τ i with a lower base priority is scheduled while of τ i is pending. If any task τ k has locked the resource that τ i is waiting for. Blocking (1) Blocking (2) Blocking (1) Blocking (2) Core 1 τ 1 Core 2 τ 2 Core 1 τ 3 Higher-Priority Task preemption

36 Blocking Definition Direct Blocking (DB) Core 1 τ ii DB is incurred when any task τ k has locked the resource that τ i is waiting for. Core 2 τ kk Prioritized Blocking (PB) is incurred when lower-priority tasks executing with priority ceilings preempt the CPU execution of τ i Core 1 Core 1 τ ii τ ll Indirect Blocking (IB) is incurred when a task τ x accessing a resource with a higher priority ceiling preempts the execution of τ j, which is holding the resource that τ i is waiting for. Core 1 Core 2 τ ii τ jj DB IB DB R1 Core 2 τ xx R2 36/26

37 Task Response Time with Suspension Worst-case response time under partition fixed-priority scheduling WW ii = CC ii + GG ii + BB ii + τ h hpppp(τii ) II ii τ ii II ii τ ii = αα ii,h EE h Worst-case CPU execution time of τ ii Worst-case execution time of τ ii Upper bound on the maximum synchronization-based blocking Worst-case preemption time due to higher priority tasks on the same CPU αα ii,h = WW ii + WW h EE h TT h Back-to-back execution effect Maximum instances of τ h released during the execution of τ ii 37/26

38 Blocking Analysis Direct Blocking (DB) is incurred when any task τ k has locked the resource that τ i is waiting for. Core 1 τ ii DB Core 2 τ kk DB Request-Driven (RD) Approach BB ii dddd = dddd 0<jj η ii BB ii,jj Critical sections per job of τ ii Worst-case response time of k th critical section of τ ll ββ ii,jj,h HH h,kk 0<kk η h τ h hpp(τ ii ) BB dddd ii,jj = max ( HH ll,kk ) + τ ll llll(τ ii ) RR τ ii,jj =RR τ ll,kk τ h hpp(τ ii ) RR τ ii,jj =RR τ h,kk ββ ii,jj,h HH h,kk ββ ii,jj,h = BB ii,jj dddd + WW h EE h TT h Maximum blocking from lower priority tasks Maximum blocking from higher priority tasks Maximum instances of τ h released dddd during the blocking duration BB ii,jj 38/26

39 Blocking Analysis Direct Blocking (DB) is incurred when any task τ k has locked the resource that τ i is waiting for. Core 1 τ ii DB Core 2 τ kk DB Job-Driven (JD) Approach iff. RR τ ll,kk = RR τ ii Maximum number of times τ ii accesses resource RR rr 0<jj η ii τ ll llll(τ ii ) max(hh ll,kk ) 0<kk η h αα ii,h HH h,kk 0<kk η h τ h hpp(τ ii ) iff. RR τ h,kk = RR τ ii BB ii ddjj = η ii,rr max (HH ll,kk ) + τ ll llll τ ii rr RR τ ii RR τ ll,kk =rr τ h hpp(τ ii ) RR τ h,kk RR τ ii αα ii,h HH h,kk αα ii,h = WW ii + WW h EE h TT h Maximum direct blocking caused by lower-priority tasks = analogous to the RD approach Direct blocking caused by each higher-priority task critical section that uses the requested resource Maximum instances of τ h released during the execution of τ ii 39/26

40 Blocking Analysis Direct Blocking (DB) is incurred when any task τ k has locked the resource that τ i is waiting for. Core 1 τ ii DB Core 2 τ kk DB Hybrid Approach BB ii ddmm = BBii ddddll + BB ii ddddh 1) Maximum direct higher-priority blocking αα ii,h = WW ii + WW h EE h ββ TT ii,jj,h = BB ii,jj dddd + WW h EE h h TT h BB ddddd ii = τ h hpp(τ ii ) RR τ h,kk =RR τ ii δδ ii,h HH h,kk δδ ii,h = min αα ii,h, 0<jj η ii RR τ h,kk RR τ ii ββ ii,jj,h Worst-case response time of k th critical section of τ h Maximum cumulative number of interfering requests by τ h to the resources accessed by critical sections of τ ii Maximum instances of τ h released during the execution of τ ii Maximum instances of τ h released during the dddd blocking duration BB ii,jj 40/26

41 Blocking Analysis Direct Blocking (DB) is incurred when any task τ k has locked the resource that τ i is waiting for. Core 1 τ ii DB Core 2 τ kk DB Hybrid Approach ddmm BB ii = ddddll ddddh BBii + BB ii 2) Maximum direct lower-priority blocking The k th longest critical section that access resource RR jj and belong to llll τ ii Maximum number of times τ ii accesses resource RR jj BB ii ddddll = 0<jj gg 0<kk QQ ii,jj ψ ii,jj,kk LL ii,jj,kk ψ ii,jj,kk = min η ii,jj 0<tt<kk ψ ii,jj,tt, θθ ii,xxii,jj,kk θθ ii,ll = WW ii + DD ll EE ll TT ll Maximum number of times that the k th longest critical section can block τ ii QQ ii,jj = HH ll,xx τ ll llll τ ii RR τ ll,kk = rr A set contains the worst-case execution times of all the critical sections that access resource RR jj and belong to llll τ ii 41/26 Maximum number of release of a lower-priority τ ll during the execution of τ ii

42 Blocking Analysis: Example Task C i G i n i G i,1 G i,2 G i,3 T τ τ τ τ 1 τ 2 τ QQ 2,1 = HH ll,xx τ ll llll τ 2 RR τ ll,kk = 1 = 20, 10, 5 LL 2,1,1 = 20 LL 2,1,2 = 10 LL 2,1,3 = 5 θθ 2,3 = 1 ψ 2,1,1 = min ψ 2,1,2 = min η 2,1 ψ 2,1,tt, θθ 2,3 = min 3, 1 = 3 0<tt<1 η 2,1 ψ 2,1,tt, θθ 2,3 = min 0, 1 = 0 0<tt<2 42/26

43 Nvidia TX2

44 Nvidia TX2 NVIDIA Jetson TX1 NVIDIA Jetson TX2 CPU ARM Cortex-A GHz ARM Cortex-A57 2GHz + NVIDIA Denver2 2GHz GPU 256-core 998MHz 256-core 1300MHz Memory 4GB 64-bit 1600MHz 25.6 GB/s Storage 16GB emmc GB emmc 5.1 8GB 128-bit 1866Mhz 59.7 GB/s Encoder* 4Kp30, (2x) 1080p60 4Kp60, (3x) 4Kp30, (8x) 1080p30 Decoder* 4Kp60, (4x) 1080p60 (2x) 4Kp60 Camera 12 lanes MIPI CSI Gb/s per lane 1400 megapixels/sec ISP Display Wireless a/b/g/n/ac Mbps Bluetooth 4.0 Ethernet 12 lanes MIPI CSI Gb/sec per lane 1400 megapixels/sec ISP 2x HDMI 2.0 / DP 1.2 / edp 1.2 2x MIPI DSI 10/100/1000 BASE-T Ethernet USB USB USB a/b/g/n/ac Mbps Bluetooth 4.1 PCIe Gen x1 Gen or CAN Not supported Dual CAN bus controller Misc I/O Socket UART, SPI, I2C, I2S, GPIOs 400-pin Samtec board-to-board connector, 50x87mm Thermals -25 C to 80 C Power 10W 7.5W Price $299 at 1K units $399 at 1K units 44/26

45 Nvidia TX2 SM is limited to 2,048 per SM. Shared memory usage is limited to 64KB per SM and 48KB per block. the total number of threads each block can use is limited to 1,024. Each thread can use up to 255 registers A block can use up to 32,768 registers (regardless of its thread count). Additionally, there is a limit of 65,536 registers in total on each SM. 45/26