Matthew Piccoli University of Pennsylvania

Matthew Piccoli University of Pennsylvania

Base Controller Lots of deprecated code Separate odometry from controller Use common basekinematics class Continuously updated throughout the summer

Choices Safe teleop (base) Velocity control Goal control Closed loop grasping Using fingertip sensors Cart pushing/trailer pulling

Safe Teleop Two methods: Velocity control The path of the commanded velocity from the joystick is projected forward If path crosses obstacle, linearly decrease speed with distance Goal control The location of the goal is controlled by the joystick Move_base or move_base_local plans to that goal

Closed Loop Grasping Initial goals: Identify objects Grasp delicate objects (like eggs)

Closed Loop Grasping Initial goals: Identify objects Grasp delicate objects (like eggs) Squishy ball Wood block

Impulse from motor momentum!

Closed Loop Grasping Initial goals: Grasp delicate objects (like eggs) Identify objects Need to prevent force spike Force spike from impulse Need to reduce impulse Impulse from momentum Need to reduce momentum Momentum from velocity (motors/materials) Need to reduce velocity

Closed Loop Grasping Initial goals: Grasp delicate objects (like eggs) Identify objects Velocity controller Will attempt to go through object at specified velocity Need to switch to a controller that won t go through the object, but will hold onto it Effort controller Need to switch at impact with object (on both fingertips) Can use fingertip sensors or change in current to motor

Closed Loop Grasping Open loop Initial goals: Grasp delicate objects (like eggs) Identify objects Closed loop

Closed Loop Grasping Initial goals: Grasp delicate objects (like eggs) Identify objects Things we can get from the controller: First contact location Peak force location Steady state location Peak force Steady state force

Classification Output (Weka) Odwalla vs Naked vs Can vs Water J48 pruned tree ------------------ first_contact_distance <= 0.059488 first_contact_distance <= 0.057874: naked (38.0) first_contact_distance > 0.057874 df/dt <= 17278.83919: naked (4.0) df/dt > 17278.83919 peak_force_distance <= 0.051701 first_contact_distance <= 0.058224: naked (4.0) first_contact_distance > 0.058224: odwalla (19.0/2.0) peak_force_distance > 0.051701: odwalla (43.0) first_contact_distance > 0.059488 fingertip_peak_force_right <= 9437: water (70.0/1.0) fingertip_peak_force_right > 9437 dx/dt <= 0.008679: can (42.0) dx/dt > 0.008679: water (9.0/1.0) Number of Leaves : 8 Size of the tree : 15 Time taken to build model: 0.01 seconds === Stratified cross-validation === === Summary === Correctly Classified Instances 215 93.8865 % Incorrectly Classified Instances 14 6.1135 % Kappa statistic 0.9167 Mean absolute error 0.0378 Root mean squared error 0.1729 Relative absolute error 10.2468 % Root relative squared error 40.2635 % Total Number of Instances 229 === Detailed Accuracy By Class === TP Rate FP Rate Precision Recall F-Measure ROC Area Class 0.967 0.047 0.879 0.967 0.921 0.963 odwalla 0.833 0.006 0.976 0.833 0.899 0.946 naked 0.932 0.005 0.976 0.932 0.953 0.962 can 0.987 0.026 0.95 0.987 0.968 0.972 water Weighted Avg. 0.939 0.023 0.942 0.939 0.938 0.962 === Confusion Matrix === a b c d <-- classified as 58 1 0 1 a = odwalla 8 40 0 0 b = naked 0 0 41 3 c = can 0 0 1 76 d = water 93.8865 % a b c d <-- classified as 58 1 0 1 a = odwalla 8 40 0 0 b = naked 0 0 41 3 c = can 0 0 1 76 d = water

Closed Loop Grasping Goals: Grasp delicate objects (like eggs) Identify objects Identify object states Identify fruit ripeness

Classification Output (Weka) Odwalla vs Naked vs Can vs Water and Open vs Closed and Full vs Empty J48 pruned tree ------------------ === Stratified cross-validation === === Summary === Correctly Classified Instances 116 50.655 % Incorrectly Classified Instances 113 49.345 % Kappa statistic 0.4702 Mean absolute error 0.0662 Root mean squared error 0.2367 Relative absolute error 53.3184 % Root relative squared error 94.967 % Total Number of Instances 229 === Detailed Accuracy By Class === TP Rate FP Rate Precision Recall F-Measure ROC Area Class 1 0.005 0.938 1 0.968 0.998 odwallafullclosed 0.733 0.033 0.611 0.733 0.667 0.855 odwallafullopen 0.6 0.014 0.75 0.6 0.667 0.885 odwallaemptyopen 0.643 0.019 0.692 0.643 0.667 0.813 nakedfullclosed 0.286 0.023 0.444 0.286 0.348 0.689 nakedfullopen 0.7 0.037 0.467 0.7 0.56 0.925 nakedemptyclosed 0.6 0.018 0.6 0.6 0.6 0.791 nakedemptyopen 0.467 0.033 0.5 0.467 0.483 0.78 odwallaemptyclosed 1 0.005 0.933 1 0.966 0.998 canfullclosed 0.6 0.028 0.6 0.6 0.6 0.847 canfullopen 0.467 0.042 0.438 0.467 0.452 0.767 canemptyopen 0.278 0.062 0.278 0.278 0.278 0.641 waterfullclosed 0.158 0.086 0.143 0.158 0.15 0.709 waterfullopen 0.15 0.038 0.273 0.15 0.194 0.597 wateremptyopen 0.35 0.091 0.269 0.35 0.304 0.704 wateremptyclosed Weighted Avg. 0.507 0.039 0.505 0.507 0.501 0.787 === Confusion Matrix === a b c d e f g h i j k l m n o <-- classified as 15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 a = odwallafullclosed 0 11 0 0 0 0 0 3 0 0 0 1 0 0 0 b = odwallafullopen 0 0 9 0 0 3 1 2 0 0 0 0 0 0 0 c = odwallaemptyopen 1 2 0 9 2 0 0 0 0 0 0 0 0 0 0 d = nakedfullclosed 0 0 2 4 4 1 1 2 0 0 0 0 0 0 0 e = nakedfullopen 0 0 0 0 1 7 2 0 0 0 0 0 0 0 0 f = nakedemptyclosed 0 0 0 0 1 3 6 0 0 0 0 0 0 0 0 g = nakedemptyopen 0 5 1 0 1 1 0 7 0 0 0 0 0 0 0 h = odwallaemptyclosed 0 0 0 0 0 0 0 0 14 0 0 0 0 0 0 i = canfullclosed 0 0 0 0 0 0 0 0 0 9 6 0 0 0 0 j = canfullopen 0 0 0 0 0 0 0 0 0 4 7 2 1 0 1 k = canemptyopen 0 0 0 0 0 0 0 0 1 2 0 5 4 1 5 l = waterfullclosed 0 0 0 0 0 0 0 0 0 0 1 5 3 4 6 m = waterfullopen 0 0 0 0 0 0 0 0 0 0 0 1 9 3 7 n = wateremptyopen 0 0 0 0 0 0 0 0 0 0 2 4 4 3 7 o = wateremptyclosed 50.655 % 0 0 0 0 0 0 0 0 1 2 0 5 4 1 5 l = waterfullclosed 0 0 0 0 0 0 0 0 0 0 1 5 3 4 6 m = waterfullopen 0 0 0 0 0 0 0 0 0 0 0 1 9 3 7 n = wateremptyopen 0 0 0 0 0 0 0 0 0 0 2 4 4 3 7 o = wateremptyclosed

Classification Output (Weka) Odwalla vs Naked vs Can and Open J48 pruned treevs Closed and Full vs Empty ------------------ === Stratified cross-validation === === Summary === Correctly Classified Instances 99 65.1316 % Incorrectly Classified Instances 53 34.8684 % Kappa statistic 0.6155 Mean absolute error 0.067 Root mean squared error 0.2413 Relative absolute error 40.5616 % Root relative squared error 83.9145 % Total Number of Instances 152 === Detailed Accuracy By Class === TP Rate FP Rate Precision Recall F-Measure ROC Area Class 0.933 0.022 0.824 0.933 0.875 0.958 odwallafullclosed 0.667 0.044 0.625 0.667 0.645 0.846 odwallafullopen 0.667 0.058 0.556 0.667 0.606 0.852 odwallaemptyopen 0.5 0.029 0.636 0.5 0.56 0.76 nakedfullclosed 0.429 0.036 0.545 0.429 0.48 0.705 nakedfullopen 0.6 0.028 0.6 0.6 0.6 0.925 nakedemptyclosed 0.7 0.014 0.778 0.7 0.737 0.842 nakedemptyopen 0.467 0.058 0.467 0.467 0.467 0.702 odwallaemptyclosed 0.929 0 1 0.929 0.963 0.964 canfullclosed 0.667 0.051 0.588 0.667 0.625 0.809 canfullopen 0.6 0.044 0.6 0.6 0.6 0.926 canemptyopen Weighted Avg. 0.651 0.036 0.653 0.651 0.649 0.843 === Confusion Matrix === a b c d e f g h i j k <-- classified as 14 1 0 0 0 0 0 0 0 0 0 a = odwallafullclosed 0 10 0 0 0 0 0 4 0 0 1 b = odwallafullopen 0 0 10 0 0 2 0 3 0 0 0 c = odwallaemptyopen 3 0 0 7 3 0 0 1 0 0 0 d = nakedfullclosed 0 1 4 3 6 0 0 0 0 0 0 e = nakedfullopen 0 0 1 0 1 6 2 0 0 0 0 f = nakedemptyclosed 0 0 0 0 1 2 7 0 0 0 0 g = nakedemptyopen 0 4 3 1 0 0 0 7 0 0 0 h = odwallaemptyclosed 0 0 0 0 0 0 0 0 13 1 0 i = canfullclosed 0 0 0 0 0 0 0 0 0 10 5 j = canfullopen 0 0 0 0 0 0 0 0 0 6 9 k = canemptyopen 65.1316 % Without Water

Classification Output (Weka) Open vs Closed and Full vs Empty J48 pruned tree ------------------ === Stratified cross-validation === === Summary === Correctly Classified Instances 115 75.6579 % Incorrectly Classified Instances 37 24.3421 % Kappa statistic 0.7316 Mean absolute error 0.0449 Root mean squared error 0.1978 Relative absolute error 27.152 % Root relative squared error 68.7782 % Total Number of Instances 152 === Detailed Accuracy By Class === TP Rate FP Rate Precision Recall F-Measure ROC Area Class 1 0.015 0.882 1 0.938 0.993 odwallafullclosed 0.8 0.007 0.923 0.8 0.857 0.928 odwallafullopen 0.8 0.044 0.667 0.8 0.727 0.911 odwallaemptyopen 0.857 0.043 0.667 0.857 0.75 0.936 nakedfullclosed 0.714 0.014 0.833 0.714 0.769 0.915 nakedfullopen 0.7 0.007 0.875 0.7 0.778 0.946 nakedemptyclosed 0.8 0.014 0.8 0.8 0.8 0.946 nakedemptyopen 0.467 0.036 0.583 0.467 0.519 0.835 odwallaemptyclosed 0.929 0 1 0.929 0.963 0.964 canfullclosed 0.667 0.051 0.588 0.667 0.625 0.809 canfullopen 0.6 0.036 0.643 0.6 0.621 0.931 canemptyopen Weighted Avg. 0.757 0.025 0.763 0.757 0.755 0.917 === Confusion Matrix === a b c d e f g h i j k <-- classified as 15 0 0 0 0 0 0 0 0 0 0 a = odwallafullclosed 0 12 0 0 0 0 0 3 0 0 0 b = odwallafullopen 1 0 12 0 0 0 0 2 0 0 0 c = odwallaemptyopen 0 0 0 12 2 0 0 0 0 0 0 d = nakedfullclosed 0 0 0 4 10 0 0 0 0 0 0 e = nakedfullopen 0 0 0 1 0 7 2 0 0 0 0 f = nakedemptyclosed 0 0 0 1 0 1 8 0 0 0 0 g = nakedemptyopen 1 1 6 0 0 0 0 7 0 0 0 h = odwallaemptyclosed 0 0 0 0 0 0 0 0 13 1 0 i = canfullclosed 0 0 0 0 0 0 0 0 0 10 5 j = canfullopen 0 0 0 0 0 0 0 0 0 6 9 k = canemptyopen 75.6579 % Without Water and Knowing Object

Classification Output (Weka) Open vs Closed and Full vs Empty J48 pruned tree ------------------ === Stratified cross-validation === === Summary === Correctly Classified Instances 45 75 % Incorrectly Classified Instances 15 25 % Kappa statistic 0.6667 Mean absolute error 0.0468 Root mean squared error 0.2024 Relative absolute error 33.0524 % Root relative squared error 77.0029 % Total Number of Instances 60 === Detailed Accuracy By Class === TP Rate FP Rate Precision Recall F-Measure ROC Area Class 1 0.044 0.882 1 0.938 0.978 odwallafullclosed 0.667 0.044 0.833 0.667 0.741 0.896 odwallafullopen 0.733 0.089 0.733 0.733 0.733 0.852 odwallaemptyopen 0 0 0 0 0? nakedfullclosed 0 0 0 0 0? nakedfullopen 0 0 0 0 0? nakedemptyclosed 0 0 0 0 0? nakedemptyopen 0.6 0.156 0.563 0.6 0.581 0.81 odwallaemptyclosed 0 0 0 0 0? canfullclosed 0 0 0 0 0? canfullopen 0 0 0 0 0? canemptyopen Weighted Avg. 0.75 0.083 0.753 0.75 0.748 0.884 === Confusion Matrix === a b c d e f g h i j k <-- classified as 15 0 0 0 0 0 0 0 0 0 0 a = odwallafullclosed 1 10 0 0 0 0 0 4 0 0 0 b = odwallafullopen 1 0 11 0 0 0 0 3 0 0 0 c = odwallaemptyopen 0 0 0 0 0 0 0 0 0 0 0 d = nakedfullclosed 0 0 0 0 0 0 0 0 0 0 0 e = nakedfullopen 0 0 0 0 0 0 0 0 0 0 0 f = nakedemptyclosed 0 0 0 0 0 0 0 0 0 0 0 g = nakedemptyopen 0 2 4 0 0 0 0 9 0 0 0 h = odwallaemptyclosed 0 0 0 0 0 0 0 0 0 0 0 i = canfullclosed 0 0 0 0 0 0 0 0 0 0 0 j = canfullopen 0 0 0 0 0 0 0 0 0 0 0 k = canemptyopen 75 % Odwalla Only

We Can Do Better Modify the controller to: Give time to velocity = 0 Repeat grasp with different forces Give time to location of lowest force trial velocity = 0 when using larger forces Give time to spring back to first contact distance

Classification Output (Weka) Open vs Closed and Full vs Empty J48 pruned tree ------------------ With Improved Algorithm time3 <= 0.542 peak_force03 <= 14609 distance_steady3 <= 0.049578: emptyopen (19.0) distance_steady3 > 0.049578 steady_force03 <= 9386: emptyopen (4.0) steady_force03 > 9386: emptyclosed (25.0/1.0) peak_force03 > 14609: fullclosed (24.0) time3 > 0.542: fullopen (24.0) Number of Leaves : 5 Size of the tree : 9 Time taken to build model: 0 seconds === Stratified cross-validation === === Summary === Correctly Classified Instances 89 92.7083 % Incorrectly Classified Instances 7 7.2917 % Kappa statistic 0.9028 Mean absolute error 0.0411 Root mean squared error 0.1858 Relative absolute error 10.9362 % Root relative squared error 42.8508 % Total Number of Instances 96 === Detailed Accuracy By Class === TP Rate FP Rate Precision Recall F-Measure ROC Area Class 1 0.014 0.96 1 0.98 0.993 fullclosed 0.833 0.028 0.909 0.833 0.87 0.912 emptyopen 1 0.014 0.96 1 0.98 0.993 fullopen 0.875 0.042 0.875 0.875 0.875 0.929 emptyclosed Weighted Avg. 0.927 0.024 0.926 0.927 0.926 0.957 === Confusion Matrix === a b c d <-- classified as 24 0 0 0 a = fullclosed 0 20 1 3 b = emptyopen 0 0 24 0 c = fullopen 1 2 0 21 d = emptyclosed 92.7083 % Odwalla Only

Closed Loop Grasping Goals: Grasp delicate objects (like eggs) Identify objects Identify object states Identify fruit ripeness And that s as far as we ve gone Todo: Identify fruit ripeness Human study