NEURAL NETWORKS REFERENCES: ARTIFICIAL INTELLIGENCE FOR GAMES "ARTIFICIAL INTELLIGENCE: A NEW SYNTHESIS HTTPS://MATTMAZUR.COM/2015/03/17/A- STEP- BY- STEP- BACKPROPAGATION- EXAMPLE/ "
HISTORY In the 70 s vs today variants (Multi-layer) Feed-forward (perceptron) Hebbian Recurrent
OVERVIEW Givens: Process: Using the Neural Network. Basically: a classifier.
EXAMPLE APPLICATION #1 Modeling a enemy AI. Supervised training Unsupervised training
EXAMPLE APPLICATION #2 (I.E. THE BORING LAB)
PERCEPTRONS Modeled after a single neuron. Components: Dendrites (Input) Axon (Output) Soma (Activation Function)
PERCEPTRON ACTIVATION FUNCTIONS Notation: II the vector of all input values. For us, input values are 0 or 1. WW the vector of weight values (same size as I) Positive, or negative, no limit. I usually initialize to -0.5 to +0.5. = II WW the total (weighted) input to the perceptron.
PERCEPTRON ACTIVATION FUNCTIONS, CONT. Square function: ff Σ = 1 iiii Σ δ 0 iiii Σ < δ Sigmoid [logistic] function (the one we ll use): 1 ff Σ = 1 + ee kk(σ δ) 1.2 1 0.8 0.6 0.4 0.2 0 0 0.07 0.14 0.21 0.28 0.35 0.42 0.49 0.56 0.63 0.7 0.77 0.84 0.91 0.98
EXAMPLES Two inputs, one output, k=5.0 AND OR WW = 0.5 0.5 δ=0.75 WW = 0.5 0.5 δ=0.75 Input Σ f(σ) [0 0] 0.0 0.023 [0 1] 0.5 0.223 [1 0] 0.5 0.223 [1 1] 1.0 0.7773 Input Σ f(σ) [0 0] 0.0 0.119 [0 1] 0.5 0.622 [1 0] 0.5 0.622 [1 1] 1.0 0.953 XOR Problem! Can t be done with a single perceptron O 0 Perceptron w 0 w 1 I 0 I 1
WHY NOT XOR? The first two examples are linearly separable WW 0 WW 1 W 2 =δ II = II 0 II 1 OUTPUT=1 OUTPUT=0 OOOOOOOOOOOO = 0 0 In higher dimensions, the dividing line is a hyper-plane, not a plane
AND AND OR 0.5 0.5 0 1 1 1 0 1 0.5 0.5 1 1 δ=0.75 δ=0.4 0 0 1 0 0 0 1 0 AND OR
XOR 0 1 1 1 0 0 1 0 You can t draw a line to separate the True s from the False s
MULTI-LAYER PERCEPTRON NETWORKS 0 1 1 1 0.5, 0.5 0.35 0 0 1 0 WW 00 = 00. 55 δ=0.35 00. 44 WW 11 = 00. 55 δ=0.35 00. 55 0 1 1 1 0.7 0 0 1 0 0.4, 0.6
XOR NNET O 0 Perceptron 1,0 <0.4> 0.5 0.5 Perceptron 0,0 <0.2> Perceptron 0,1 <0.3> 0.5 0.4 0.4 0.6 I 0 I 1
FEED FORWARD Feed Input to bottom of NNet Each Perceptron outputs its activation result to next layer as input I used a k=20 here Input Step# P(layer,#) Σ f(σ) [0,0] 1 P0,0 0.0 0.001 2 P0,1 0.0 0.001 3 P1,1 0.001 0.0003 [0,1] 1 P0,0 0.5 0.953 2 P0,1-0.5 4.14e-8 3 P1,1 0.476 0.821 [1, 0] 1 P0,0-0.5 4.14e-8 2 P0,1 0.47 0.953 3 P1,1 0.476 0.821 [1, 1] 1 P0,0 0.0 0.001 2 P0,1 0.0 0.001 3 P1,1 0.001 0.0003
TRAINING INTRO Feed Forward is using a Nnet. Training is creating a NNet. Error-correction procedure (single perceptron) o WWnnnnnn = WW + cc II ff EEEEEE o δδ = δδ cc ff EEEEEE BTW (Keith): the derivation of this formula involves partialderivatives
1.2 1 0.8 0.6 0.4 0.2 TRAINING INTRO, CONT. f f(1-f) WWWWWWWW = WW + cc II ff EEEEEE Err = dd oo 0 0 0.055 0.11 0.165 0.22 0.275 0.33 0.385 0.44 0.495 0.55 0.605 0.66 0.715 0.77 0.825 0.88 0.935 0.99 Observations
MY TRAINING RESULTS One perceptron, trying to learn ADD Initial weights and threshold = 0 total_error = sum of err 2 for all 4 cases (shuffled) Repeated for 1000 iterations 1.4 1.2 1 0.8 0.6 0.4 0.2 0 1 18 35 52 69 86 103 120 137 154 171 188 205 222 239 256 273 290 307 324 341 358 375 392 409 426 443 460 477 494 511 528 545 562 579 596 613 630 647 664 681 698 715 732 749 766 783 800 817 834 851 868 885 902 919 936 953 970 987
TRAINING (1 LAYER, M OUTPUTS) Similarities and differences EEEEEE = dddddddddddddd aaaaaaaaaaaa Correction update for i in range(num_perceptrons): WW ii += cc II ff ii EEEEEE ii δ = cc ff ii EEEEEE ii I found this useful: ttttttttll eeeeee = EEEEEE (i.e. vector-magnitude)
TRAINING (N LAYERS, M OUTPUTS) The most general case. For all perceptrons, calculate the following (in reverse (output to input)) BBBBBBBBBB ii,jj = nn kk=0 EEEEEE jj wwww jj kk BBBBBBBBBB ii+1,kk iiii llllllllll ii iiii aaaa oooooooooooo llllllllll eeeeeeee II SSSSSSSSSSSS ii = OOOOOOOOOOOO oooo llllllllll ii 1 Then to update a perceptron: WW ii,jj += cc SSSSSSSSSSSS ii ff ii,jj BBBBBBBBBB ii,jj iiii ii iiii ttttttt iiii ttttt bottommost llllllllll eeeeeeee δ = cc ff ii BBBBBBBBBB ii,jj Make sure you use the existing weights for blame calculations. Perhaps wait until the back-prop is done to update weights?
MY SUGGESTION Start with a Perceptron / NNet class Manually set up AND, OR, XOR, test feed-forward Learn AND Learn XOR Learn XOR w/ weird structure 2 input, 1 output hidden-size=[5, 3] Learn a challenging problem (i.e. the lab)