Daniel@0: Daniel@0: Daniel@0: Daniel@0: Netlab Reference Manual olgd Daniel@0: Daniel@0: Daniel@0: Daniel@0:

olgd Daniel@0:

Daniel@0:

Daniel@0: Purpose Daniel@0:

Daniel@0: On-line gradient descent optimization. Daniel@0: Daniel@0:

Daniel@0: Description Daniel@0:

Daniel@0: [net, options, errlog, pointlog] = olgd(net, options, x, t) uses Daniel@0: on-line gradient descent to find a local minimum of the error function for the Daniel@0: network Daniel@0: net computed on the input data x and target values Daniel@0: t. A log of the error values Daniel@0: after each cycle is (optionally) returned in errlog, and a log Daniel@0: of the points visited is (optionally) returned in pointlog. Daniel@0: Because the gradient is computed on-line (i.e. after each pattern) Daniel@0: this can be quite inefficient in Matlab. Daniel@0: Daniel@0:

The error function value at final weight vector is returned Daniel@0: in options(8). Daniel@0: Daniel@0:

The optional parameters have the following interpretations. Daniel@0: Daniel@0:

options(1) is set to 1 to display error values; also logs error Daniel@0: values in the return argument errlog, and the points visited Daniel@0: in the return argument pointslog. If options(1) is set to 0, Daniel@0: then only warning messages are displayed. If options(1) is -1, Daniel@0: then nothing is displayed. Daniel@0: Daniel@0:

options(2) is the precision required for the value Daniel@0: of x at the solution. If the absolute difference between Daniel@0: the values of x between two successive steps is less than Daniel@0: options(2), then this condition is satisfied. Daniel@0: Daniel@0:

options(3) is the precision required of the objective Daniel@0: function at the solution. If the absolute difference between the Daniel@0: error functions between two successive steps is less than Daniel@0: options(3), then this condition is satisfied. Daniel@0: Both this and the previous condition must be Daniel@0: satisfied for termination. Note that testing the function value at each Daniel@0: iteration roughly halves the speed of the algorithm. Daniel@0: Daniel@0:

options(5) determines whether the patterns are sampled randomly Daniel@0: with replacement. If it is 0 (the default), then patterns are sampled Daniel@0: in order. Daniel@0: Daniel@0:

options(6) determines if the learning rate decays. If it is 1 Daniel@0: then the learning rate decays at a rate of 1/t. If it is 0 Daniel@0: (the default) then the learning rate is constant. Daniel@0: Daniel@0:

options(9) should be set to 1 to check the user defined gradient Daniel@0: function. Daniel@0: Daniel@0:

options(10) returns the total number of function evaluations (including Daniel@0: those in any line searches). Daniel@0: Daniel@0:

options(11) returns the total number of gradient evaluations. Daniel@0: Daniel@0:

options(14) is the maximum number of iterations (passes through Daniel@0: the complete pattern set); default 100. Daniel@0: Daniel@0:

options(17) is the momentum; default 0.5. Daniel@0: Daniel@0:

options(18) is the learning rate; default 0.01. Daniel@0: Daniel@0:

Daniel@0: Examples Daniel@0:

Daniel@0: The following example performs on-line gradient descent on an MLP with Daniel@0: random sampling from the pattern set. Daniel@0:

Daniel@0: 
Daniel@0: net = mlp(5, 3, 1, 'linear');
Daniel@0: options = foptions;
Daniel@0: options(18) = 0.01;
Daniel@0: options(5) = 1;
Daniel@0: net = olgd(net, options, x, t);
Daniel@0:

Daniel@0: Daniel@0: Daniel@0:

Daniel@0: See Also Daniel@0:

Daniel@0: graddesc

Daniel@0: Pages: Daniel@0: Index Daniel@0:

Daniel@0: