idamnjanovic@6: %% DICTIONARY LEARNING FOR IMAGE DENOISING
idamnjanovic@6: %   This file contains an example of how SMALLbox can be used to test different
idamnjanovic@6: %   dictionary learning techniques in Image Denoising problem.
idamnjanovic@6: %   It calls generateImageDenoiseProblem that will let you to choose image,
idamnjanovic@6: %   add noise and use noisy image to generate training set for dictionary
idamnjanovic@6: %   learning.
idamnjanovic@6: %   Three dictionary learning techniques were compared:
idamnjanovic@6: %   -   KSVD - M. Elad, R. Rubinstein, and M. Zibulevsky, "Efficient
idamnjanovic@6: %              Implementation of the K-SVD Algorithm using Batch Orthogonal
idamnjanovic@6: %              Matching Pursuit", Technical Report - CS, Technion, April 2008.
idamnjanovic@6: %   -   KSVDS - R. Rubinstein, M. Zibulevsky, and M. Elad, "Learning Sparse
idamnjanovic@6: %               Dictionaries for Sparse Signal Approximation", Technical
idamnjanovic@6: %               Report - CS, Technion, June 2009.
idamnjanovic@6: %   -   SPAMS - J. Mairal, F. Bach, J. Ponce and G. Sapiro. Online
idamnjanovic@6: %               Dictionary Learning for Sparse Coding. International
idamnjanovic@6: %               Conference on Machine Learning,Montreal, Canada, 2009
idamnjanovic@6: %
idamnjanovic@6: %
idamnjanovic@6: % Ivan Damnjanovic 2010
idamnjanovic@6: %%
idamnjanovic@6: 
idamnjanovic@6: clear;
idamnjanovic@6: 
idamnjanovic@6: %   If you want to load the image outside of generateImageDenoiseProblem
idamnjanovic@6: %   function uncomment following lines. This can be useful if you want to
idamnjanovic@6: %   denoise more then one image for example.
idamnjanovic@6: 
idamnjanovic@6: % TMPpath=pwd;
idamnjanovic@6: % FS=filesep;
idamnjanovic@6: % [pathstr1, name, ext, versn] = fileparts(which('SMALLboxSetup.m'));
idamnjanovic@6: % cd([pathstr1,FS,'data',FS,'images']);
idamnjanovic@6: % [filename,pathname] = uigetfile({'*.png;'},'Select a file containin pre-calculated notes');
idamnjanovic@6: % [pathstr, name, ext, versn] = fileparts(filename);
idamnjanovic@6: % test_image = imread(filename);
idamnjanovic@6: % test_image = double(test_image);
idamnjanovic@6: % cd(TMPpath);
idamnjanovic@6: % SMALL.Problem.name=name;
idamnjanovic@6: 
idamnjanovic@6: 
idamnjanovic@6: % Defining Image Denoising Problem as Dictionary Learning
idamnjanovic@6: % Problem. As an input we set the number of training patches.
idamnjanovic@6: 
idamnjanovic@6: SMALL.Problem = generateImageDenoiseProblem('', 40000);
idamnjanovic@6: 
idamnjanovic@6: 
idamnjanovic@6: %%
idamnjanovic@6: %   Use KSVD Dictionary Learning Algorithm to Learn overcomplete dictionary
idamnjanovic@6: 
idamnjanovic@6: %   Initialising Dictionary structure
idamnjanovic@6: %   Setting Dictionary structure fields (toolbox, name, param, D and time)
idamnjanovic@6: %   to zero values
idamnjanovic@6: 
idamnjanovic@6: SMALL.DL(1)=SMALL_init_DL();
idamnjanovic@6: 
idamnjanovic@6: % Defining the parameters needed for dictionary learning
idamnjanovic@6: 
idamnjanovic@6: SMALL.DL(1).toolbox = 'KSVD';
idamnjanovic@6: SMALL.DL(1).name = 'ksvd';
idamnjanovic@6: 
idamnjanovic@6: %   Defining the parameters for KSVD
idamnjanovic@6: %   In this example we are learning 256 atoms in 20 iterations, so that
idamnjanovic@6: %   every patch in the training set can be represented with target error in
idamnjanovic@6: %   L2-norm (EData)
idamnjanovic@6: %   Type help ksvd in MATLAB prompt for more options.
idamnjanovic@6: 
idamnjanovic@6: Edata=sqrt(prod(SMALL.Problem.blocksize)) * SMALL.Problem.sigma * SMALL.Problem.gain;
idamnjanovic@6: SMALL.DL(1).param=struct(...
idamnjanovic@6:     'Edata', Edata,...
idamnjanovic@6:     'initdict', SMALL.Problem.initdict,...
idamnjanovic@6:     'dictsize', SMALL.Problem.p,...
idamnjanovic@6:     'iternum', 20,...
idamnjanovic@6:     'memusage', 'high');
idamnjanovic@6: 
idamnjanovic@6: %   Learn the dictionary
idamnjanovic@6: 
idamnjanovic@6: SMALL.DL(1) = SMALL_learn(SMALL.Problem, SMALL.DL(1));
idamnjanovic@6: 
idamnjanovic@6: %   Set SMALL.Problem.A dictionary
idamnjanovic@6: %   (backward compatiblity with SPARCO: solver structure communicate
idamnjanovic@6: %   only with Problem structure, ie no direct communication between DL and
idamnjanovic@6: %   solver structures)
idamnjanovic@6: 
idamnjanovic@6: SMALL.Problem.A = SMALL.DL(1).D;
idamnjanovic@6: 
idamnjanovic@6: 
idamnjanovic@6: %%
idamnjanovic@6: %   Initialising solver structure
idamnjanovic@6: %   Setting solver structure fields (toolbox, name, param, solution,
idamnjanovic@6: %   reconstructed and time) to zero values
idamnjanovic@6: 
idamnjanovic@6: SMALL.solver(1)=SMALL_init_solver;
idamnjanovic@6: 
idamnjanovic@6: % Defining the parameters needed for image denoising
idamnjanovic@6: 
idamnjanovic@6: SMALL.solver(1).toolbox='ompbox';
idamnjanovic@6: SMALL.solver(1).name='ompdenoise';
idamnjanovic@6: 
idamnjanovic@6: %   Denoising the image - SMALL_denoise function is similar to SMALL_solve,
idamnjanovic@6: %   but backward compatible with KSVD definition of denoising
idamnjanovic@6: 
idamnjanovic@6: SMALL.solver(1)=SMALL_denoise(SMALL.Problem, SMALL.solver(1));
idamnjanovic@6: 
idamnjanovic@6: %%
idamnjanovic@6: % Use KSVDS Dictionary Learning Algorithm to denoise image
idamnjanovic@6: 
idamnjanovic@6: %   Initialising solver structure
idamnjanovic@6: %   Setting solver structure fields (toolbox, name, param, solution,
idamnjanovic@6: %   reconstructed and time) to zero values
idamnjanovic@6: 
idamnjanovic@6: SMALL.DL(2)=SMALL_init_DL();
idamnjanovic@6: 
idamnjanovic@6: % Defining the parameters needed for dictionary learning
idamnjanovic@6: 
idamnjanovic@6: SMALL.DL(2).toolbox = 'KSVDS';
idamnjanovic@6: SMALL.DL(2).name = 'ksvds';
idamnjanovic@6: 
idamnjanovic@6: %   Defining the parameters for KSVDS
idamnjanovic@6: %   In this example we are learning 256 atoms in 20 iterations, so that
idamnjanovic@6: %   every patch in the training set can be represented with target error in
idamnjanovic@6: %   L2-norm (EDataS). We also impose "double sparsity" - dictionary itself
idamnjanovic@6: %   has to be sparse in the given base dictionary (Tdict - number of
idamnjanovic@6: %   nonzero elements per atom).
idamnjanovic@6: %   Type help ksvds in MATLAB prompt for more options.
idamnjanovic@6: 
idamnjanovic@6: EdataS=sqrt(prod(SMALL.Problem.blocksize)) * SMALL.Problem.sigma * SMALL.Problem.gain;
idamnjanovic@6: SMALL.DL(2).param=struct(...
idamnjanovic@6:     'Edata', EdataS, ...
idamnjanovic@6:     'Tdict', 6,...
idamnjanovic@6:     'stepsize', 1,...
idamnjanovic@6:     'dictsize', SMALL.Problem.p,...
idamnjanovic@6:     'iternum', 20,...
idamnjanovic@6:     'memusage', 'high');
idamnjanovic@6: SMALL.DL(2).param.initA = speye(SMALL.Problem.p);
idamnjanovic@6: SMALL.DL(2).param.basedict{1} = odctdict(8,16);
idamnjanovic@6: SMALL.DL(2).param.basedict{2} = odctdict(8,16);
idamnjanovic@6: 
idamnjanovic@6: % Learn the dictionary
idamnjanovic@6: 
idamnjanovic@6: SMALL.DL(2) = SMALL_learn(SMALL.Problem, SMALL.DL(2));
idamnjanovic@6: 
idamnjanovic@6: %   Set SMALL.Problem.A dictionary and SMALL.Problem.basedictionary
idamnjanovic@6: %   (backward compatiblity with SPARCO: solver structure communicate
idamnjanovic@6: %   only with Problem structure, ie no direct communication between DL and
idamnjanovic@6: %   solver structures)
idamnjanovic@6: 
idamnjanovic@6: SMALL.Problem.A = SMALL.DL(2).D;
idamnjanovic@6: SMALL.Problem.basedict{1} = SMALL.DL(2).param.basedict{1};
idamnjanovic@6: SMALL.Problem.basedict{2} = SMALL.DL(2).param.basedict{2};
idamnjanovic@6: 
idamnjanovic@6: %%
idamnjanovic@6: %   Initialising solver structure
idamnjanovic@6: %   Setting solver structure fields (toolbox, name, param, solution,
idamnjanovic@6: %   reconstructed and time) to zero values
idamnjanovic@6: 
idamnjanovic@6: SMALL.solver(2)=SMALL_init_solver;
idamnjanovic@6: 
idamnjanovic@6: % Defining the parameters needed for image denoising
idamnjanovic@6: 
idamnjanovic@6: SMALL.solver(2).toolbox='ompsbox';
idamnjanovic@6: SMALL.solver(2).name='ompsdenoise';
idamnjanovic@6: 
idamnjanovic@6: %   Denoising the image - SMALL_denoise function is similar to SMALL_solve,
idamnjanovic@6: %   but backward compatible with KSVD definition of denoising
idamnjanovic@6: %   Pay attention that since implicit base dictionary is used, denoising
idamnjanovic@6: %   can be much faster then using explicit dictionary in KSVD example.
idamnjanovic@6: 
idamnjanovic@6: SMALL.solver(2)=SMALL_denoise(SMALL.Problem, SMALL.solver(2));
idamnjanovic@6: 
idamnjanovic@6: %%
idamnjanovic@6: %   Use SPAMS Online Dictionary Learning Algorithm
idamnjanovic@6: %   to Learn overcomplete dictionary (Julien Mairal 2009)
idamnjanovic@6: %   (If you have not installed SPAMS please comment the following two cells)
idamnjanovic@6: 
idamnjanovic@6: %   Initialising Dictionary structure
idamnjanovic@6: %   Setting Dictionary structure fields (toolbox, name, param, D and time)
idamnjanovic@6: %   to zero values
idamnjanovic@6: 
idamnjanovic@6: SMALL.DL(3)=SMALL_init_DL();
idamnjanovic@6: 
idamnjanovic@6: %   Defining fields needed for dictionary learning
idamnjanovic@6: 
idamnjanovic@6: SMALL.DL(3).toolbox = 'SPAMS';
idamnjanovic@6: SMALL.DL(3).name = 'mexTrainDL';
idamnjanovic@6: 
idamnjanovic@6: %   Type 'help mexTrainDL in MATLAB prompt for explanation of parameters.
idamnjanovic@6: 
idamnjanovic@6: SMALL.DL(3).param=struct(...
idamnjanovic@6:     'D', SMALL.Problem.initdict,...
idamnjanovic@6:     'K', SMALL.Problem.p,...
idamnjanovic@6:     'lambda', 2,...
idamnjanovic@6:     'iter', 200,...
idamnjanovic@6:     'mode', 3, ...
idamnjanovic@6:     'modeD', 0);
idamnjanovic@6: 
idamnjanovic@6: %   Learn the dictionary
idamnjanovic@6: 
idamnjanovic@6: SMALL.DL(3) = SMALL_learn(SMALL.Problem, SMALL.DL(3));
idamnjanovic@6: 
idamnjanovic@6: %   Set SMALL.Problem.A dictionary
idamnjanovic@6: %   (backward compatiblity with SPARCO: solver structure communicate
idamnjanovic@6: %   only with Problem structure, ie no direct communication between DL and
idamnjanovic@6: %   solver structures)
idamnjanovic@6: 
idamnjanovic@6: SMALL.Problem.A = SMALL.DL(3).D;
idamnjanovic@6: 
idamnjanovic@6: 
idamnjanovic@6: %%
idamnjanovic@6: %   Initialising solver structure
idamnjanovic@6: %   Setting solver structure fields (toolbox, name, param, solution,
idamnjanovic@6: %   reconstructed and time) to zero values
idamnjanovic@6: 
idamnjanovic@6: SMALL.solver(3)=SMALL_init_solver;
idamnjanovic@6: 
idamnjanovic@6: % Defining the parameters needed for denoising
idamnjanovic@6: 
idamnjanovic@6: SMALL.solver(3).toolbox='ompbox';
idamnjanovic@6: SMALL.solver(3).name='ompdenoise';
idamnjanovic@6: 
idamnjanovic@6: %   Denoising the image - SMALL_denoise function is similar to SMALL_solve,
idamnjanovic@6: %   but backward compatible with KSVD definition of denoising
idamnjanovic@6: 
idamnjanovic@6: SMALL.solver(3)=SMALL_denoise(SMALL.Problem, SMALL.solver(3));
idamnjanovic@6: 
idamnjanovic@6: %%
idamnjanovic@6: % Plot results and save midi files
idamnjanovic@6: 
idamnjanovic@6: % show results %
idamnjanovic@6: 
idamnjanovic@6: SMALL_ImgDeNoiseResult(SMALL);