log audioDB.cpp @ 354:4871a3ed9e36 pre_api

age author description
Wed, 08 Oct 2008 11:06:11 +0000 mas01cr Quieten newer compilers with MORE CONST CORRECTNESS.
Fri, 05 Sep 2008 20:01:58 +0000 mas01mc Added new INDEX tests: LARGE_ADB, PATH PREFIX, cmdline and WS. Removed redundant feature PREFIX in audioDB.cpp. Moved tests/0050 to tests/0041 so INDEX tests are all in 0040-0049 range.
Fri, 05 Sep 2008 16:40:41 +0000 mas01mc fixed startServer() reporting of non-locatable database or index. Achieved by setting isServer=1 AFTER checking database and loading INDEX.
Fri, 05 Sep 2008 14:35:44 +0000 mas01mc gengetopt does not support multiple dependencies, so added test for database_given for INDEX command. Still cannot figure out why sometimes error(...) throws char* exception and SIG_ABORT
Tue, 02 Sep 2008 16:16:59 +0000 mas01mc Added command-line and WS methods to (LI)st key-(S)trings and si(Z)es of (T)racks --LISZT --lisztOffset offset --lisztLength len
Sun, 31 Aug 2008 08:45:22 +0000 mas01mc Added new tests for indexed query over WS. Fixed potential NULL string dereference in audioDB.cpp
Sat, 30 Aug 2008 14:33:24 +0000 mas01mc Added some extra logging to alert the user to which query is called
Sat, 30 Aug 2008 13:41:50 +0000 mas01mc Added WS support for Radius search by featureFileName via adb__sequenceQueryByKey(). Added relative path support for non-large ADB.
Thu, 21 Aug 2008 21:28:33 +0000 mas01mc Merge of branches/large_adb -r 514:524 onto the trunk. No conflicts. Added LARGE_ADB support. Turn on with --ntracks 20001 or greater. Use --adb_feature_root to locate feature files at QUERY time. A bug fix in LSH indexing that was incorrectly thresholding large numbers of shingles.
Thu, 21 Aug 2008 19:58:55 +0000 mas01mc Added --adb_root and --adb_feature_root support to server-side Web Services instance. Client does not have to know location of database and features on the server when server performs path substitutions with these arguments. large_adb
Thu, 21 Aug 2008 19:16:21 +0000 mas01mc Added --adb_feature_root=path filename prefix for locating LARGE_ADB features with relative paths at QUERY time. Also added convenience argument --adb_root=path prefix for -d database command option. large_adb
Wed, 20 Aug 2008 13:50:58 +0000 mas01mc working LARGE_ADB support. Activiate at creation time with -N --ntracks 20001 or greater, or with --datasize 1356 or greater, or both. LARGE_ADB blocks non-indexed QUERY. large_adb
Tue, 19 Aug 2008 20:27:15 +0000 mas01mc Added indexing support for O2_FLAG_LARGE_ADB. Tested on indexed query by features. No indexed query-by-key yet. No --lsh_exact yet. large_adb
Tue, 19 Aug 2008 15:50:26 +0000 mas01mc Ammended batchinsert and status to memory map featureFilenameTables, powerFileNameTable and timesFileNameTable for O2_FLAG_LARGE_ADB large_adb
Tue, 19 Aug 2008 14:38:39 +0000 mas01mc Added large_adb Support for batchinsert and status commands large_adb
Tue, 12 Aug 2008 01:21:44 +0000 mas01mc working SIIGRAPH08 version. Fixed powerTable mmap memory leak in WS calls (only showed up in big databases). Implements radius queries over WS with new wsdl file
Sat, 09 Aug 2008 15:46:42 +0000 mas01mc Check for index-exists on WS --load_index and exit if it does not. Revert unmap_tables because cleanup() did it fine, but there is a memory leak in ws_query calls.
Sat, 09 Aug 2008 15:17:01 +0000 mas01mc unmap tables on ws_query exit.
Sat, 09 Aug 2008 15:04:16 +0000 mas01mc Added sequencehop on Indexing to optionally reduce size of large indexes. Added cleanup() call on WS query exit to fix MMAP memory leak
Thu, 07 Aug 2008 01:53:38 +0000 mas01mc Added server-side loading of persistent index (LSH hashtables) via --load_index -d dbName -R radius -l sequenceLength. Queries using these parameters will lookup the memory-resident hashtable instead of loading one from disk.
Wed, 06 Aug 2008 21:23:14 +0000 mas01mc Added reporters radius search via Web Services. The ordering for radius search is degenerate if usingQueryPoint, because the query point can only be counted once. This behaviour is changed by specifying -n 1 to emulate the trackAveragingReporter ordering behaviour
Tue, 05 Aug 2008 13:38:36 +0000 mas01cr Wow, so many bugs in one short stanza of code. Really fix batchinsert
Fri, 01 Aug 2008 16:21:51 +0000 mas01mc added more informative status display for flags
Fri, 01 Aug 2008 15:48:40 +0000 mas01tc Corrected error message in audioDB line 332
Fri, 01 Aug 2008 15:33:00 +0000 mas01tc Fixed bug in audioDB.cpp line 331:
Fri, 01 Aug 2008 15:13:21 +0000 mas01mc set the default flag value to 'off' for --lsh_on_disk and made lsh_in_core the compliment of this value
Fri, 01 Aug 2008 15:04:31 +0000 mas01mc fixed file size allocation for FORMAT2 files. Made LSH index size() in bytes an unsigned long long. Changed the name of lsh_inCore flag to lsh_on_disk (to reverse the sense of the 'flag').
Tue, 29 Jul 2008 22:01:17 +0000 mas01mc Completed merge of lshlib back to current version of the trunk.
Mon, 07 Jul 2008 08:57:06 +0000 mas01cr Really finish with the sampling branch, this time merging all of it,
Wed, 02 Jul 2008 14:07:10 +0000 mas01cr Merge sampling branch (-r361:405, though I hope that the branch is now
Wed, 02 Jul 2008 13:53:23 +0000 mas01cr make the RNG a part of the audioDB object. sampling
Tue, 01 Jul 2008 09:12:40 +0000 mas01cr Merge trunk changes -r326:386 into audiodb-debian branch. audiodb-debian
Mon, 23 Jun 2008 09:49:22 +0000 mas01cr Let the user specify the number of samples to take on the command-line. sampling
Mon, 16 Jun 2008 11:14:21 +0000 mas01cr Actually use the sequenceLength -l parameter passed on the command-line sampling
Sat, 14 Jun 2008 17:13:26 +0000 mas01cr Add initial sampling of database distances. Zillions of FIXME comments sampling
Mon, 09 Jun 2008 19:20:39 +0000 mas01mc Added new query type "onetoonensequence" for matching each query sequence to the single closest match in the database.
Wed, 16 Apr 2008 09:59:43 +0000 mas01cr Commit patch deprecating the --size argument, replacing it with
Sun, 17 Feb 2008 14:56:02 +0000 mas01mc Added new query type: nsequence : this reports the n-nearest neighbours for each track in a sequence averaging query.
Fri, 14 Dec 2007 14:41:37 +0000 mas01cr Merge trunk changes -r290:313 into audiodb-debian branch. audiodb-debian
Thu, 13 Dec 2007 14:23:32 +0000 mas01cr Merge refactoring branch.
Wed, 12 Dec 2007 16:43:04 +0000 mas01cr VERB_LOG macro into audioDB.h refactoring
Wed, 05 Dec 2007 14:11:04 +0000 mas01cr Merge trunk changes -r275:289 into audiodb-debian branch audiodb-debian
Wed, 05 Dec 2007 14:06:21 +0000 mas01cr Merged trunk changes -r264:274 into audiodb-debian branch. audiodb-debian
Wed, 05 Dec 2007 13:52:51 +0000 mas01cr Fixes from refactoring branch:
Thu, 29 Nov 2007 16:31:09 +0000 mas01cr Deal with x86-64 OS X vs Linux structure padding ABI.
Wed, 28 Nov 2007 15:10:28 +0000 mas01cr Filewise refactor. refactoring
Tue, 27 Nov 2007 18:14:19 +0000 mas01cr Fix test 0034
Fri, 23 Nov 2007 15:53:37 +0000 mas01cr Merge trunk changes -r261:263 to audiodb-debian branch audiodb-debian
Fri, 23 Nov 2007 15:43:51 +0000 mas01cr Alter timesTable handling.
Fri, 23 Nov 2007 11:18:33 +0000 mas01cr Merge trunk changes -r259:260 into audiodb-debian branch audiodb-debian
Fri, 23 Nov 2007 11:08:15 +0000 mas01cr Merge -r228:254 from no-big-mmap branch.
Wed, 21 Nov 2007 11:44:41 +0000 mas01cr Merged trunk changes -r197:258 into audiodb-debian branch. audiodb-debian
Wed, 21 Nov 2007 11:38:19 +0000 mas01cr Remove workaround for previous file format compatibility changes.
Wed, 21 Nov 2007 11:35:44 +0000 mas01cr Merge powertable branch -r168:227 to trunk.
Mon, 19 Nov 2007 18:37:12 +0000 mas01cr Create and status of huge databases works. no-big-mmap
Mon, 19 Nov 2007 17:19:46 +0000 mas01cr mmap dataBuf for the remaining cases no-big-mmap
Fri, 16 Nov 2007 18:21:37 +0000 mas01cr remove two more uses of dataBuf no-big-mmap
Fri, 16 Nov 2007 16:55:40 +0000 mas01cr remove stray dataBuf use in trackPointQuery no-big-mmap
Fri, 16 Nov 2007 16:39:59 +0000 mas01cr Test against running off the end of the timesTable properly. no-big-mmap
Fri, 16 Nov 2007 16:31:36 +0000 mas01cr Wow, two changes for the price of one no-big-mmap
Fri, 16 Nov 2007 11:44:55 +0000 mas01cr Typo fix in cleanup() no-big-mmap
Thu, 15 Nov 2007 17:40:46 +0000 mas01cr no dataBuf use in dump() no-big-mmap
Thu, 15 Nov 2007 17:14:04 +0000 mas01cr whitespace. no-big-mmap
Thu, 15 Nov 2007 17:09:17 +0000 mas01cr Delete a stray remaining memcpy(). no-big-mmap
Thu, 15 Nov 2007 17:04:55 +0000 mas01cr inserting the data vectors goes through lseek()/write() no-big-mmap
Thu, 15 Nov 2007 16:38:53 +0000 mas01cr checks for enough free data space don't use dataBuf any more no-big-mmap
Thu, 15 Nov 2007 16:22:59 +0000 mas01cr fewer accesses through dataBuf no-big-mmap
Thu, 15 Nov 2007 15:18:37 +0000 mas01cr munmap() the separate mappings properly no-big-mmap
Thu, 15 Nov 2007 14:58:16 +0000 mas01cr forWrite audioDB flag (rather than function argument) so that we can no-big-mmap
Wed, 14 Nov 2007 17:58:57 +0000 mas01cr map only the database header page as db, not the entire database. no-big-mmap
Wed, 14 Nov 2007 16:40:07 +0000 mas01cr munmap() the tables too! no-big-mmap
Wed, 14 Nov 2007 16:32:18 +0000 mas01cr mmap() the various tables separately on init. no-big-mmap
Wed, 14 Nov 2007 15:57:15 +0000 mas01cr Don't use mmap() in database creation no-big-mmap
Wed, 14 Nov 2007 14:00:53 +0000 mas01cr Web services interface to the powertable functionality. powertable
Thu, 08 Nov 2007 14:18:47 +0000 mas01cr Better error detection (and messages) on invalid feature files
Tue, 06 Nov 2007 17:16:32 +0000 mas01cr Merge trunk changes -r156:196 to audiodb-debian branch. audiodb-debian
Tue, 06 Nov 2007 16:29:56 +0000 mas01cr Defensive treatment of SOAP input
Thu, 01 Nov 2007 14:56:22 +0000 mas01cr sequence_sqrt() and sequence_average() functions for common patterns powertable
Thu, 01 Nov 2007 14:43:19 +0000 mas01cr Fix qpPtr to point to the query point, if applicable. powertable
Thu, 01 Nov 2007 14:37:08 +0000 mas01cr Implement power thresholding for radius search. powertable
Thu, 01 Nov 2007 11:50:34 +0000 mas01cr Delete now useless code from trackSequenceQueryNN powertable
Wed, 31 Oct 2007 18:22:23 +0000 mas01cr dump/restore support for the power table. powertable
Wed, 31 Oct 2007 16:53:25 +0000 mas01cr Correct schoolboy cut'n'paste error. powertable
Wed, 31 Oct 2007 15:38:11 +0000 mas01cr More defensive command-line error checking:
Wed, 31 Oct 2007 14:37:55 +0000 mas01cr Maybe implement silence thresholding for the sequence query (without an powertable
Wed, 31 Oct 2007 13:08:21 +0000 mas01cr Abstract the operation to perform a windowed sequence sum into a powertable
Wed, 31 Oct 2007 12:01:28 +0000 mas01cr Add absolute and relative threshold arguments, and initialize variables powertable
Wed, 31 Oct 2007 10:20:09 +0000 mas01cr Define a power command-line option; use it to turn on the DB flag. powertable
Tue, 30 Oct 2007 15:28:48 +0000 mas01cr Hack in batchinsert support for power. powertable
Tue, 30 Oct 2007 15:13:01 +0000 mas01cr Hack in power table filling for single track insert. powertable
Tue, 30 Oct 2007 14:26:50 +0000 mas01cr Beginnings of power table support powertable
Mon, 29 Oct 2007 11:47:42 +0000 mas01cr Smaller databases
Tue, 23 Oct 2007 19:57:06 +0000 mas01cr Merge trunk changes -r146:155 to audio-debian branch. audiodb-debian
Tue, 23 Oct 2007 17:21:08 +0000 mas01cr Friendlier Actionscript/SOAP behaviour.
Fri, 19 Oct 2007 17:04:46 +0000 mas01cr Initial cut at a dump utility; binary-based for now.
Fri, 19 Oct 2007 14:41:54 +0000 mas01cr Variable size databases, part 2: new --size argument on database
Fri, 19 Oct 2007 13:51:53 +0000 mas01cr Variable-size databases, step 1: introduce dbSize field into the
Wed, 17 Oct 2007 14:52:39 +0000 mas01cr Merge trunk changes -r143:145 to audiodb-debian branch audiodb-debian
Wed, 17 Oct 2007 14:44:53 +0000 mas01cr Be a bit more careful about qNorm handling and freeing: don't assign qNorm
Wed, 17 Oct 2007 14:14:02 +0000 mas01cr Merge trunk changes -r129:142 to audiodb-debian branch audiodb-debian
Wed, 17 Oct 2007 14:07:15 +0000 mas01cr Now with even fewer memory leaks!
Wed, 17 Oct 2007 13:49:20 +0000 mas01cr Whoops! Leaking an l2normtable's worth of data for every request is
Tue, 16 Oct 2007 11:47:51 +0000 mas01cr Make a start on refactoring: work the DB header initialization into its
Fri, 12 Oct 2007 12:58:30 +0000 mas01cr Towards endian-neutrality, part 4. endian-neutral
Fri, 12 Oct 2007 11:52:28 +0000 mas01cr Towards endian-neutrality, part 3. endian-neutral
Fri, 12 Oct 2007 11:37:38 +0000 mas01cr Towards endian-neutrality, step 2. endian-neutral
Fri, 12 Oct 2007 11:20:35 +0000 mas01cr Towards endian-neutrality, step 1. endian-neutral
Fri, 05 Oct 2007 15:42:50 +0000 mas01cr Merge trunk changes -r123:128 to audiodb-debian branch audiodb-debian
Fri, 05 Oct 2007 14:21:43 +0000 mas01cr New on-disk format!
Fri, 05 Oct 2007 11:45:03 +0000 mas01cr Merge trunk changes -r96:122 to audiodb-debian branch. audiodb-debian
Fri, 05 Oct 2007 11:37:56 +0000 mas01cr More small refactoring. last-magic=O2DB
Fri, 05 Oct 2007 11:05:22 +0000 mas01cr Miscellaneous small refactorings:
Thu, 04 Oct 2007 15:02:29 +0000 mas01cr Temporary signal handling.
Wed, 03 Oct 2007 15:16:47 +0000 mas01cr Clean up a bit better when a server audioDB constructor throws an
Wed, 03 Oct 2007 13:53:39 +0000 mas01cr Give in to expediency and use SO_REUSEADDR on the bind() call.
Tue, 02 Oct 2007 15:36:22 +0000 mas01cr Fix for bug from test 0014 (WS point query result list)
Tue, 02 Oct 2007 14:32:23 +0000 mas01cr Make the Qpos and Spos arrays for Web services results unsigned.
Tue, 02 Oct 2007 13:26:10 +0000 mas01cr Cosmetic fix to the file format comment: prefer O2 to 02.
Tue, 02 Oct 2007 13:16:00 +0000 mas01cr Hack the right answer out of the L2Norm offset computation.
Mon, 01 Oct 2007 18:37:35 +0000 mas01cr Comment regarding leakiness of server error handling.
Mon, 01 Oct 2007 14:59:03 +0000 mas01cr Merge trunk changes -r94:95 onto audiodb-debian branch audiodb-debian
Mon, 01 Oct 2007 14:55:18 +0000 mas01cr Whoops! Really actually bullet-proof the server for query requests too.
Mon, 01 Oct 2007 14:40:08 +0000 mas01cr Merge trunk changes -r52:93 onto audiodb-debian branch audiodb-debian
Mon, 01 Oct 2007 14:30:58 +0000 mas01cr Slightly more bullet-proof server. In audioDB::error(), We check to see
Mon, 01 Oct 2007 13:54:22 +0000 mas01cr Make the status request Web Services enabled.
Mon, 01 Oct 2007 13:32:55 +0000 mas01cr Whoops. %lud is not "long unsigned int", it's "long unsigned int + d".
Thu, 27 Sep 2007 09:28:23 +0000 mas01cr Fix trackNN/pointNN confusion leading to segfaults in optimized code for
Wed, 26 Sep 2007 15:14:54 +0000 mas01cr Fix various off-by-one errors in the radius search (makes tests/0010
Wed, 26 Sep 2007 12:47:20 +0000 mas01cr Fix bugs revealed by test 0009
Wed, 26 Sep 2007 11:29:45 +0000 mas01cr Change the default port for the web services server to 14475 (which is
Thu, 20 Sep 2007 14:29:30 +0000 mas01cr Fix tests/0004 by initializing the distance vector for point queries
Thu, 20 Sep 2007 13:59:16 +0000 mas01cr Decorate if(verbosity>...) clauses with braces, for avoidance of doubt.
Thu, 20 Sep 2007 13:54:28 +0000 mas01cr Make the system compile without warnings, and turn on -Werror for
Thu, 20 Sep 2007 11:53:11 +0000 mas01cr Delete some (wrong) FIXME notes about off-by-one errors.
Thu, 20 Sep 2007 11:30:24 +0000 mas01cr Fix a couple of off-by-one errors, including one which was preventing a
Thu, 20 Sep 2007 09:52:40 +0000 mas01cr Fix segfault when the query is shorter than the sequence length
Thu, 20 Sep 2007 08:50:35 +0000 mas01cr Fix what is I think an off-by-one in query and sequence searching.
Tue, 18 Sep 2007 09:31:41 +0000 mas01cr Fix bug revealed by test case 0005: L2NORM needs write access to the
Wed, 29 Aug 2007 16:12:46 +0000 mas01cr Merge trunk changes -r44:51 onto audio-debian branch audiodb-debian
Tue, 21 Aug 2007 17:16:32 +0000 mas01cr Where audioDB::error() is called because of a failed system call,
Tue, 21 Aug 2007 17:05:20 +0000 mas01cr Don't truncate on creation: instead error if the requested database file
Tue, 21 Aug 2007 16:59:33 +0000 mas01cr Add fcntl() locking to the database file handle.
Thu, 16 Aug 2007 11:42:19 +0000 mas01cr Merged trunk changes -r39:43 onto audiodb-debian branch audiodb-debian
Thu, 16 Aug 2007 11:39:46 +0000 mas01cr add a forWrite flag to audioDB::initTables() so that clients can specify
Thu, 16 Aug 2007 11:19:35 +0000 mas01cr Cosmetic error message string fix.
Thu, 16 Aug 2007 10:04:55 +0000 mas01cr Merged trunk changes -r30:38 onto audiodb-debian branch audiodb-debian
Thu, 16 Aug 2007 08:51:00 +0000 mas01cr Pass the needed "-d" flag to the audioDB constructor in adb__status
Mon, 13 Aug 2007 23:25:16 +0000 mas01mc New major version, mostly tested: both sequence queries (-Q seq --pointnn N and -Q seq --radius R) now work, all reported distances are Euclidean.
Mon, 13 Aug 2007 19:14:33 +0000 mas01mc minor revisions: mainly tidying up naming conventions (segment->track)
Fri, 10 Aug 2007 04:52:33 +0000 mas01mc added Euclidean distance for sequences with -R (--radus) (via dot product of unit norm vectors), re-worked L2-norm behaviour, fixed a load of bugs there, fixed shingle norming. Cosine dist sequence match not working now because of L2 norm behaviour
Thu, 02 Aug 2007 11:34:27 +0000 mas01cr Merged trunk changes -r14:29 onto audiodb-debian branch audiodb-debian
Thu, 26 Jul 2007 14:56:47 +0000 mas01mc fixed keyList loop logic for segment search
Thu, 26 Jul 2007 14:08:11 +0000 mas01mc fixed keyList logic (in sequence match only) to avoid halting just because the database's last key was visited before end of keylist sequence.
Thu, 26 Jul 2007 10:14:51 +0000 mas01mc munmap on batch insert, removed 'ignore' command line option -g
Thu, 26 Jul 2007 06:58:39 +0000 mas01mc added ignore flag for first column of features (not working yet)
Wed, 25 Jul 2007 16:06:21 +0000 mas01cr Apparently, gengetopt only allows single dependencies between options.
Wed, 25 Jul 2007 15:39:26 +0000 mas01cr open(2) needs a third argument when O_CREAT is specified: give it
Fri, 20 Jul 2007 15:51:39 +0000 mas01cr Initial import