Mercurial > hg > audiodb
changeset 136:5bac833c41b5 audiodb-debian
Merge trunk changes -r146:155 to audio-debian branch.
Also minor adjustments to debian/ files for new .txt files, and new
debian/changelog version. (-13, for those keeping count at home...)
| author | mas01cr |
|---|---|
| date | Tue, 23 Oct 2007 19:57:06 +0000 |
| parents | b9f1c375f28a |
| children | 100b7ba08df3 |
| files | INSTALL.txt LICENCE.txt Makefile README.txt audioDB.cpp audioDB.h audioDBws.h debian/changelog debian/docs debian/rules gengetopt.in gpl.txt tests/0024/run-test.sh tests/0024/short-description |
| diffstat | 14 files changed, 672 insertions(+), 525 deletions(-) [+] |
line wrap: on
line diff
--- a/INSTALL.txt Wed Oct 17 14:52:39 2007 +0000 +++ b/INSTALL.txt Tue Oct 23 19:57:06 2007 +0000 @@ -1,28 +1,19 @@ -audioDB -Goldsmiths, University of London -GPL - Dependencies ------------ -audioDB uses libgsoap++ for SOAP Web Services support, this can be installed on most linux distributions using the package manager. -The binary distribution will suffice. +gsoap: audioDB uses libgsoap++ for SOAP Web Services support. This + can be installed on most linux distributions using the package + manager; the binary distribution will suffice. Other systems + will need to install the library. -For OSX and Win32 systems, the gsoap distribution is currently included in the bin_x86_{osx,win32} directories +gengetopt: audioDB uses gengetopt (as of 2007-10-23, version 2.18 + exactly) for command-line parsing. -Use the following Makefiles to compile audioDB on your system. +help2man: a spartan manual page is automatically generated from the + usage text using help2man. -Linux ------ -make -f Makefile +Use the following Makefiles to compile audioDB on your system: -OSX ---- - -make -f Makefile.osx - -Win32 ------ - -make -f Makefile.win32 - +Linux: Makefile +Mac OS X: Makefile.osx +Win32: Makefile.win32
--- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/LICENCE.txt Tue Oct 23 19:57:06 2007 +0000 @@ -0,0 +1,339 @@ + GNU GENERAL PUBLIC LICENSE + Version 2, June 1991 + + Copyright (C) 1989, 1991 Free Software Foundation, Inc., + 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + + Preamble + + The licenses for most software are designed to take away your +freedom to share and change it. By contrast, the GNU General Public +License is intended to guarantee your freedom to share and change free +software--to make sure the software is free for all its users. This +General Public License applies to most of the Free Software +Foundation's software and to any other program whose authors commit to +using it. (Some other Free Software Foundation software is covered by +the GNU Lesser General Public License instead.) You can apply it to +your programs, too. + + When we speak of free software, we are referring to freedom, not +price. Our General Public Licenses are designed to make sure that you +have the freedom to distribute copies of free software (and charge for +this service if you wish), that you receive source code or can get it +if you want it, that you can change the software or use pieces of it +in new free programs; and that you know you can do these things. + + To protect your rights, we need to make restrictions that forbid +anyone to deny you these rights or to ask you to surrender the rights. +These restrictions translate to certain responsibilities for you if you +distribute copies of the software, or if you modify it. + + For example, if you distribute copies of such a program, whether +gratis or for a fee, you must give the recipients all the rights that +you have. You must make sure that they, too, receive or can get the +source code. And you must show them these terms so they know their +rights. + + We protect your rights with two steps: (1) copyright the software, and +(2) offer you this license which gives you legal permission to copy, +distribute and/or modify the software. + + Also, for each author's protection and ours, we want to make certain +that everyone understands that there is no warranty for this free +software. If the software is modified by someone else and passed on, we +want its recipients to know that what they have is not the original, so +that any problems introduced by others will not reflect on the original +authors' reputations. + + Finally, any free program is threatened constantly by software +patents. We wish to avoid the danger that redistributors of a free +program will individually obtain patent licenses, in effect making the +program proprietary. To prevent this, we have made it clear that any +patent must be licensed for everyone's free use or not licensed at all. + + The precise terms and conditions for copying, distribution and +modification follow. + + GNU GENERAL PUBLIC LICENSE + TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION + + 0. This License applies to any program or other work which contains +a notice placed by the copyright holder saying it may be distributed +under the terms of this General Public License. The "Program", below, +refers to any such program or work, and a "work based on the Program" +means either the Program or any derivative work under copyright law: +that is to say, a work containing the Program or a portion of it, +either verbatim or with modifications and/or translated into another +language. (Hereinafter, translation is included without limitation in +the term "modification".) Each licensee is addressed as "you". + +Activities other than copying, distribution and modification are not +covered by this License; they are outside its scope. The act of +running the Program is not restricted, and the output from the Program +is covered only if its contents constitute a work based on the +Program (independent of having been made by running the Program). +Whether that is true depends on what the Program does. + + 1. You may copy and distribute verbatim copies of the Program's +source code as you receive it, in any medium, provided that you +conspicuously and appropriately publish on each copy an appropriate +copyright notice and disclaimer of warranty; keep intact all the +notices that refer to this License and to the absence of any warranty; +and give any other recipients of the Program a copy of this License +along with the Program. + +You may charge a fee for the physical act of transferring a copy, and +you may at your option offer warranty protection in exchange for a fee. + + 2. You may modify your copy or copies of the Program or any portion +of it, thus forming a work based on the Program, and copy and +distribute such modifications or work under the terms of Section 1 +above, provided that you also meet all of these conditions: + + a) You must cause the modified files to carry prominent notices + stating that you changed the files and the date of any change. + + b) You must cause any work that you distribute or publish, that in + whole or in part contains or is derived from the Program or any + part thereof, to be licensed as a whole at no charge to all third + parties under the terms of this License. + + c) If the modified program normally reads commands interactively + when run, you must cause it, when started running for such + interactive use in the most ordinary way, to print or display an + announcement including an appropriate copyright notice and a + notice that there is no warranty (or else, saying that you provide + a warranty) and that users may redistribute the program under + these conditions, and telling the user how to view a copy of this + License. (Exception: if the Program itself is interactive but + does not normally print such an announcement, your work based on + the Program is not required to print an announcement.) + +These requirements apply to the modified work as a whole. If +identifiable sections of that work are not derived from the Program, +and can be reasonably considered independent and separate works in +themselves, then this License, and its terms, do not apply to those +sections when you distribute them as separate works. But when you +distribute the same sections as part of a whole which is a work based +on the Program, the distribution of the whole must be on the terms of +this License, whose permissions for other licensees extend to the +entire whole, and thus to each and every part regardless of who wrote it. + +Thus, it is not the intent of this section to claim rights or contest +your rights to work written entirely by you; rather, the intent is to +exercise the right to control the distribution of derivative or +collective works based on the Program. + +In addition, mere aggregation of another work not based on the Program +with the Program (or with a work based on the Program) on a volume of +a storage or distribution medium does not bring the other work under +the scope of this License. + + 3. You may copy and distribute the Program (or a work based on it, +under Section 2) in object code or executable form under the terms of +Sections 1 and 2 above provided that you also do one of the following: + + a) Accompany it with the complete corresponding machine-readable + source code, which must be distributed under the terms of Sections + 1 and 2 above on a medium customarily used for software interchange; or, + + b) Accompany it with a written offer, valid for at least three + years, to give any third party, for a charge no more than your + cost of physically performing source distribution, a complete + machine-readable copy of the corresponding source code, to be + distributed under the terms of Sections 1 and 2 above on a medium + customarily used for software interchange; or, + + c) Accompany it with the information you received as to the offer + to distribute corresponding source code. (This alternative is + allowed only for noncommercial distribution and only if you + received the program in object code or executable form with such + an offer, in accord with Subsection b above.) + +The source code for a work means the preferred form of the work for +making modifications to it. For an executable work, complete source +code means all the source code for all modules it contains, plus any +associated interface definition files, plus the scripts used to +control compilation and installation of the executable. However, as a +special exception, the source code distributed need not include +anything that is normally distributed (in either source or binary +form) with the major components (compiler, kernel, and so on) of the +operating system on which the executable runs, unless that component +itself accompanies the executable. + +If distribution of executable or object code is made by offering +access to copy from a designated place, then offering equivalent +access to copy the source code from the same place counts as +distribution of the source code, even though third parties are not +compelled to copy the source along with the object code. + + 4. You may not copy, modify, sublicense, or distribute the Program +except as expressly provided under this License. Any attempt +otherwise to copy, modify, sublicense or distribute the Program is +void, and will automatically terminate your rights under this License. +However, parties who have received copies, or rights, from you under +this License will not have their licenses terminated so long as such +parties remain in full compliance. + + 5. You are not required to accept this License, since you have not +signed it. However, nothing else grants you permission to modify or +distribute the Program or its derivative works. These actions are +prohibited by law if you do not accept this License. Therefore, by +modifying or distributing the Program (or any work based on the +Program), you indicate your acceptance of this License to do so, and +all its terms and conditions for copying, distributing or modifying +the Program or works based on it. + + 6. Each time you redistribute the Program (or any work based on the +Program), the recipient automatically receives a license from the +original licensor to copy, distribute or modify the Program subject to +these terms and conditions. You may not impose any further +restrictions on the recipients' exercise of the rights granted herein. +You are not responsible for enforcing compliance by third parties to +this License. + + 7. If, as a consequence of a court judgment or allegation of patent +infringement or for any other reason (not limited to patent issues), +conditions are imposed on you (whether by court order, agreement or +otherwise) that contradict the conditions of this License, they do not +excuse you from the conditions of this License. If you cannot +distribute so as to satisfy simultaneously your obligations under this +License and any other pertinent obligations, then as a consequence you +may not distribute the Program at all. For example, if a patent +license would not permit royalty-free redistribution of the Program by +all those who receive copies directly or indirectly through you, then +the only way you could satisfy both it and this License would be to +refrain entirely from distribution of the Program. + +If any portion of this section is held invalid or unenforceable under +any particular circumstance, the balance of the section is intended to +apply and the section as a whole is intended to apply in other +circumstances. + +It is not the purpose of this section to induce you to infringe any +patents or other property right claims or to contest validity of any +such claims; this section has the sole purpose of protecting the +integrity of the free software distribution system, which is +implemented by public license practices. Many people have made +generous contributions to the wide range of software distributed +through that system in reliance on consistent application of that +system; it is up to the author/donor to decide if he or she is willing +to distribute software through any other system and a licensee cannot +impose that choice. + +This section is intended to make thoroughly clear what is believed to +be a consequence of the rest of this License. + + 8. If the distribution and/or use of the Program is restricted in +certain countries either by patents or by copyrighted interfaces, the +original copyright holder who places the Program under this License +may add an explicit geographical distribution limitation excluding +those countries, so that distribution is permitted only in or among +countries not thus excluded. In such case, this License incorporates +the limitation as if written in the body of this License. + + 9. The Free Software Foundation may publish revised and/or new versions +of the General Public License from time to time. Such new versions will +be similar in spirit to the present version, but may differ in detail to +address new problems or concerns. + +Each version is given a distinguishing version number. If the Program +specifies a version number of this License which applies to it and "any +later version", you have the option of following the terms and conditions +either of that version or of any later version published by the Free +Software Foundation. If the Program does not specify a version number of +this License, you may choose any version ever published by the Free Software +Foundation. + + 10. If you wish to incorporate parts of the Program into other free +programs whose distribution conditions are different, write to the author +to ask for permission. For software which is copyrighted by the Free +Software Foundation, write to the Free Software Foundation; we sometimes +make exceptions for this. Our decision will be guided by the two goals +of preserving the free status of all derivatives of our free software and +of promoting the sharing and reuse of software generally. + + NO WARRANTY + + 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY +FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN +OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES +PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED +OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS +TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE +PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, +REPAIR OR CORRECTION. + + 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING +WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR +REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, +INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING +OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED +TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY +YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER +PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE +POSSIBILITY OF SUCH DAMAGES. + + END OF TERMS AND CONDITIONS + + How to Apply These Terms to Your New Programs + + If you develop a new program, and you want it to be of the greatest +possible use to the public, the best way to achieve this is to make it +free software which everyone can redistribute and change under these terms. + + To do so, attach the following notices to the program. It is safest +to attach them to the start of each source file to most effectively +convey the exclusion of warranty; and each file should have at least +the "copyright" line and a pointer to where the full notice is found. + + <one line to give the program's name and a brief idea of what it does.> + Copyright (C) <year> <name of author> + + This program is free software; you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation; either version 2 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License along + with this program; if not, write to the Free Software Foundation, Inc., + 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. + +Also add information on how to contact you by electronic and paper mail. + +If the program is interactive, make it output a short notice like this +when it starts in an interactive mode: + + Gnomovision version 69, Copyright (C) year name of author + Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'. + This is free software, and you are welcome to redistribute it + under certain conditions; type `show c' for details. + +The hypothetical commands `show w' and `show c' should show the appropriate +parts of the General Public License. Of course, the commands you use may +be called something other than `show w' and `show c'; they could even be +mouse-clicks or menu items--whatever suits your program. + +You should also get your employer (if you work as a programmer) or your +school, if any, to sign a "copyright disclaimer" for the program, if +necessary. Here is a sample; alter the names: + + Yoyodyne, Inc., hereby disclaims all copyright interest in the program + `Gnomovision' (which makes passes at compilers) written by James Hacker. + + <signature of Ty Coon>, 1 April 1989 + Ty Coon, President of Vice + +This General Public License does not permit incorporating your program into +proprietary programs. If your program is a subroutine library, you may +consider it more useful to permit linking proprietary applications with the +library. If this is what you want to do, use the GNU Lesser General +Public License instead of this License.
--- a/Makefile Wed Oct 17 14:52:39 2007 +0000 +++ b/Makefile Tue Oct 23 19:57:06 2007 +0000 @@ -1,4 +1,4 @@ -CFLAGS=-O3 +CFLAGS=-O3 -g LIBS=-lgsoap++ EXECUTABLE=audioDB @@ -10,8 +10,8 @@ ${EXECUTABLE}.1: ${EXECUTABLE} help2man ./${EXECUTABLE} > ${EXECUTABLE}.1 -README.txt: ${EXECUTABLE} - ./${EXECUTABLE} --help > README.txt +HELP.txt: ${EXECUTABLE} + ./${EXECUTABLE} --help > HELP.txt cmdline.c cmdline.h: gengetopt.in gengetopt <gengetopt.in @@ -27,7 +27,7 @@ -rm cmdline.c cmdline.h -rm soapServer.cpp soapClient.cpp soapC.cpp soapObject.h soapStub.h soapProxy.h soapH.h soapServerLib.cpp soapClientLib.cpp -rm adb.nsmap adb.xsd adb.wsdl adb.query.req.xml adb.query.res.xml adb.status.req.xml adb.status.res.xml - -rm README.txt + -rm HELP.txt -rm ${EXECUTABLE} ${EXECUTABLE}.1 audioDB.o -sh -c "cd tests && sh ./clean.sh"
--- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/README.txt Tue Oct 23 19:57:06 2007 +0000 @@ -0,0 +1,8 @@ +This is audioDB, a database system specialized towards approximate +content-based retrieval, developed at Goldsmiths, University of London +as part of the OMRAS2 research project (funded by EPSRC). + +Please see INSTALL.txt for build and installation instructions. +Portions of this work are copyright Michael Casey, Christophe Rhodes, +and Goldsmiths, University of London; please see LICENCE.txt for +restrictions on use and redistribution.
--- a/audioDB.cpp Wed Oct 17 14:52:39 2007 +0000 +++ b/audioDB.cpp Tue Oct 23 19:57:06 2007 +0000 @@ -79,26 +79,26 @@ error("Unrecognized command",command); } -audioDB::audioDB(const unsigned argc, char* const argv[], adb__queryResult *adbQueryResult): O2_AUDIODB_INITIALIZERS +audioDB::audioDB(const unsigned argc, char* const argv[], adb__queryResponse *adbQueryResponse): O2_AUDIODB_INITIALIZERS { try { processArgs(argc, argv); isServer = 1; // FIXME: Hack assert(O2_ACTION(COM_QUERY)); - query(dbName, inFile, adbQueryResult); + query(dbName, inFile, adbQueryResponse); } catch(char *err) { cleanup(); throw(err); } } -audioDB::audioDB(const unsigned argc, char* const argv[], adb__statusResult *adbStatusResult): O2_AUDIODB_INITIALIZERS +audioDB::audioDB(const unsigned argc, char* const argv[], adb__statusResponse *adbStatusResponse): O2_AUDIODB_INITIALIZERS { try { processArgs(argc, argv); isServer = 1; // FIXME: Hack assert(O2_ACTION(COM_STATUS)); - status(dbName, adbStatusResult); + status(dbName, adbStatusResponse); } catch(char *err) { cleanup(); throw(err); @@ -110,7 +110,7 @@ if(indata) munmap(indata,statbuf.st_size); if(db) - munmap(db,O2_DEFAULTDBSIZE); + munmap(db,dbH->dbSize); if(dbfid>0) close(dbfid); if(infid>0) @@ -152,6 +152,13 @@ } } + if(args_info.size_given) { + if (args_info.size_arg < 250 || args_info.size_arg > 4000) { + error("Size out of range", ""); + } + size = args_info.size_arg * 1000000; + } + if(args_info.radius_given){ radius=args_info.radius_arg; if(radius<=0 || radius>1000000000){ @@ -203,6 +210,7 @@ if(args_info.DUMP_given){ command=COM_DUMP; dbName=args_info.database_arg; + output = args_info.output_arg; return 0; } @@ -372,7 +380,7 @@ get_lock(dbfid, 1); // go to the location corresponding to the last byte - if (lseek (dbfid, O2_DEFAULTDBSIZE - 1, SEEK_SET) == -1) + if (lseek (dbfid, size - 1, SEEK_SET) == -1) error("lseek error in db file", "", "lseek"); // write a dummy byte at the last location @@ -383,7 +391,7 @@ if(verbosity) { cerr << "header size:" << O2_HEADERSIZE << endl; } - if ((db = (char*) mmap(0, O2_DEFAULTDBSIZE, PROT_READ | PROT_WRITE, + if ((db = (char*) mmap(0, size, PROT_READ | PROT_WRITE, MAP_SHARED, dbfid, 0)) == (caddr_t) -1) error("mmap error for creating database", "", "mmap"); @@ -400,8 +408,9 @@ dbH->fileTableOffset = ALIGN_UP(O2_HEADERSIZE, 8); dbH->trackTableOffset = ALIGN_UP(dbH->fileTableOffset + O2_FILETABLESIZE*O2_MAXFILES, 8); dbH->dataOffset = ALIGN_UP(dbH->trackTableOffset + O2_TRACKTABLESIZE*O2_MAXFILES, 8); - dbH->l2normTableOffset = ALIGN_DOWN(O2_DEFAULTDBSIZE - O2_MAXFILES*O2_MEANNUMVECTORS*sizeof(double), 8); + dbH->l2normTableOffset = ALIGN_DOWN(size - O2_MAXFILES*O2_MEANNUMVECTORS*sizeof(double), 8); dbH->timesTableOffset = ALIGN_DOWN(dbH->l2normTableOffset - O2_MAXFILES*O2_MEANNUMVECTORS*sizeof(double), 8); + dbH->dbSize = size; memcpy (db, dbH, O2_HEADERSIZE); if(verbosity) { @@ -443,8 +452,13 @@ error("database file has incorect version", dbName); } + // FIXME: when changing file format version, remove this workaround. + if(dbH->dbSize == 0) { + dbH->dbSize = O2_DEFAULTDBSIZE; + } + // mmap the database file - if ((db = (char*) mmap(0, O2_DEFAULTDBSIZE, PROT_READ | (forWrite ? PROT_WRITE : 0), + if ((db = (char*) mmap(0, dbH->dbSize, PROT_READ | (forWrite ? PROT_WRITE : 0), MAP_SHARED, dbfid, 0)) == (caddr_t) -1) error("mmap error for initting tables of database", "", "mmap"); @@ -523,7 +537,7 @@ } // CLEAN UP munmap(indata,statbuf.st_size); - munmap(db,O2_DEFAULTDBSIZE); + munmap(db,dbH->dbSize); close(infid); return; } @@ -583,7 +597,7 @@ if(!timesFile->is_open()){ if(dbH->flags & O2_FLAG_TIMES){ munmap(indata,statbuf.st_size); - munmap(db,O2_DEFAULTDBSIZE); + munmap(db,dbH->dbSize); error("problem opening times file on timestamped database",timesFileName); } else{ @@ -611,7 +625,7 @@ } if(numtimes<numVectors || numtimes>numVectors+2){ munmap(indata,statbuf.st_size); - munmap(db,O2_DEFAULTDBSIZE); + munmap(db,dbH->dbSize); close(infid); cerr << "expected " << numVectors << " found " << numtimes << endl; error("Times file is incorrect length for features file",inFile); @@ -773,17 +787,17 @@ // this. -- CSR, 2007-10-01 void audioDB::ws_status(const char*dbName, char* hostport){ struct soap soap; - adb__statusResult adbStatusResult; + adb__statusResponse adbStatusResponse; // Query an existing adb database soap_init(&soap); - if(soap_call_adb__status(&soap,hostport,NULL,(char*)dbName,adbStatusResult)==SOAP_OK) { - cout << "numFiles = " << adbStatusResult.numFiles << endl; - cout << "dim = " << adbStatusResult.dim << endl; - cout << "length = " << adbStatusResult.length << endl; - cout << "dudCount = " << adbStatusResult.dudCount << endl; - cout << "nullCount = " << adbStatusResult.nullCount << endl; - cout << "flags = " << adbStatusResult.flags << endl; + if(soap_call_adb__status(&soap,hostport,NULL,(char*)dbName,adbStatusResponse)==SOAP_OK) { + cout << "numFiles = " << adbStatusResponse.result.numFiles << endl; + cout << "dim = " << adbStatusResponse.result.dim << endl; + cout << "length = " << adbStatusResponse.result.length << endl; + cout << "dudCount = " << adbStatusResponse.result.dudCount << endl; + cout << "nullCount = " << adbStatusResponse.result.nullCount << endl; + cout << "flags = " << adbStatusResponse.result.flags << endl; } else { soap_print_fault(&soap,stderr); } @@ -795,16 +809,16 @@ void audioDB::ws_query(const char*dbName, const char *trackKey, const char* hostport){ struct soap soap; - adb__queryResult adbQueryResult; + adb__queryResponse adbQueryResponse; soap_init(&soap); if(soap_call_adb__query(&soap,hostport,NULL, (char*)dbName,(char*)trackKey,(char*)trackFileName,(char*)timesFileName, - queryType, queryPoint, pointNN, trackNN, sequenceLength, adbQueryResult)==SOAP_OK){ - //std::cerr << "result list length:" << adbQueryResult.__sizeRlist << std::endl; - for(int i=0; i<adbQueryResult.__sizeRlist; i++) - std::cout << adbQueryResult.Rlist[i] << " " << adbQueryResult.Dist[i] - << " " << adbQueryResult.Qpos[i] << " " << adbQueryResult.Spos[i] << std::endl; + queryType, queryPoint, pointNN, trackNN, sequenceLength, adbQueryResponse)==SOAP_OK){ + //std::cerr << "result list length:" << adbQueryResponse.result.__sizeRlist << std::endl; + for(int i=0; i<adbQueryResponse.result.__sizeRlist; i++) + std::cout << adbQueryResponse.result.Rlist[i] << " " << adbQueryResponse.result.Dist[i] + << " " << adbQueryResponse.result.Qpos[i] << " " << adbQueryResponse.result.Spos[i] << std::endl; } else soap_print_fault(&soap,stderr); @@ -816,7 +830,7 @@ } -void audioDB::status(const char* dbName, adb__statusResult *adbStatusResult){ +void audioDB::status(const char* dbName, adb__statusResponse *adbStatusResponse){ if(!dbH) initTables(dbName, 0, 0); @@ -830,7 +844,7 @@ } } - if(adbStatusResult == 0) { + if(adbStatusResponse == 0) { // Update Header information cout << "num files:" << dbH->numFiles << endl; @@ -846,24 +860,127 @@ cout << "null count: " << nullCount << " small sequence count " << dudCount-nullCount << endl; } else { - adbStatusResult->numFiles = dbH->numFiles; - adbStatusResult->dim = dbH->dim; - adbStatusResult->length = dbH->length; - adbStatusResult->dudCount = dudCount; - adbStatusResult->nullCount = nullCount; - adbStatusResult->flags = dbH->flags; + adbStatusResponse->result.numFiles = dbH->numFiles; + adbStatusResponse->result.dim = dbH->dim; + adbStatusResponse->result.length = dbH->length; + adbStatusResponse->result.dudCount = dudCount; + adbStatusResponse->result.nullCount = nullCount; + adbStatusResponse->result.flags = dbH->flags; } } void audioDB::dump(const char* dbName){ - if(!dbH) + if(!dbH) { initTables(dbName, 0, 0); - - for(unsigned k=0, j=0; k<dbH->numFiles; k++){ - cout << fileTable+k*O2_FILETABLESIZE << " " << trackTable[k] << endl; - j+=trackTable[k]; } + if((mkdir(output, S_IRWXU|S_IRWXG|S_IRWXO)) < 0) { + error("error making output directory", output, "mkdir"); + } + + char *cwd = new char[PATH_MAX]; + + if ((getcwd(cwd, PATH_MAX)) == 0) { + error("error getting working directory", "", "getcwd"); + } + + if((chdir(output)) < 0) { + error("error changing working directory", output, "chdir"); + } + + int fLfd, tLfd = 0, kLfd; + FILE *fLFile, *tLFile = 0, *kLFile; + + if ((fLfd = open("featureList.txt", O_CREAT|O_RDWR|O_EXCL, S_IRUSR|S_IWUSR|S_IRGRP|S_IWGRP|S_IROTH|S_IWOTH)) < 0) { + error("error creating featureList file", "featureList.txt", "open"); + } + int times = dbH->flags & O2_FLAG_TIMES; + if (times) { + if ((tLfd = open("timesList.txt", O_CREAT|O_RDWR|O_EXCL, S_IRUSR|S_IWUSR|S_IRGRP|S_IWGRP|S_IROTH|S_IWOTH)) < 0) { + error("error creating timesList file", "timesList.txt", "open"); + } + } + if ((kLfd = open("keyList.txt", O_CREAT|O_RDWR|O_EXCL, S_IRUSR|S_IWUSR|S_IRGRP|S_IWGRP|S_IROTH|S_IWOTH)) < 0) { + error("error creating keyList file", "keyList.txt", "open"); + } + + /* can these fail? I sincerely hope not. */ + fLFile = fdopen(fLfd, "w"); + if (times) { + tLFile = fdopen(tLfd, "w"); + } + kLFile = fdopen(kLfd, "w"); + + char *fName = new char[256]; + int ffd; + FILE *tFile; + unsigned pos = 0; + for(unsigned k = 0; k < dbH->numFiles; k++) { + fprintf(kLFile, "%s\n", fileTable + k*O2_FILETABLESIZE); + snprintf(fName, 256, "%05d.features", k); + if ((ffd = open(fName, O_CREAT|O_RDWR|O_EXCL, S_IRUSR|S_IWUSR|S_IRGRP|S_IWGRP|S_IROTH|S_IWOTH)) < 0) { + error("error creating feature file", fName, "open"); + } + if ((write(ffd, &dbH->dim, sizeof(uint32_t))) < 0) { + error("error writing dimensions", fName, "write"); + } + + if ((write(ffd, dataBuf + pos * dbH->dim, trackTable[k] * dbH->dim * sizeof(double))) < 0) { + error("error writing data", fName, "write"); + } + fprintf(fLFile, "%s\n", fName); + close(ffd); + + if(times) { + snprintf(fName, 256, "%05d.times", k); + tFile = fopen(fName, "w"); + for(unsigned i = 0; i < trackTable[k]; i++) { + // KLUDGE: specifying 16 digits of precision after the decimal + // point is (but check this!) sufficient to uniquely identify + // doubles; however, that will cause ugliness, as that's + // vastly too many for most values of interest. Moving to %a + // here and scanf() in the timesFile reading might fix this. + // -- CSR, 2007-10-19 + fprintf(tFile, "%.16e\n", *(timesTable + pos + i)); + } + fprintf(tLFile, "%s\n", fName); + } + + pos += trackTable[k]; + cout << fileTable+k*O2_FILETABLESIZE << " " << trackTable[k] << endl; + } + + FILE *scriptFile; + scriptFile = fopen("restore.sh", "w"); + fprintf(scriptFile, "\ +#! /bin/sh\n\ +#\n\ +# usage: AUDIODB=/path/to/audioDB sh ./restore.sh <newdb>\n\ +\n\ +if [ -z \"${AUDIODB}\" ]; then echo set AUDIODB variable; exit 1; fi\n\ +if [ -z \"$1\" ]; then echo usage: $0 newdb; exit 1; fi\n\n\ +\"${AUDIODB}\" -d \"$1\" -N --size=%d\n", dbH->dbSize / 1000000); + if(dbH->flags & O2_FLAG_L2NORM) { + fprintf(scriptFile, "\"${AUDIODB}\" -d \"$1\" -L\n"); + } + fprintf(scriptFile, "\"${AUDIODB}\" -d \"$1\" -B -F featureList.txt -K keyList.txt"); + if(times) { + fprintf(scriptFile, " -T timesList.txt"); + } + fprintf(scriptFile, "\n"); + fclose(scriptFile); + + if((chdir(cwd)) < 0) { + error("error changing working directory", cwd, "chdir"); + } + + fclose(fLFile); + if(times) { + fclose(tLFile); + } + fclose(kLFile); + delete[] fName; + status(dbName); } @@ -880,19 +997,19 @@ -void audioDB::query(const char* dbName, const char* inFile, adb__queryResult *adbQueryResult){ +void audioDB::query(const char* dbName, const char* inFile, adb__queryResponse *adbQueryResponse){ switch(queryType){ case O2_POINT_QUERY: - pointQuery(dbName, inFile, adbQueryResult); + pointQuery(dbName, inFile, adbQueryResponse); break; case O2_SEQUENCE_QUERY: if(radius==0) - trackSequenceQueryNN(dbName, inFile, adbQueryResult); + trackSequenceQueryNN(dbName, inFile, adbQueryResponse); else - trackSequenceQueryRad(dbName, inFile, adbQueryResult); + trackSequenceQueryRad(dbName, inFile, adbQueryResponse); break; case O2_TRACK_QUERY: - trackPointQuery(dbName, inFile, adbQueryResult); + trackPointQuery(dbName, inFile, adbQueryResponse); break; default: error("unrecognized queryType in query()"); @@ -910,7 +1027,7 @@ } // Basic point query engine -void audioDB::pointQuery(const char* dbName, const char* inFile, adb__queryResult *adbQueryResult){ +void audioDB::pointQuery(const char* dbName, const char* inFile, adb__queryResponse *adbQueryResponse){ initTables(dbName, 0, inFile); @@ -1031,7 +1148,7 @@ cerr << endl << " elapsed time:" << ( tv2.tv_sec*1000 + tv2.tv_usec/1000 ) - ( tv1.tv_sec*1000+tv1.tv_usec/1000 ) << " msec" << endl; } - if(adbQueryResult==0){ + if(adbQueryResponse==0){ // Output answer // Loop over nearest neighbours for(k=0; k < pointNN; k++){ @@ -1055,27 +1172,27 @@ } listLen = k; - adbQueryResult->__sizeRlist=listLen; - adbQueryResult->__sizeDist=listLen; - adbQueryResult->__sizeQpos=listLen; - adbQueryResult->__sizeSpos=listLen; - adbQueryResult->Rlist= new char*[listLen]; - adbQueryResult->Dist = new double[listLen]; - adbQueryResult->Qpos = new unsigned int[listLen]; - adbQueryResult->Spos = new unsigned int[listLen]; - for(k=0; k<(unsigned)adbQueryResult->__sizeRlist; k++){ - adbQueryResult->Rlist[k]=new char[O2_MAXFILESTR]; - adbQueryResult->Dist[k]=distances[k]; - adbQueryResult->Qpos[k]=qIndexes[k]; + adbQueryResponse->result.__sizeRlist=listLen; + adbQueryResponse->result.__sizeDist=listLen; + adbQueryResponse->result.__sizeQpos=listLen; + adbQueryResponse->result.__sizeSpos=listLen; + adbQueryResponse->result.Rlist= new char*[listLen]; + adbQueryResponse->result.Dist = new double[listLen]; + adbQueryResponse->result.Qpos = new unsigned int[listLen]; + adbQueryResponse->result.Spos = new unsigned int[listLen]; + for(k=0; k<(unsigned)adbQueryResponse->result.__sizeRlist; k++){ + adbQueryResponse->result.Rlist[k]=new char[O2_MAXFILESTR]; + adbQueryResponse->result.Dist[k]=distances[k]; + adbQueryResponse->result.Qpos[k]=qIndexes[k]; unsigned cumTrack=0; for(l=0 ; l<dbH->numFiles; l++){ cumTrack+=trackTable[l]; if(sIndexes[k]<cumTrack){ - sprintf(adbQueryResult->Rlist[k], "%s", fileTable+l*O2_FILETABLESIZE); + sprintf(adbQueryResponse->result.Rlist[k], "%s", fileTable+l*O2_FILETABLESIZE); break; } } - adbQueryResult->Spos[k]=sIndexes[k]+trackTable[l]-cumTrack; + adbQueryResponse->result.Spos[k]=sIndexes[k]+trackTable[l]-cumTrack; } } @@ -1093,7 +1210,7 @@ // trackPointQuery // return the trackNN closest tracks to the query track // uses average of pointNN points per track -void audioDB::trackPointQuery(const char* dbName, const char* inFile, adb__queryResult *adbQueryResult){ +void audioDB::trackPointQuery(const char* dbName, const char* inFile, adb__queryResponse *adbQueryResponse){ initTables(dbName, 0, inFile); // For each input vector, find the closest pointNN matching output vectors and report @@ -1298,7 +1415,7 @@ << " elapsed time:" << ( tv2.tv_sec*1000 + tv2.tv_usec/1000 ) - ( tv1.tv_sec*1000+tv1.tv_usec/1000 ) << " msec" << endl; } - if(adbQueryResult==0){ + if(adbQueryResponse==0){ if(verbosity>1) { cerr<<endl; } @@ -1310,20 +1427,20 @@ } else{ // Process Web Services Query int listLen = min(trackNN, processedTracks); - adbQueryResult->__sizeRlist=listLen; - adbQueryResult->__sizeDist=listLen; - adbQueryResult->__sizeQpos=listLen; - adbQueryResult->__sizeSpos=listLen; - adbQueryResult->Rlist= new char*[listLen]; - adbQueryResult->Dist = new double[listLen]; - adbQueryResult->Qpos = new unsigned int[listLen]; - adbQueryResult->Spos = new unsigned int[listLen]; - for(k=0; k<(unsigned)adbQueryResult->__sizeRlist; k++){ - adbQueryResult->Rlist[k]=new char[O2_MAXFILESTR]; - adbQueryResult->Dist[k]=trackDistances[k]; - adbQueryResult->Qpos[k]=trackQIndexes[k]; - adbQueryResult->Spos[k]=trackSIndexes[k]; - sprintf(adbQueryResult->Rlist[k], "%s", fileTable+trackIDs[k]*O2_FILETABLESIZE); + adbQueryResponse->result.__sizeRlist=listLen; + adbQueryResponse->result.__sizeDist=listLen; + adbQueryResponse->result.__sizeQpos=listLen; + adbQueryResponse->result.__sizeSpos=listLen; + adbQueryResponse->result.Rlist= new char*[listLen]; + adbQueryResponse->result.Dist = new double[listLen]; + adbQueryResponse->result.Qpos = new unsigned int[listLen]; + adbQueryResponse->result.Spos = new unsigned int[listLen]; + for(k=0; k<(unsigned)adbQueryResponse->result.__sizeRlist; k++){ + adbQueryResponse->result.Rlist[k]=new char[O2_MAXFILESTR]; + adbQueryResponse->result.Dist[k]=trackDistances[k]; + adbQueryResponse->result.Qpos[k]=trackQIndexes[k]; + adbQueryResponse->result.Spos[k]=trackSIndexes[k]; + sprintf(adbQueryResponse->result.Rlist[k], "%s", fileTable+trackIDs[k]*O2_FILETABLESIZE); } } @@ -1347,7 +1464,7 @@ // efficient implementation based on matched filter // assumes normed shingles // outputs distances of retrieved shingles, max retreived = pointNN shingles per per track -void audioDB::trackSequenceQueryNN(const char* dbName, const char* inFile, adb__queryResult *adbQueryResult){ +void audioDB::trackSequenceQueryNN(const char* dbName, const char* inFile, adb__queryResponse *adbQueryResponse){ initTables(dbName, 0, inFile); @@ -1797,7 +1914,7 @@ cerr << "sampleCount: " << sampleCount << " sampleSum: " << sampleSum << " logSampleSum: " << logSampleSum << " minSample: " << minSample << " maxSample: " << maxSample << endl; } - if(adbQueryResult==0){ + if(adbQueryResponse==0){ if(verbosity>1) { cerr<<endl; } @@ -1809,20 +1926,20 @@ } else{ // Process Web Services Query int listLen = min(trackNN, processedTracks); - adbQueryResult->__sizeRlist=listLen; - adbQueryResult->__sizeDist=listLen; - adbQueryResult->__sizeQpos=listLen; - adbQueryResult->__sizeSpos=listLen; - adbQueryResult->Rlist= new char*[listLen]; - adbQueryResult->Dist = new double[listLen]; - adbQueryResult->Qpos = new unsigned int[listLen]; - adbQueryResult->Spos = new unsigned int[listLen]; - for(k=0; k<(unsigned)adbQueryResult->__sizeRlist; k++){ - adbQueryResult->Rlist[k]=new char[O2_MAXFILESTR]; - adbQueryResult->Dist[k]=trackDistances[k]; - adbQueryResult->Qpos[k]=trackQIndexes[k]; - adbQueryResult->Spos[k]=trackSIndexes[k]; - sprintf(adbQueryResult->Rlist[k], "%s", fileTable+trackIDs[k]*O2_FILETABLESIZE); + adbQueryResponse->result.__sizeRlist=listLen; + adbQueryResponse->result.__sizeDist=listLen; + adbQueryResponse->result.__sizeQpos=listLen; + adbQueryResponse->result.__sizeSpos=listLen; + adbQueryResponse->result.Rlist= new char*[listLen]; + adbQueryResponse->result.Dist = new double[listLen]; + adbQueryResponse->result.Qpos = new unsigned int[listLen]; + adbQueryResponse->result.Spos = new unsigned int[listLen]; + for(k=0; k<(unsigned)adbQueryResponse->result.__sizeRlist; k++){ + adbQueryResponse->result.Rlist[k]=new char[O2_MAXFILESTR]; + adbQueryResponse->result.Dist[k]=trackDistances[k]; + adbQueryResponse->result.Qpos[k]=trackQIndexes[k]; + adbQueryResponse->result.Spos[k]=trackSIndexes[k]; + sprintf(adbQueryResponse->result.Rlist[k], "%s", fileTable+trackIDs[k]*O2_FILETABLESIZE); } } @@ -1854,7 +1971,7 @@ // efficient implementation based on matched filter // assumes normed shingles // outputs count of retrieved shingles, max retreived = one shingle per query shingle per track -void audioDB::trackSequenceQueryRad(const char* dbName, const char* inFile, adb__queryResult *adbQueryResult){ +void audioDB::trackSequenceQueryRad(const char* dbName, const char* inFile, adb__queryResponse *adbQueryResponse){ initTables(dbName, 0, inFile); @@ -2279,7 +2396,7 @@ << " minSample: " << minSample << " maxSample: " << maxSample << endl; } - if(adbQueryResult==0){ + if(adbQueryResponse==0){ if(verbosity>1) { cerr<<endl; } @@ -2290,20 +2407,20 @@ } else{ // Process Web Services Query int listLen = min(trackNN, processedTracks); - adbQueryResult->__sizeRlist=listLen; - adbQueryResult->__sizeDist=listLen; - adbQueryResult->__sizeQpos=listLen; - adbQueryResult->__sizeSpos=listLen; - adbQueryResult->Rlist= new char*[listLen]; - adbQueryResult->Dist = new double[listLen]; - adbQueryResult->Qpos = new unsigned int[listLen]; - adbQueryResult->Spos = new unsigned int[listLen]; - for(k=0; k<(unsigned)adbQueryResult->__sizeRlist; k++){ - adbQueryResult->Rlist[k]=new char[O2_MAXFILESTR]; - adbQueryResult->Dist[k]=trackDistances[k]; - adbQueryResult->Qpos[k]=trackQIndexes[k]; - adbQueryResult->Spos[k]=trackSIndexes[k]; - sprintf(adbQueryResult->Rlist[k], "%s", fileTable+trackIDs[k]*O2_FILETABLESIZE); + adbQueryResponse->result.__sizeRlist=listLen; + adbQueryResponse->result.__sizeDist=listLen; + adbQueryResponse->result.__sizeQpos=listLen; + adbQueryResponse->result.__sizeSpos=listLen; + adbQueryResponse->result.Rlist= new char*[listLen]; + adbQueryResponse->result.Dist = new double[listLen]; + adbQueryResponse->result.Qpos = new unsigned int[listLen]; + adbQueryResponse->result.Spos = new unsigned int[listLen]; + for(k=0; k<(unsigned)adbQueryResponse->result.__sizeRlist; k++){ + adbQueryResponse->result.Rlist[k]=new char[O2_MAXFILESTR]; + adbQueryResponse->result.Dist[k]=trackDistances[k]; + adbQueryResponse->result.Qpos[k]=trackQIndexes[k]; + adbQueryResponse->result.Spos[k]=trackSIndexes[k]; + sprintf(adbQueryResponse->result.Rlist[k], "%s", fileTable+trackIDs[k]*O2_FILETABLESIZE); } } @@ -2462,11 +2579,11 @@ // web services // SERVER SIDE -int adb__status(struct soap* soap, xsd__string dbName, adb__statusResult &adbStatusResult){ +int adb__status(struct soap* soap, xsd__string dbName, adb__statusResponse &adbStatusResponse){ char* const argv[]={"audioDB",COM_STATUS,"-d",dbName}; const unsigned argc = 4; try { - audioDB(argc, argv, &adbStatusResult); + audioDB(argc, argv, &adbStatusResponse); return SOAP_OK; } catch(char *err) { soap_receiver_fault(soap, err, ""); @@ -2476,7 +2593,7 @@ // Literal translation of command line to web service -int adb__query(struct soap* soap, xsd__string dbName, xsd__string qKey, xsd__string keyList, xsd__string timesFileName, xsd__int qType, xsd__int qPos, xsd__int pointNN, xsd__int trackNN, xsd__int seqLen, adb__queryResult &adbQueryResult){ +int adb__query(struct soap* soap, xsd__string dbName, xsd__string qKey, xsd__string keyList, xsd__string timesFileName, xsd__int qType, xsd__int qPos, xsd__int pointNN, xsd__int trackNN, xsd__int seqLen, adb__queryResponse &adbQueryResponse){ char queryType[256]; for(int k=0; k<256; k++) queryType[k]='\0'; @@ -2529,7 +2646,7 @@ const unsigned argc = 19; try { - audioDB(argc, (char* const*)argv, &adbQueryResult); + audioDB(argc, (char* const*)argv, &adbQueryResponse); return SOAP_OK; } catch (char *err) { soap_receiver_fault(soap, err, "");
--- a/audioDB.h Wed Oct 17 14:52:39 2007 +0000 +++ b/audioDB.h Tue Oct 23 19:57:06 2007 +0000 @@ -96,13 +96,15 @@ // unportable between 32 and 64-bit architectures. Making them // uint32_t isn't the real answer, as it means we won't be able to // scale to really large collections easily but it works around the - // problem. -- CSR, 2007-10-05 + // problem. Expanding to 64 bits will of course need a change in + // file format version. -- CSR, 2007-10-05 uint32_t length; uint32_t fileTableOffset; uint32_t trackTableOffset; uint32_t dataOffset; uint32_t l2normTableOffset; uint32_t timesTableOffset; + uint32_t dbSize; } dbTableHeaderT, *dbTableHeaderPtr; @@ -118,6 +120,7 @@ const char* trackFileName; ifstream *trackFile; const char *command; + const char *output; const char *timesFileName; ifstream *timesFile; @@ -139,6 +142,7 @@ // Flags and parameters unsigned verbosity; // how much do we want to know? + unsigned size; // given size (for creation) unsigned queryType; // point queries default unsigned pointNN; // how many point NNs ? unsigned trackNN; // how many track NNs ? @@ -159,10 +163,10 @@ // private methods void error(const char* a, const char* b = "", const char *sysFunc = 0); - void pointQuery(const char* dbName, const char* inFile, adb__queryResult *adbQueryResult=0); - void trackPointQuery(const char* dbName, const char* inFile, adb__queryResult *adbQueryResult=0); - void trackSequenceQueryNN(const char* dbName, const char* inFile, adb__queryResult *adbQueryResult=0); - void trackSequenceQueryRad(const char* dbName, const char* inFile, adb__queryResult *adbQueryResult=0); + void pointQuery(const char* dbName, const char* inFile, adb__queryResponse *adbQueryResponse=0); + void trackPointQuery(const char* dbName, const char* inFile, adb__queryResponse *adbQueryResponse=0); + void trackSequenceQueryNN(const char* dbName, const char* inFile, adb__queryResponse *adbQueryResponse=0); + void trackSequenceQueryRad(const char* dbName, const char* inFile, adb__queryResponse *adbQueryResponse=0); void initDBHeader(const char *dbName, bool forWrite); void initTables(const char* dbName, bool forWrite, const char* inFile); @@ -173,8 +177,8 @@ public: audioDB(const unsigned argc, char* const argv[]); - audioDB(const unsigned argc, char* const argv[], adb__queryResult *adbQueryResult); - audioDB(const unsigned argc, char* const argv[], adb__statusResult *adbStatusResult); + audioDB(const unsigned argc, char* const argv[], adb__queryResponse *adbQueryResponse); + audioDB(const unsigned argc, char* const argv[], adb__statusResponse *adbStatusResponse); void cleanup(); ~audioDB(); int processArgs(const unsigned argc, char* const argv[]); @@ -184,8 +188,8 @@ void drop(); void insert(const char* dbName, const char* inFile); void batchinsert(const char* dbName, const char* inFile); - void query(const char* dbName, const char* inFile, adb__queryResult *adbQueryResult=0); - void status(const char* dbName, adb__statusResult *adbStatusResult=0); + void query(const char* dbName, const char* inFile, adb__queryResponse *adbQueryResponse=0); + void status(const char* dbName, adb__statusResponse *adbStatusResponse=0); void ws_status(const char*dbName, char* hostport); void ws_query(const char*dbName, const char *trackKey, const char* hostport); void l2norm(const char* dbName); @@ -204,6 +208,7 @@ trackFileName(0), \ trackFile(0), \ command(0), \ + output(0), \ timesFileName(0), \ timesFile(0), \ dbfid(0), \ @@ -218,6 +223,7 @@ qNorm(0), \ timesTable(0), \ verbosity(1), \ + size(O2_DEFAULTDBSIZE), \ queryType(O2_POINT_QUERY), \ pointNN(O2_DEFAULT_POINTNN), \ trackNN(O2_DEFAULT_TRACKNN), \
--- a/audioDBws.h Wed Oct 17 14:52:39 2007 +0000 +++ b/audioDBws.h Tue Oct 23 19:57:06 2007 +0000 @@ -12,7 +12,7 @@ typedef char* xsd__string; // Supports result lists of arbitrary length -class adb__queryResult{ +struct adb__queryResult{ int __sizeRlist; char **Rlist; // Maximum size of result list int __sizeDist; @@ -23,7 +23,7 @@ unsigned int *Spos; }; -class adb__statusResult { +struct adb__statusResult { unsigned numFiles; unsigned dim; unsigned length; @@ -31,9 +31,17 @@ unsigned nullCount; unsigned flags; }; - + +struct adb__statusResponse { + struct adb__statusResult result; +}; + +struct adb__queryResponse { + struct adb__queryResult result; +}; + // Print the status of an existing adb database -int adb__status(xsd__string dbName, adb__statusResult &adbStatusResult); +int adb__status(xsd__string dbName, struct adb__statusResponse &adbStatusResponse); // Query an existing adb database -int adb__query(xsd__string dbName, xsd__string qKey, xsd__string keyList, xsd__string timesFileName, xsd__int qType, xsd__int qPos, xsd__int pointNN, xsd__int segNN, xsd__int segLen, adb__queryResult &adbQueryResult); +int adb__query(xsd__string dbName, xsd__string qKey, xsd__string keyList, xsd__string timesFileName, xsd__int qType, xsd__int qPos, xsd__int pointNN, xsd__int segNN, xsd__int segLen, struct adb__queryResponse &adbQueryResponse);
--- a/debian/changelog Wed Oct 17 14:52:39 2007 +0000 +++ b/debian/changelog Tue Oct 23 19:57:06 2007 +0000 @@ -1,3 +1,9 @@ +audiodb (1.0-13) unstable; urgency=low + + * Updated to svn version #155 + + -- Christophe Rhodes <c.rhodes@gold.ac.uk> Tue, 23 Oct 2007 19:41:27 +0100 + audiodb (1.0-12) unstable; urgency=low * Updated to svn version #145
--- a/debian/docs Wed Oct 17 14:52:39 2007 +0000 +++ b/debian/docs Tue Oct 23 19:57:06 2007 +0000 @@ -1,2 +1,5 @@ docs/TODO.txt +HELP.txt README.txt +INSTALL.txt +
--- a/debian/rules Wed Oct 17 14:52:39 2007 +0000 +++ b/debian/rules Tue Oct 23 19:57:06 2007 +0000 @@ -36,7 +36,7 @@ # Add here commands to compile the package. $(MAKE) EXECUTABLE=audiodb $(MAKE) EXECUTABLE=audiodb audiodb.1 - $(MAKE) EXECUTABLE=audiodb README.txt + $(MAKE) EXECUTABLE=audiodb HELP.txt touch build-stamp
--- a/gengetopt.in Wed Oct 17 14:52:39 2007 +0000 +++ b/gengetopt.in Tue Oct 23 19:57:06 2007 +0000 @@ -1,37 +1,3 @@ -# POSIX-COMPLIANT COMMAND LINE ARGUMENTS: -# -# -v --verbosity -# -# -d --database dbName -# -N --NEW -# -S --STATUS -# -D --DUMP -# -L --L2NORM -# -# -f --features features.bin -# -k --key key -# -t --timePoints.txt -# -I --INSERT -# -U --UPDATE -# -# -F --featureList featureList.txt -# -K --keyList keyList.txt -# -T --timePointsList.txt -# -B --BATCHINSERT dbName -# -# -Q --QUERY {point|track|sequence} -# -p --qpoint - ordinal position of query vector in feature input file -# -n --pointnn n - number of nearest neigbour points per track -# -r --resultlength n - number of tracks (nearest neighbours) to return -# -l --seqlen len - length of sequence -# -h --seqhop hop - hop size of sequence -# -R --radius - radius-based search, return all points/tracks/sequences < radius (0...Inf) -# -x --x - time expand (compress) factor (ratio of result length to query length (and visa-versa)) [1..Inf] -# -o --rotate - rotate the query feature vectors on search -# -# -s --SERVER port -# -c --client host:port - package "audioDB" version "version 1.0" purpose "A feature vector database management system for content-based retrieval." @@ -43,9 +9,12 @@ option "database" d "database file required by Database commands." string typestr="filename" optional option "NEW" N "make a new (initially empty) database." dependon="database" optional +option "size" - "size of database file (in MB)" int dependon="NEW" default="2000" optional option "STATUS" S "output database information to stdout." dependon="database" optional option "DUMP" D "output all entries: index key size." dependon="database" optional +option "output" - "output directory" string dependon="DUMP" default="audioDB.dump" optional option "L2NORM" L "unit norm vectors and norm all future inserts." dependon="database" optional + section "Database Insertion" sectiondesc="The following commands insert feature files, with optional keys and timestamps.\n" option "INSERT" I "add feature vectors to an existing database." dependon="features" optional option "UPDATE" U "replace inserted vectors associated with key with new input vectors." dependon="features" dependon="key" dependon="database" optional hidden
--- a/gpl.txt Wed Oct 17 14:52:39 2007 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,339 +0,0 @@ - GNU GENERAL PUBLIC LICENSE - Version 2, June 1991 - - Copyright (C) 1989, 1991 Free Software Foundation, Inc., - 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA - Everyone is permitted to copy and distribute verbatim copies - of this license document, but changing it is not allowed. - - Preamble - - The licenses for most software are designed to take away your -freedom to share and change it. By contrast, the GNU General Public -License is intended to guarantee your freedom to share and change free -software--to make sure the software is free for all its users. This -General Public License applies to most of the Free Software -Foundation's software and to any other program whose authors commit to -using it. (Some other Free Software Foundation software is covered by -the GNU Lesser General Public License instead.) You can apply it to -your programs, too. - - When we speak of free software, we are referring to freedom, not -price. Our General Public Licenses are designed to make sure that you -have the freedom to distribute copies of free software (and charge for -this service if you wish), that you receive source code or can get it -if you want it, that you can change the software or use pieces of it -in new free programs; and that you know you can do these things. - - To protect your rights, we need to make restrictions that forbid -anyone to deny you these rights or to ask you to surrender the rights. -These restrictions translate to certain responsibilities for you if you -distribute copies of the software, or if you modify it. - - For example, if you distribute copies of such a program, whether -gratis or for a fee, you must give the recipients all the rights that -you have. You must make sure that they, too, receive or can get the -source code. And you must show them these terms so they know their -rights. - - We protect your rights with two steps: (1) copyright the software, and -(2) offer you this license which gives you legal permission to copy, -distribute and/or modify the software. - - Also, for each author's protection and ours, we want to make certain -that everyone understands that there is no warranty for this free -software. If the software is modified by someone else and passed on, we -want its recipients to know that what they have is not the original, so -that any problems introduced by others will not reflect on the original -authors' reputations. - - Finally, any free program is threatened constantly by software -patents. We wish to avoid the danger that redistributors of a free -program will individually obtain patent licenses, in effect making the -program proprietary. To prevent this, we have made it clear that any -patent must be licensed for everyone's free use or not licensed at all. - - The precise terms and conditions for copying, distribution and -modification follow. - - GNU GENERAL PUBLIC LICENSE - TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION - - 0. This License applies to any program or other work which contains -a notice placed by the copyright holder saying it may be distributed -under the terms of this General Public License. The "Program", below, -refers to any such program or work, and a "work based on the Program" -means either the Program or any derivative work under copyright law: -that is to say, a work containing the Program or a portion of it, -either verbatim or with modifications and/or translated into another -language. (Hereinafter, translation is included without limitation in -the term "modification".) Each licensee is addressed as "you". - -Activities other than copying, distribution and modification are not -covered by this License; they are outside its scope. The act of -running the Program is not restricted, and the output from the Program -is covered only if its contents constitute a work based on the -Program (independent of having been made by running the Program). -Whether that is true depends on what the Program does. - - 1. You may copy and distribute verbatim copies of the Program's -source code as you receive it, in any medium, provided that you -conspicuously and appropriately publish on each copy an appropriate -copyright notice and disclaimer of warranty; keep intact all the -notices that refer to this License and to the absence of any warranty; -and give any other recipients of the Program a copy of this License -along with the Program. - -You may charge a fee for the physical act of transferring a copy, and -you may at your option offer warranty protection in exchange for a fee. - - 2. You may modify your copy or copies of the Program or any portion -of it, thus forming a work based on the Program, and copy and -distribute such modifications or work under the terms of Section 1 -above, provided that you also meet all of these conditions: - - a) You must cause the modified files to carry prominent notices - stating that you changed the files and the date of any change. - - b) You must cause any work that you distribute or publish, that in - whole or in part contains or is derived from the Program or any - part thereof, to be licensed as a whole at no charge to all third - parties under the terms of this License. - - c) If the modified program normally reads commands interactively - when run, you must cause it, when started running for such - interactive use in the most ordinary way, to print or display an - announcement including an appropriate copyright notice and a - notice that there is no warranty (or else, saying that you provide - a warranty) and that users may redistribute the program under - these conditions, and telling the user how to view a copy of this - License. (Exception: if the Program itself is interactive but - does not normally print such an announcement, your work based on - the Program is not required to print an announcement.) - -These requirements apply to the modified work as a whole. If -identifiable sections of that work are not derived from the Program, -and can be reasonably considered independent and separate works in -themselves, then this License, and its terms, do not apply to those -sections when you distribute them as separate works. But when you -distribute the same sections as part of a whole which is a work based -on the Program, the distribution of the whole must be on the terms of -this License, whose permissions for other licensees extend to the -entire whole, and thus to each and every part regardless of who wrote it. - -Thus, it is not the intent of this section to claim rights or contest -your rights to work written entirely by you; rather, the intent is to -exercise the right to control the distribution of derivative or -collective works based on the Program. - -In addition, mere aggregation of another work not based on the Program -with the Program (or with a work based on the Program) on a volume of -a storage or distribution medium does not bring the other work under -the scope of this License. - - 3. You may copy and distribute the Program (or a work based on it, -under Section 2) in object code or executable form under the terms of -Sections 1 and 2 above provided that you also do one of the following: - - a) Accompany it with the complete corresponding machine-readable - source code, which must be distributed under the terms of Sections - 1 and 2 above on a medium customarily used for software interchange; or, - - b) Accompany it with a written offer, valid for at least three - years, to give any third party, for a charge no more than your - cost of physically performing source distribution, a complete - machine-readable copy of the corresponding source code, to be - distributed under the terms of Sections 1 and 2 above on a medium - customarily used for software interchange; or, - - c) Accompany it with the information you received as to the offer - to distribute corresponding source code. (This alternative is - allowed only for noncommercial distribution and only if you - received the program in object code or executable form with such - an offer, in accord with Subsection b above.) - -The source code for a work means the preferred form of the work for -making modifications to it. For an executable work, complete source -code means all the source code for all modules it contains, plus any -associated interface definition files, plus the scripts used to -control compilation and installation of the executable. However, as a -special exception, the source code distributed need not include -anything that is normally distributed (in either source or binary -form) with the major components (compiler, kernel, and so on) of the -operating system on which the executable runs, unless that component -itself accompanies the executable. - -If distribution of executable or object code is made by offering -access to copy from a designated place, then offering equivalent -access to copy the source code from the same place counts as -distribution of the source code, even though third parties are not -compelled to copy the source along with the object code. - - 4. You may not copy, modify, sublicense, or distribute the Program -except as expressly provided under this License. Any attempt -otherwise to copy, modify, sublicense or distribute the Program is -void, and will automatically terminate your rights under this License. -However, parties who have received copies, or rights, from you under -this License will not have their licenses terminated so long as such -parties remain in full compliance. - - 5. You are not required to accept this License, since you have not -signed it. However, nothing else grants you permission to modify or -distribute the Program or its derivative works. These actions are -prohibited by law if you do not accept this License. Therefore, by -modifying or distributing the Program (or any work based on the -Program), you indicate your acceptance of this License to do so, and -all its terms and conditions for copying, distributing or modifying -the Program or works based on it. - - 6. Each time you redistribute the Program (or any work based on the -Program), the recipient automatically receives a license from the -original licensor to copy, distribute or modify the Program subject to -these terms and conditions. You may not impose any further -restrictions on the recipients' exercise of the rights granted herein. -You are not responsible for enforcing compliance by third parties to -this License. - - 7. If, as a consequence of a court judgment or allegation of patent -infringement or for any other reason (not limited to patent issues), -conditions are imposed on you (whether by court order, agreement or -otherwise) that contradict the conditions of this License, they do not -excuse you from the conditions of this License. If you cannot -distribute so as to satisfy simultaneously your obligations under this -License and any other pertinent obligations, then as a consequence you -may not distribute the Program at all. For example, if a patent -license would not permit royalty-free redistribution of the Program by -all those who receive copies directly or indirectly through you, then -the only way you could satisfy both it and this License would be to -refrain entirely from distribution of the Program. - -If any portion of this section is held invalid or unenforceable under -any particular circumstance, the balance of the section is intended to -apply and the section as a whole is intended to apply in other -circumstances. - -It is not the purpose of this section to induce you to infringe any -patents or other property right claims or to contest validity of any -such claims; this section has the sole purpose of protecting the -integrity of the free software distribution system, which is -implemented by public license practices. Many people have made -generous contributions to the wide range of software distributed -through that system in reliance on consistent application of that -system; it is up to the author/donor to decide if he or she is willing -to distribute software through any other system and a licensee cannot -impose that choice. - -This section is intended to make thoroughly clear what is believed to -be a consequence of the rest of this License. - - 8. If the distribution and/or use of the Program is restricted in -certain countries either by patents or by copyrighted interfaces, the -original copyright holder who places the Program under this License -may add an explicit geographical distribution limitation excluding -those countries, so that distribution is permitted only in or among -countries not thus excluded. In such case, this License incorporates -the limitation as if written in the body of this License. - - 9. The Free Software Foundation may publish revised and/or new versions -of the General Public License from time to time. Such new versions will -be similar in spirit to the present version, but may differ in detail to -address new problems or concerns. - -Each version is given a distinguishing version number. If the Program -specifies a version number of this License which applies to it and "any -later version", you have the option of following the terms and conditions -either of that version or of any later version published by the Free -Software Foundation. If the Program does not specify a version number of -this License, you may choose any version ever published by the Free Software -Foundation. - - 10. If you wish to incorporate parts of the Program into other free -programs whose distribution conditions are different, write to the author -to ask for permission. For software which is copyrighted by the Free -Software Foundation, write to the Free Software Foundation; we sometimes -make exceptions for this. Our decision will be guided by the two goals -of preserving the free status of all derivatives of our free software and -of promoting the sharing and reuse of software generally. - - NO WARRANTY - - 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY -FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN -OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES -PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED -OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF -MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS -TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE -PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, -REPAIR OR CORRECTION. - - 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING -WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR -REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, -INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING -OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED -TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY -YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER -PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE -POSSIBILITY OF SUCH DAMAGES. - - END OF TERMS AND CONDITIONS - - How to Apply These Terms to Your New Programs - - If you develop a new program, and you want it to be of the greatest -possible use to the public, the best way to achieve this is to make it -free software which everyone can redistribute and change under these terms. - - To do so, attach the following notices to the program. It is safest -to attach them to the start of each source file to most effectively -convey the exclusion of warranty; and each file should have at least -the "copyright" line and a pointer to where the full notice is found. - - <one line to give the program's name and a brief idea of what it does.> - Copyright (C) <year> <name of author> - - This program is free software; you can redistribute it and/or modify - it under the terms of the GNU General Public License as published by - the Free Software Foundation; either version 2 of the License, or - (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License along - with this program; if not, write to the Free Software Foundation, Inc., - 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. - -Also add information on how to contact you by electronic and paper mail. - -If the program is interactive, make it output a short notice like this -when it starts in an interactive mode: - - Gnomovision version 69, Copyright (C) year name of author - Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'. - This is free software, and you are welcome to redistribute it - under certain conditions; type `show c' for details. - -The hypothetical commands `show w' and `show c' should show the appropriate -parts of the General Public License. Of course, the commands you use may -be called something other than `show w' and `show c'; they could even be -mouse-clicks or menu items--whatever suits your program. - -You should also get your employer (if you work as a programmer) or your -school, if any, to sign a "copyright disclaimer" for the program, if -necessary. Here is a sample; alter the names: - - Yoyodyne, Inc., hereby disclaims all copyright interest in the program - `Gnomovision' (which makes passes at compilers) written by James Hacker. - - <signature of Ty Coon>, 1 April 1989 - Ty Coon, President of Vice - -This General Public License does not permit incorporating your program into -proprietary programs. If your program is a subroutine library, you may -consider it more useful to permit linking proprietary applications with the -library. If this is what you want to do, use the GNU Lesser General -Public License instead of this License.
--- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/tests/0024/run-test.sh Tue Oct 23 19:57:06 2007 +0000 @@ -0,0 +1,38 @@ +#! /bin/sh + +. ../test-utils.sh + +if [ -f testdb ]; then rm -f testdb; fi + +${AUDIODB} -d testdb -N + +intstring 2 > testfeature01 +floatstring 0 1 >> testfeature01 +intstring 2 > testfeature10 +floatstring 1 0 >> testfeature10 + +cat > testfeaturefiles <<EOF +testfeature01 +testfeature10 +EOF + +${AUDIODB} -d testdb -B -F testfeaturefiles + +# sequence queries require L2NORM +${AUDIODB} -d testdb -L + +echo "exhaustive search" +intstring 2 > testquery +floatstring 0 0.5 >> testquery +floatstring 0.5 0 >> testquery + +${AUDIODB} -d testdb -Q sequence -l 1 -f testquery -e > testoutput +echo testfeature01 1 0 0 > test-expected-output +echo testfeature10 1 1 0 >> test-expected-output +cmp testoutput test-expected-output +${AUDIODB} -d testdb -Q sequence -l 1 -f testquery -n 1 -e > testoutput +echo testfeature01 0 0 0 > test-expected-output +echo testfeature10 0 1 0 >> test-expected-output +cmp testoutput test-expected-output + +exit 104