Bug #1761
Audio file reader tests still not quite right
| Status: | New | Start date: | 2016-12-02 | |
|---|---|---|---|---|
| Priority: | Normal | Due date: | ||
| Assignee: | - | % Done: | 0% | |
| Category: | - | |||
| Target version: | - |
Description
The svcore audio file reader tests have logic like this:
- take a test signal
- encode it using the various (mostly lossy) encoders and store the test file in the repo
- for the test itself, decode the test file and compare with the original test signal
The problem is that the test signal is very artificial and perceptually motivated lossy encoders don't necessarily produce a signal that is all that similar to it. So our naive comparisons are too loose and therefore a bit rubbish.
A better (and simpler to do) approach would be
- take a test signal
- encode it using the various (mostly lossy) encoders and store the test file in the repo
- decode each of those test files using a trusted decoder application, check the decoded audio "manually" to make sure the signal looks and sounds sensible, has low noise, no offset etc
- store the decoded versions in the repo
- for the test itself, decode the test file and compare with the previously validated decoded version
Now we expect the two to be very similar and so, hopefully, can compare with confidence.
Since we already have the code to do it the other way, we could potentially do both, though I'm not sure what is really gained apart from generating the diff files which are quite interesting.