gnupic: Thread: gputils regression tests

I think I discovered some serious problems in the gputils regression test system, and I'm not quite sure how to proceed.

The first problem is that I believe the test system will never report a failure on a set of tests.  The conditional to test for a failure is something like this:
 if ! test $tested=$passed then;
   return 1
 fi
Maybe some bash guru can fill me in on what "test $a=$b" actually tests, but apparently "test 1=2" returns true.  For that problem, I can easily change the test to:
 if (($tested==$passed)) then;

However, I also found that several of the gpasm.mchip tests fail every time, even if you backtrack to the gputils 13.4 release version.

In fact, I don't think they've ever passed!  I looked through one of them, gpasm/testsuite/gpasm.mchip/pseudo14.asm, and found that it has labels in column 1 and and an implicit assembler variable declaration.  The former is an error in gpasm but only a warning in MPASM, and on that topic Craig has said, "gpasm may be modified in the future to allow this, but for now only the documented behavior of mpasm will be implemented."  However, the undeclared variable causes a failure even under the latest MPLAB 7.60.  I'm not sure that the same is true for all of the failed tests, but it probably is for a lot of them.  Still, if they never worked under MPASM, I'm not sure how they got the baseline HEX files to compare against.

So, the long and short of it is that if I fix the reporting of failures, the gpasm.mchip tests will fail and then the other tests won't run.  As it is, the test system does print out the figures for how many passed and failed and whatnot after each run, but the failures are buried in a ton of other output.  I'm not sure what to do with those tests that always fail, especially the ones that fail even under MPASM.

Any ideas?

David Barnett