We test Soot nightly, by building and running a few tools that use Soot internally:

To run the tests yourself, e.g., to test whether a Patch works, please proceed as follows:

Check out and build Clara and all its dependencies. This is fairly automated, for instructions see here: http://www.bodden.de/clara/downloads/

Then execute the ant script in the directory "build":

ant test-and-email will run all tests and email you the results

If tests fail, you will see the output in abc-testing/ajc-harness/tests/ failed.output. There are some tests that fail regularly. You can find the current listing here:

Running test 53: new/handle multiple nested inner classes Commandline: abc -d new -warn-unused-advice:off new/InnerHell?.java Compilation produced an unexpected number of errors: 1, should be 0 Actual errors found: Semantic Error at new/InnerHell?.java:12,4-13:5: No valid constructor found for javax.swing.JComponent.AccessibleJComponent(). FAIL: Test 53: "new/handle multiple nested inner classes" failed in 612ms, memory usage: 17787496.

Running test 115: new/Not binding constructor when using more than one compilation Commandline: abc -d new -warn-unused-advice:off new/OuterAbstract?_PR408.java Commandline: abc -cp new:../../../clara/lib/clara-complete.jar:../../../abc/lib/abc-testing.jar -d new -warn-unused-advice:off new/ExtendsOuterAbstract?_PR408.java Compilation produced an unexpected number of errors: 1, should be 0 Actual errors found: Semantic Error at new/ExtendsOuterAbstract?_PR408.java:4,8-21: Could not find type "InnerAbstract?". FAIL: Test 115: "new/Not binding constructor when using more than one compilation" failed in 1018ms, memory usage: 17848384.

Running test 165: pureJava/nameConflicts/type and package name conflicts caught as errors (1) Commandline: abc -d pureJava/nameConflicts -warn-unused-advice:off pureJava/nameConflicts/p1/p2.java pureJava/nameConflicts/p1/Main1.java pureJava/nameConflicts/p1/Foo.java pureJava/nameConflicts/p1/p2/Foo.java Compilation succeeded but was expected to fail. FAIL: Test 165: "pureJava/nameConflicts/type and package name conflicts caught as errors (1)" failed in 669ms, memory usage: 20986784.

Running test 178: pureJava/nested finally blocks have interesting frame location problems Commandline: abc -d pureJava -warn-unused-advice:off pureJava/NestedFinally?.java Compilation produced an unexpected number of errors: 1, should be 0 Actual errors found: Semantic Error at pureJava/NestedFinally?.java:5,18-35: Missing return statement. FAIL: Test 178: "pureJava/nested finally blocks have interesting frame location problems" failed in 548ms, memory usage: 8225592.

Running test 190: pureJava/interfaces with non-explicitly static inner classes Commandline: abc -d pureJava -warn-unused-advice:off pureJava/InterfaceAndInnerHelper?.java Commandline: abc -cp pureJava:../../../clara/lib/clara-complete.jar:../../../abc/lib/abc-testing.jar -d pureJava -warn-unused-advice:off pureJava/InterfaceAndInner?.java Compilation produced an unexpected number of errors: 2, should be 0 Actual errors found: Semantic Error at pureJava/InterfaceAndInner?.java:7,23-48: Class "C" not found in scope of InterfaceAndInnerHelper? Semantic Error at pureJava/InterfaceAndInner?.java:8,34-59: Class "C" not found in scope of InterfaceAndInnerHelper? FAIL: Test 190: "pureJava/interfaces with non-explicitly static inner classes" failed in 979ms, memory usage: 15711080.

Running test 440: new/around advice on calls within inner classes (including protected method calls) 1 Commandline: abc -d new -warn-unused-advice:off new/AroundInnerCalls13.java File /mnt/sdb1/var/lib/jenkins/jobs/ABC/workspace/abc-testing/ajc-harness/tests/new/[Ljava/lang/String;.class not found, skipping... Outer.foo() calls Outer.Inner.mi() Outer@31c7ac55.foo calls Outer$Inner@2c0319fb.mi Outer.Inner.mi() calls PrintStream?.println(..) Outer$Inner@2c0319fb.mi calls java.io.PrintStream?@7f9ec44f.println . Outer.foo() calls Outer.InnerRandom?.nextHook(..) Outer@31c7ac55.foo calls Outer$InnerRandom?@20111326.nextHook Outer.InnerRandom?.nextHook(..) calls Outer.InnerRandom?.next(..) Outer$InnerRandom?@20111326.nextHook calls Outer$InnerRandom?@20111326.next Outer.foo() calls Outer.1.nextInt(..) Outer@31c7ac55.foo calls X.nextInt Test failed, compiled class behaved incorrectly Message: fail expected note "Outer.InnerRandom?.nextHook(..) calls Random.next(..)" org.aspectj.bridge.AbortException?: expected note "Outer.InnerRandom?.nextHook(..) calls Random.next(..)"

at org.aspectj.bridge.MessageWriter?.handleMessage(MessageWriter?.java:47) at org.aspectj.testing.Tester.handle(Tester.java:963) at org.aspectj.testing.Tester.checkFailed(Tester.java:177) at org.aspectj.testing.Tester.check(Tester.java:204) at org.aspectj.testing.Tester.check(Tester.java:145) at org.aspectj.testing.Tester.check(Tester.java:133) at AroundInnerCalls13.shadow$7(AroundInnerCalls13.java:7) at A.inline$1$around$942(AroundInnerCalls13.java:41) at AroundInnerCalls13.main(AroundInnerCalls13.java:5) at sun.reflect.NativeMethodAccessorImpl?.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl?.invoke(NativeMethodAccessorImpl?.java:57) at sun.reflect.DelegatingMethodAccessorImpl?.invoke(DelegatingMethodAccessorImpl?.java:43) at java.lang.reflect.Method.invoke(Method.java:616) at abc.testing.TestCase?.runTest(TestCase?.java:413) at abc.testing.Main.doCase(Main.java:288) at abc.testing.Main.main(Main.java:122)

FAIL: Test 440: "new/around advice on calls within inner classes (including protected method calls) 1" failed in 2182ms, memory usage: 15129632.

Running test 496: new/elaborated into testing of around on all join points Commandline: abc -d new -warn-unused-advice:off new/AroundAll?.java File /mnt/sdb1/var/lib/jenkins/jobs/ABC/workspace/abc-testing/ajc-harness/tests/new/[Ljava/lang/String;.class not found, skipping... Test failed, compiled class behaved incorrectly Message: fail steps expected array of length 114 got 132 org.aspectj.bridge.AbortException?: steps expected array of length 114 got 132

at org.aspectj.bridge.MessageWriter?.handleMessage(MessageWriter?.java:47) at org.aspectj.testing.Tester.handle(Tester.java:963) at org.aspectj.testing.Tester.checkFailed(Tester.java:177) at org.aspectj.testing.Tester.checkEqual(Tester.java:228) at A.checkLog(AroundAll?.java:176) at AroundAll?.shadow$8(AroundAll?.java:9) at A.inline$21$around$1600(AroundAll?.java:199) at AroundAll?.main(AroundAll?.java:198) at sun.reflect.NativeMethodAccessorImpl?.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl?.invoke(NativeMethodAccessorImpl?.java:57) at sun.reflect.DelegatingMethodAccessorImpl?.invoke(DelegatingMethodAccessorImpl?.java:43) at java.lang.reflect.Method.invoke(Method.java:616) at abc.testing.TestCase?.runTest(TestCase?.java:413) at abc.testing.Main.doCase(Main.java:288) at abc.testing.Main.main(Main.java:122)

FAIL: Test 496: "new/elaborated into testing of around on all join points" failed in 2510ms, memory usage: 14537456.

Running test 555: new/accessing protected superclass members in and outside CCC from body of method introduction Commandline: abc -d new -warn-unused-advice:off new/SuperInIntroduction?.java InvocationTargetException? while trying to run compiled class: java.lang.VerifyError?: (class: A, method: getClone signature: (LObjectSub;)LObjectSub;) Bad access to protected data java.lang.reflect.InvocationTargetException?

at sun.reflect.NativeMethodAccessorImpl?.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl?.invoke(NativeMethodAccessorImpl?.java:57) at sun.reflect.DelegatingMethodAccessorImpl?.invoke(DelegatingMethodAccessorImpl?.java:43) at java.lang.reflect.Method.invoke(Method.java:616) at abc.testing.TestCase?.runTest(TestCase?.java:413) at abc.testing.Main.doCase(Main.java:288) at abc.testing.Main.main(Main.java:122)

Caused by: java.lang.VerifyError?: (class: A, method: getClone signature: (LObjectSub;)LObjectSub;) Bad access to protected data

at Sub.getInt(SuperInIntroduction?.java) at SuperInIntroduction?.main(SuperInIntroduction?.java:6) ... 7 more

FAIL: Test 555: "new/accessing protected superclass members in and outside CCC from body of method introduction" failed in 1785ms, memory usage: 12630656.

Running test 64301: abctests/new/type pattern matching for inner classes (from Ken Horn) Commandline: abc -d abctests/new -warn-unused-advice:off abctests/new/test/TypePat?.java Test failed, compiled class behaved incorrectly Message: fail expected event "A.before2: C.foo()" not found? org.aspectj.bridge.AbortException?: expected event "A.before2: C.foo()" not found?

at org.aspectj.bridge.MessageWriter?.handleMessage(MessageWriter?.java:47) at org.aspectj.testing.Tester.handle(Tester.java:963) at org.aspectj.testing.Tester.checkFailed(Tester.java:177) at org.aspectj.testing.Tester.check(Tester.java:204) at org.aspectj.testing.Tester.checkEqualIgnoreDups(Tester.java:702) at org.aspectj.testing.Tester.checkEqual(Tester.java:742) at org.aspectj.testing.Tester.checkEvents(Tester.java:922) at org.aspectj.testing.Tester.checkAndClearEvents(Tester.java:933) at test.TypePat?.main(TypePat?.java:42) at sun.reflect.NativeMethodAccessorImpl?.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl?.invoke(NativeMethodAccessorImpl?.java:57) at sun.reflect.DelegatingMethodAccessorImpl?.invoke(DelegatingMethodAccessorImpl?.java:43) at java.lang.reflect.Method.invoke(Method.java:616) at abc.testing.TestCase?.runTest(TestCase?.java:413) at abc.testing.Main.doCase(Main.java:288) at abc.testing.Main.main(Main.java:122)

FAIL: Test 64301: "abctests/new/type pattern matching for inner classes (from Ken Horn)" failed in 2028ms, memory usage: 14051440.

Running test 740: new/PR862/import any inner from interface implementor Commandline: abc -d new/PR862 -warn-unused-advice:off new/PR862/pack/ImportInnerFromInterfaceImplementor?.java File /mnt/sdb1/var/lib/jenkins/jobs/ABC/workspace/abc-testing/ajc-harness/tests/new/PR862/pack/ImportInnerFromInterfaceImplementor?.class not found, skipping... Failed to find class pack.ImportInnerFromInterfaceImplementor? FAIL: Test 740: "new/PR862/import any inner from interface implementor" failed in 615ms, memory usage: 19799592.

Running test 752: bugs/problems with finalize call Commandline: abc -d bugs -warn-unused-advice:off bugs/Finalizer.java Compilation succeeded but was expected to fail. FAIL: Test 752: "bugs/problems with finalize call" failed in 652ms, memory usage: 18861512.

Running test 769: new/illegal name binding in around cflow Commandline: abc -d new -warn-unused-advice:off new/ArgsInCflowCf?.java Compilation succeeded but was expected to fail. FAIL: Test 769: "new/illegal name binding in around cflow" failed in 769ms, memory usage: 14558080.

Running test 77201: abctests/bugs/declare soft and throw statements Commandline: abc -d abctests/bugs -warn-unused-advice:off abctests/bugs/ConvertToUnchecked?.java Compilation produced an unexpected number of errors: 1, should be 0 Actual errors found: Semantic Error at abctests/bugs/ConvertToUnchecked?.java:44,2-11: The exception java.io.IOException must be either caught or declared to be thrown FAIL: Test 77201: "abctests/bugs/declare soft and throw statements" failed in 704ms, memory usage: 21795528.

Running test 792: new/arrays via Class.forName() Commandline: abc -d new -warn-unused-advice:off new/ClassForName?.java File /mnt/sdb1/var/lib/jenkins/jobs/ABC/workspace/abc-testing/ajc-harness/tests/new/[Ljava/lang/String;.class not found, skipping... execution(void ClassForName?.main(ClassNotFoundException?)) File /mnt/sdb1/var/lib/jenkins/jobs/ABC/workspace/abc-testing/ajc-harness/tests/new/[Ljava/lang/String;.class not found, skipping... InvocationTargetException? while trying to run compiled class: java.lang.ClassNotFoundException?: [Ljava.lang.String; java.lang.reflect.InvocationTargetException?

at sun.reflect.NativeMethodAccessorImpl?.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl?.invoke(NativeMethodAccessorImpl?.java:57) at sun.reflect.DelegatingMethodAccessorImpl?.invoke(DelegatingMethodAccessorImpl?.java:43) at java.lang.reflect.Method.invoke(Method.java:616) at abc.testing.TestCase?.runTest(TestCase?.java:413) at abc.testing.Main.doCase(Main.java:288) at abc.testing.Main.main(Main.java:122)

Caused by: java.lang.ClassNotFoundException?: [Ljava.lang.String;

at java.lang.ClassLoader?.findClass(ClassLoader?.java:373) at abc.testing.TestCase?$FileClassLoader?.findClass(TestCase?.java:794) at java.lang.ClassLoader?.loadClass(ClassLoader?.java:321) at java.lang.ClassLoader?.loadClass(ClassLoader?.java:266) at ClassForName?.main(ClassForName?.java:9) ... 7 more

FAIL: Test 792: "new/arrays via Class.forName()" failed in 1156ms, memory usage: 14452424.

Running test 812: harness/CLE: no dir specified for sourceroots Commandline: abc -d harness -warn-unused-advice:off -sourceroots harness/ErrorTest?.java Unexpected exception while compiling: polyglot.util.InternalCompilerError?: unhandled exception during compilation polyglot.util.InternalCompilerError?: unhandled exception during compilation

at abc.main.Main.run(Main.java:454) at abc.testing.TestCase?.runTest(TestCase?.java:289) at abc.testing.Main.doCase(Main.java:288) at abc.testing.Main.main(Main.java:122)

Caused by: java.lang.IllegalArgumentException?: Sourceroot harness/ErrorTest?.java is not a directory

at abc.main.Main.findSourcesInDir(Main.java:587) at abc.main.Main.loadSourceRoots(Main.java:604) at abc.main.Main.run(Main.java:398) ... 3 more

FAIL: Test 812: "harness/CLE: no dir specified for sourceroots" failed in 655ms, memory usage: 31243840.

Running test 951: bugs/deprecated/-deprecation not working? Commandline: abc -d bugs/deprecated -warn-unused-advice:off -deprecation bugs/deprecated/Deprecation.java bugs/deprecated/Deprecated.java Illegal arguments: Unknown option -deprecation FAIL: Test 951: "bugs/deprecated/-deprecation not working?" failed in 208ms, memory usage: 12396600.

Running test 986: abctests/bugs/Matching int.clone()" failed in 574ms, memory usage: 18563536.

Running test 992: abctests/bugs/cast pointcut can interact badly with jimple type assignment Commandline: abc -d abctests/bugs -injars abctests/bugs/cast.jar -warn-unused-advice:off -ext abc.eaj abctests/bugs/CastAspect?.java Test failed, compiled class behaved incorrectly Message: fail unexpected event "cast happened" found? org.aspectj.bridge.AbortException?: unexpected event "cast happened" found?

at org.aspectj.bridge.MessageWriter?.handleMessage(MessageWriter?.java:47) at org.aspectj.testing.Tester.handle(Tester.java:963) at org.aspectj.testing.Tester.checkFailed(Tester.java:177) at org.aspectj.testing.Tester.check(Tester.java:204) at org.aspectj.testing.Tester.checkEqualIgnoreDups(Tester.java:702) at org.aspectj.testing.Tester.checkEqual(Tester.java:742) at org.aspectj.testing.Tester.checkEvents(Tester.java:922) at org.aspectj.testing.Tester.checkAndClearEvents(Tester.java:933) at org.aspectj.testing.Tester.checkAllEvents(Tester.java:846) at Cast.main(Cast.java:5) at sun.reflect.NativeMethodAccessorImpl?.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl?.invoke(NativeMethodAccessorImpl?.java:57) at sun.reflect.DelegatingMethodAccessorImpl?.invoke(DelegatingMethodAccessorImpl?.java:43) at java.lang.reflect.Method.invoke(Method.java:616) at abc.testing.TestCase?.runTest(TestCase?.java:413) at abc.testing.Main.doCase(Main.java:288) at abc.testing.Main.main(Main.java:122)

FAIL: Test 992: "abctests/bugs/cast pointcut can interact badly with jimple type assignment" failed in 2435ms, memory usage: 13731280.

Running test 1001: abctests/bugs/resolving anonymous classes inside classes with dollar signs Commandline: abc -d abctests/bugs -warn-unused-advice:off abctests/bugs/ExtraAnon?.java Unexpected exception while compiling: polyglot.util.InternalCompilerError?: unhandled exception during compilation polyglot.util.InternalCompilerError?: unhandled exception during compilation

at abc.main.CompileSequence?.runSequence(CompileSequence?.java:110) at abc.main.Main.run(Main.java:406) at abc.testing.TestCase?.runTest(TestCase?.java:289) at abc.testing.Main.doCase(Main.java:288) at abc.testing.Main.main(Main.java:122)

Caused by: java.lang.RuntimeException?: couldn't find class: E$A2$1 (is your soot-class-path set properly?)

at soot.SootResolver?.bringToHierarchy(SootResolver?.java:205) at soot.SootResolver?.bringToSignatures(SootResolver?.java:239) at soot.SootResolver?.processResolveWorklist(SootResolver?.java:154) at soot.SootResolver?.resolveClass(SootResolver?.java:124) at soot.Scene.loadClass(Scene.java:448) at abc.main.CompileSequence?.compile(CompileSequence?.java:338) at abc.main.CompileSequence?.runSequence(CompileSequence?.java:100) ... 4 more

FAIL: Test 1001: "abctests/bugs/resolving anonymous classes inside classes with dollar signs" failed in 695ms, memory usage: 35964696.

Running test 1032: abctests/tracematches/Tracematches: Testing self-applying around TM-advice Commandline: abc -d abctests/tracematches -warn-unused-advice:off -ext abc.tm abctests/tracematches/AdviseSelf?.java Compilation produced unexpected warnings:

Warning at Compiler Generated: Invalid Jimple body for <AdviseSelfTM: void tracematch$3119$body_real(AdviseSelf?,java.lang.Integer,Abc$proceed$AdviseSelfTM$tracematch$3119$body,int,int,int,java.lang.Object,java.lang.Object,java.lang.Object[],int)>
All identity statements should occur at the start of a Jimple method. abc

will try to fix this, but really it's a bug in code generation. FAIL: Test 1032: "abctests/tracematches/Tracematches: Testing self-applying around TM-advice" failed in 1122ms, memory usage: 24447656.

Running test 1208: abctests/bugs/inpath/Class being woven twice when using -inpath Commandline: abc -d abctests/bugs/inpath -warn-unused-advice:off File /mnt/sdb1/var/lib/jenkins/jobs/ABC/workspace/abc-testing/ajc-harness/tests/abctests/bugs/inpath/Test.class not found, skipping... Failed to find class Test FAIL: Test 1208: "abctests/bugs/inpath/Class being woven twice when using -inpath" failed in 522ms, memory usage: 14993544.