BZ #174: make check fails with "gc_out_of_memory: out of memory"

Status fields:

make check:

make[3]: Entering directory
`/localtmp/zapster/cacao/builds/cacao/classpath/x86_64/tests/regression/base'
javac -source 1.5 -target 1.5 -nowarn -bootclasspath ../../../src/classes/classes:/local
tmp/zapster/cacao/install/classpath/x86_64//share/classpath/glibj.zip -classpath
/usr/share/java/junit4.jar -d . /
localtmp/zapster/cacao/repos/cacao.hg/tests/regression/base/*.java
LD_LIBRARY_PATH=../../../src/cacao/.libs ../../../src/cacao/cacao -Xbootclasspath:../../
../src/classes/classes:/localtmp/zapster/cacao/install/classpath/x86_64//share/classpath
/glibj.zip -classpath /usr
/share/java/junit4.jar:. org.junit.runner.JUnitCore All
LOG: [0x00002b15c1da5200] gc_out_of_memory: out of memory
LOG: [0x00002b15c1da5200] Backtrace (8 stack frames):
LOG: [0x00002b15c1da5200]
../../../src/cacao/.libs/libjvm.so(_ZN2os15print_backtraceEv+0x24) [0x2b15c1e2f094]
LOG: [0x00002b15c1da5200] ../../../src/cacao/.libs/libjvm.so(_ZN2os5abortEPKcz+0xa3)
[0x2b15c1e2f3f3]
LOG: [0x00002b15c1da5200] ../../../src/cacao/.libs/libjvm.so(gc_out_of_memory+0x1b)
[0x2b15c1df4cfb]
LOG: [0x00002b15c1da5200] ../../../src/cacao/.libs/libjvm.so(GC_core_malloc_atomic+0xe8)
[0x2b15c1dfcf58]
LOG: [0x00002b15c1da5200] ../../../src/cacao/.libs/libjvm.so(heap_alloc+0x75)
[0x2b15c1df4da5]
LOG: [0x00002b15c1da5200] ../../../src/cacao/.libs/libjvm.so(stacktrace_get+0xb4)
[0x2b15c1e56924]
LOG: [0x00002b15c1da5200]
../../../src/cacao/.libs/libjvm.so(Java_java_lang_VMThrowable_fillInStackTrace+0x1d)
[0x2b15c1e1428d]
LOG: [0x00002b15c1da5200] [0x2b15c33742b1]
make[3]: *** [run] Aborted
make[3]: Leaving directory
`/localtmp/zapster/cacao/builds/cacao/classpath/x86_64/tests/regression/base'
make[2]: *** [check-recursive] Error 1
make[2]: Leaving directory
`/localtmp/zapster/cacao/builds/cacao/classpath/x86_64/tests/regression'
make[1]: *** [check-recursive] Error 1
make[1]: Leaving directory `/localtmp/zapster/cacao/builds/cacao/classpath/x86_64/tests'
make: *** [check-recursive] Error 1


classpath was compiled with --enable-debug:

/localtmp/zapster/cacao/repos/classpath.git/configure
--prefix=/localtmp/zapster/cacao/install/classpath/x86_64 --disable-plugin --disable-
Werror --enable-debug

cacao was configured as follows:

/localtmp/zapster/cacao/repos/cacao.hg/configure --with-java-runtime-library-
prefix=/localtmp/zapster/cacao/install/classpath/x86_64
--prefix=/localtmp/zapster/cacao/install/cacao/classpath/x86_64


The problem can be avoided by omitting the --enable-debug flag from the classpath
configuration.

This issues can be reproduced on Ubuntu 12.10 and Debian squeeze but not on CentOS 6
(all x86_64).

This seems to be a problem with initialization order again. With DEBUG=true, it tries to
print something from src/classes/gnuclasspath/java/security/VMAccessController.java, and
that's where trouble starts unfolding.

It's quite challenging to even obtain a stack trace, but here it is:

        at
java.security.VMAccessController.debug(Ljava/lang/String;)V(VMAccessController.java:82)
        at java.security.VMAccessController.pushContext(Ljava/security/AccessControlCont
ext;)V(VMAccessController.java:107)
        at java.security.AccessController.doPrivileged(Ljava/security/PrivilegedAction;)
Ljava/lang/Object;(AccessController.java:93)
        at gnu.java.nio.charset.Provider.provider()Lgnu/java/nio/charset/Provider;(Provi
der.java:261)
        at java.nio.charset.Charset.provider()Ljava/nio/charset/spi/CharsetProvider;(Cha
rset.java:253)
        at java.nio.charset.Charset.charsetForName(Ljava/lang/String;)Ljava/nio/charset/
Charset;(Charset.java:208)
        at java.nio.charset.Charset.forName(Ljava/lang/String;)Ljava/nio/charset/Charset
;(Charset.java:188)
        at java.lang.String.stringToCharset(Ljava/lang/String;)Ljava/nio/charset/Charset
;(String.java:358)
        at java.lang.String.getBytes(Ljava/lang/String;)[B(String.java:798)
        at java.io.PrintStream.writeChars(Ljava/lang/String;II)V(PrintStream.java:354)
        at java.io.PrintStream.print(Ljava/lang/String;Z)V(PrintStream.java:307)
        at java.io.PrintStream.print(Ljava/lang/String;)V(PrintStream.java:434)
        at
java.security.VMAccessController.debug(Ljava/lang/String;)V(VMAccessController.java:82)
        at java.security.VMAccessController.pushContext(Ljava/security/AccessControlCont
ext;)V(VMAccessController.java:107)

... and so on.

Interestingly, I get the oom only when the test is run from make. If I try to run it
directly from the shell, it crashes right away, as strange as that sounds.

regarding different behaviour between shell and `make', this might helps:
http://hackage.haskell.org/trac/ghc/ticket/6059#comment:3

(read the full ticket entry for more information)

Regarding the different behavior between shell and make:

Since the available stack is completely used up, and CACAO doesn’t have good stack-
overflow handling, the minor difference in behavior is not completely unexpected – just
minor variations in memory layout due to additional environment variables or anything
else could easily cause this.

That doesn't look so much like an initialization problem, rather like a genuine endless
recursion. I think I should give it a try with JamVM.

JamVM is cheating: what does a comparison of its VMAccessController.java against GNU
classpath's reveal?

  -  private static final boolean DEBUG = gnu.classpath.Configuration.DEBUG;
  +  private static final boolean DEBUG = false;

Let's change that back, and sure enough, running a simple hello world produces:

  Exception occurred while printing exception (java/lang/StackOverflowError)...
  Original exception was java/lang/StackOverflowError

This should be reported against GNU classpath.