BZ #139: CACAO+OpenJDK7 b89 fails to bootstrap

Status fields:

creation_ts:2010-05-11 15:08
component:unspecified
version:0.99.4
rep_platform:All
op_sys:Linux
bug_status:RESOLVED
resolution:FIXED
reporter:gnu_andrew@member.fsf.org
mkdir -p /mnt/builder/cacao-icedtea7/pulseaudio/classes
(cd /home/andrew/projects/openjdk/icedtea7/pulseaudio/src/java; \
         /mnt/builder/cacao-icedtea7/bootstrap/jdk1.6.0/bin/javac -g -encoding utf-8
-source 5 -target 5 -d /mnt/builder/cacao-icedtea7/pulseaudio/classes\
         -bootclasspath '/mnt/builder/cacao-icedtea7/bootstrap/jdk1.6.0/jre/lib/rt.jar'
\
         org/classpath/icedtea/pulseaudio/*.java\
        )
Exception in thread "null" java.lang.UnsatisfiedLinkError: registerNatives
        at sun.misc.Perf.<clinit>(Perf.java:536)
        at sun.misc.Perf$GetPerfAction.run(Perf.java:97)
        at sun.misc.Perf$GetPerfAction.run(Perf.java:1)
        at java.security.AccessController.doPrivileged(Native Method)
        at sun.misc.PerfCounter.<clinit>(PerfCounter.java:52)
        at java.util.zip.ZipFile.<init>(ZipFile.java:213)
        at java.util.zip.ZipFile.<init>(ZipFile.java:142)
        at java.util.jar.JarFile.<init>(JarFile.java:150)
        at java.util.jar.JarFile.<init>(JarFile.java:87)
        at sun.misc.URLClassPath$JarLoader.getJarFile(URLClassPath.java:694)
        at sun.misc.URLClassPath$JarLoader.access$1(URLClassPath.java:687)
        at sun.misc.URLClassPath$JarLoader$1.run(URLClassPath.java:655)
        at sun.misc.URLClassPath$JarLoader$1.run(URLClassPath.java:1)
        at java.security.AccessController.doPrivileged(Native Method)
        at sun.misc.URLClassPath$JarLoader.ensureOpen(URLClassPath.java:647)
        at sun.misc.URLClassPath$JarLoader.<init>(URLClassPath.java:620)
        at sun.misc.URLClassPath$3.run(URLClassPath.java:362)
        at sun.misc.URLClassPath$3.run(URLClassPath.java:1)
        at java.security.AccessController.doPrivileged(Native Method)
        at sun.misc.URLClassPath.getLoader(URLClassPath.java:351)
        at sun.misc.URLClassPath.getLoader(URLClassPath.java:328)
        at sun.misc.URLClassPath.getResource(URLClassPath.java:194)
        at java.net.URLClassLoader$2.run(URLClassLoader.java:291)
        at java.net.URLClassLoader$2.run(URLClassLoader.java:1)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:287)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:422)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:325)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:355)
        at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:195)

Comment #1 by stefan@complang.tuwien.ac.at on 2010-05-18 22:36:59

The reason is that performance counters were added to various basic classes in OpenJDK
in this changeset: http://hg.openjdk.java.net/jdk7/jdk7/jdk/rev/c7e469ae3edb

http://mips.complang.tuwien.ac.at/hg/cacao/rev/fbda1e01b241 contains a stub
implementation, but the cacao-1.0.x branch from which the releases are built does not
contain it. However, this changeset alone is not enough, as a NPE would occur just a few
lines later.

Comment #2 by stefan@complang.tuwien.ac.at on 2010-05-19 15:50:10

http://mips.complang.tuwien.ac.at/hg/cacao/rev/3a39f2b5f3e3

The change above is all that's needed; however, there is currently no easy way to
bootstrap icedtea with a CACAO 1.1 build.

Comment #3 by gnu_andrew@member.fsf.org on 2010-05-20 14:20:31

If you can suggest a suitable changeset ID or tag in cacao's hg we could use, it would
be trivial to update the CACAO URL and SHA256sum to that instead.

Comment #4 by stefan@complang.tuwien.ac.at on 2010-05-20 14:31:13

It's not that easy, because you would need to do ./autogen.sh, and some of the patches
would still need to be applied, but would have to be regenerated.

We still need:
6714758
jsig
no-strict-aliasing
version (not sure about this one)

Also, it would then be built with -O0.

I will try to sort this out in time, but I'm not there yet. But I definitely think that
icedtea should start using the C++ CACAO, at least icedtea7.

Comment #5 by gnu_andrew@member.fsf.org on 2010-05-20 15:11:53

Give me an appropriate changeset/tag and I can switch to 1.1.

Comment #6 by stefan@complang.tuwien.ac.at on 2010-05-20 15:14:22

http://mips.complang.tuwien.ac.at/hg/cacao/rev/8948a434c10d

Comment #7 by gnu_andrew@member.fsf.org on 2010-06-21 17:53:01

I've started trying to build with that snapshot.  So far I've gone through the following
patches.

patches/cacao/no-mmap-first-page.patch: Rather than a removal as in the patch, this
seems to depend on an option.  What's the state of that option by default?
patches/cacao/native-resolve.patch: I'm assuming this is in, as the original patch was
by you.  It's hard to tell as the code has changed so much.
patches/cacao/xxoption-warning.patch: This is in.
patches/cacao/no-strict-aliasing.patch: I've disabled this to see if we still need it or
not.  It was a result of the build through strict-aliasing warnings and then the
resulting VM crashing immediately (visible on normal Classpath builds too)
patches/cacao/openjdk7.patch: I'm not sure how this has been upstreamed.  The methods
have been renamed with a _jdk6 or _jdk7 suffix.  It's not clear how one is selected to
be used.  Is there some option?

I'll try and look at the rest next week.

Comment #8 by stefan@complang.tuwien.ac.at on 2010-06-21 18:08:06

I've already determined the list of patches which are still relevant in comment #4,
unfortunately you seem to have missed it.

* no-mmap-first-page: It's off by default, i.e. it should work out-of-the-box.
* native-resolve: True, this is in.
* openjdk7.patch: Not needed anymore, both versions are enabled; can be linked against
both jdk6 or jdk7.

Comment #9 by gnu_andrew@member.fsf.org on 2010-06-21 19:30:41

No, I saw your post but wanted to check them for myself to be sure.

I'm still not clear on openjdk7.patch.  How does the appropriate version get used?

Comment #10 by stefan@complang.tuwien.ac.at on 2010-06-21 20:13:27

This is the changeset: http://mips.complang.tuwien.ac.at/hg/cacao/rev/ce8ab67c7b5d

The methods are just looked up by signature, and since these are different for jdk6 and
jdk7, we can just put in both, without conflict.

Comment #11 by gnu_andrew@member.fsf.org on 2010-06-21 22:41:11

I've pushed the necessary changes to Mercurial.  arm-arch-defines is still
needed, no-strict-aliasing was dropped as I no longer see aliasing errors and
cacao -version worked fine.

The build currently fails with an out-of-memory error in the langtools build
using CACAO+OpenJDK as the build VM (i.e. the second stage).  Do you not see
this?  Any ideas?

    [javac] Compiling 685 source files to
/mnt/builder/cacao-icedtea7/openjdk.build/langtools/build/classes
    [javac]
    [javac]
    [javac] The system is out of resources.
    [javac] Consult the following stack trace for details.
    [javac] java.lang.OutOfMemoryError
    [javac]     at
com.sun.tools.javac.util.SharedNameTable.fromChars(SharedNameTable.java:99)

Comment #12 by gnu_andrew@member.fsf.org on 2010-06-21 22:44:22

Aliasing bug is http://server.complang.tuwien.ac.at/cgi-
bin/bugzilla/show_bug.cgi?id=129; you may be able to close that.  At least, it works
here with GCC 4.5.

I know the methods are looked up by signature, but none now have the signature to match
the method in sun.misc.Unsafe.  So how does your fix work?

Comment #13 by stefan@complang.tuwien.ac.at on 2010-06-22 01:00:15

(In reply to comment #11)
> I've pushed the necessary changes to Mercurial.  arm-arch-defines is still
> needed, no-strict-aliasing was dropped as I no longer see aliasing errors and
> cacao -version worked fine.

That’s because you’re building with -O0. We should change that (the 0.99.4 release
tarball had -O2 in it).

> The build currently fails with an out-of-memory error in the langtools build
> using CACAO+OpenJDK as the build VM (i.e. the second stage).  Do you not see
> this?  Any ideas?

Never seen it when running my builds last week. I’ve just started a build with your
version, but I would be very surprised if this made a difference.

> I know the methods are looked up by signature, but none now have the signature
> to match the method in sun.misc.Unsafe.  So how does your fix work?

Only the Java signatures matter, and they do match. It wouldn’t work otherwise.

Comment #14 by stefan@complang.tuwien.ac.at on 2010-06-22 01:14:07

(In reply to comment #13)
> > The build currently fails with an out-of-memory error in the langtools build
> > using CACAO+OpenJDK as the build VM (i.e. the second stage).  Do you not see
> > this?  Any ideas?
>
> Never seen it when running my builds last week. I’ve just started a build
> with your version, but I would be very surprised if this made a difference.

Interestingly, it did in fact fail for me in the exact same way.

Comment #15 by stefan@complang.tuwien.ac.at on 2010-06-22 01:27:41

Created an attachment (id=70)
Added patch for -O2

This would be good to apply.

Comment #16 by stefan@complang.tuwien.ac.at on 2010-06-22 11:19:15

(In reply to comment #14)
> (In reply to comment #13)
> > > The build currently fails with an out-of-memory error in the langtools build
> > > using CACAO+OpenJDK as the build VM (i.e. the second stage).  Do you not see
> > > this?  Any ideas?
> >
> > Never seen it when running my builds last week. I’ve just started a build
> > with your version, but I would be very surprised if this made a difference.
>
> Interestingly, it did in fact fail for me in the exact same way.
>

The really curious thing it that it just works when I rerun 'make'.

Comment #17 by gnu_andrew@member.fsf.org on 2010-06-22 18:17:35

I've added a patch this morning to use 256mb instead of the default 128mb when building
langtools with cacao.  That fixes the issue.  It will work when you run make again as
it'll have built some of the classes.  It's the load of all of them that runs out of
memory.

As to the -O2 fix, can you not just do this in CACAO and then we'll grab a later
snapshot? I'd prefer to avoid adding to the stack of patches where possible.

Comment #18 by stefan@complang.tuwien.ac.at on 2010-06-23 10:25:35

Ok, I will check this in and immediately back it out, "blessing" the changeset for
icedtea usage.

Comment #19 by gnu_andrew@member.fsf.org on 2010-06-23 10:33:21

Why would you need to back it out?  Wasn't wrong with this change for CACAO generally?

Comment #20 by stefan@complang.tuwien.ac.at on 2010-06-23 17:28:28

Ok, you can use http://mips.complang.tuwien.ac.at/hg/cacao/rev/dd4532afd08a

I don't want it in the development repo because when you debug it you want to build with
-O0, usually. It's been this way forever, and I like it with -O0. Although things should
improve a lot with gcc 4.5.

Comment #21 by gnu_andrew@member.fsf.org on 2010-06-23 18:25:31

Couldn't you instead have an --enable-debug option which sets -O0?

Comment #22 by stefan@complang.tuwien.ac.at on 2010-06-23 18:26:28

Would be a good idea, actually...

Comment #23 by gnu_andrew@member.fsf.org on 2010-06-24 20:07:21

The changeset http://mips.complang.tuwien.ac.at/hg/cacao/rev/dd4532afd08a breaks the
parsing of the version number with the change from pre to pre1:

Before:

#define VERSION_MAJOR 1
#define VERSION_MINOR 1
#define VERSION_MICRO 0
#define VERSION_EXTRA "pre"

After:

#define VERSION_MAJOR 1
#define VERSION_MINOR 1
#define VERSION_MICRO 0pre1
#define VERSION_EXTRA "pre1"

Comment #24 by gnu_andrew@member.fsf.org on 2010-06-24 22:57:19

Created an attachment (id=71)
Version parsing fix

Comment #25 by gnu_andrew@member.fsf.org on 2010-06-24 22:57:45

The attached patch fixes the version parsing bug.  Either apply this or drop the 1.  The
build is currently broken.

Comment #26 by stefan@complang.tuwien.ac.at on 2010-06-24 23:57:37

Sorry for the breakage, I didn’t notice it because I only tested a build with GNU
classpath, not with OpenJDK. Thanks a lot for the patch.

http://mips.complang.tuwien.ac.at/hg/cacao/rev/66e762d869dd

Attachment id=70

date:2010-06-22 01:27
desc:Added patch for -O2
type:text/plain
download:optflags.patch

Attachment id=71

date:2010-06-24 22:57
desc:Version parsing fix
type:text/plain
download:version-parsing.patch