Tag Archive: Java

Java garbage collection and device resources

For Java programmers the garbage collector is a pretty nice thing. Peter wrote a bit about Java and the garbage collector on this blog a while ago. In short, the garbage collector will make sure everything that isn’t needed anymore will just quietly disappear. Most of the time you’ll never know it is there. And after a while you sort of forget its existence, and when you do encounter it, it might be a painful experience.

I had one of those encounters when I was nearly finished with our new render-core and rolled it out to all developer machines. One of the laptops (with only an AMD APU) suddenly started running out of video memory very fast. That was strange, the memory footprint of Caromble! isn’t big and hasn’t changed with the new core. So what was happening?

After a bit of debugging it proved related to the rendering of text. Each time some in-game text would change, we would allocate a new vertex array on the GPU with exactly enough space for each letter. For each text (and the game has about 5 of these at any one time) only one buffer was used. Hardly exciting and not something you would expect to cause problems with video memory. Nevertheless, these buffers were causing the problem.

I looked at how the video buffers were disposed. All references to a buffer on video RAM (with a backing system RAM copy) were being managed by a ReferenceQueue. It is a solution from theArdor project that I think is both elegant and powerful. As long as the data can still be used it will remain on the video card. When it can’t be reached anymore, because all references to it are destroyed, it will be scheduled for garbage collection. If we poll the RefenceQueue regularly we’ll get a reference to all buffers scheduled for destruction and can tell the GPU that it can free the corresponding vertex array. So it isn’t necessary to keep track of who is referencing the data, but we still get to execute custom code for cleaning up the memory from the GPU. Perfect right?

Except it isn’t. As soon as the garbage collector detects that a video buffer can’t be used anymore it will return a reference to the object holding that buffer for us. But there lies the problem. We won’t get that reference until the garbage collector determines that it can safely be removed. The only guarantee the JRE makes on this matter is that this will happen before Java runs out of heap-space. So as long as there is sufficient system RAM available the garbage collector might decide that it will be better for performance to just let the garbage lie around for a bit longer. Since it doesn’t know about the video memory we are using, it can’t know that leaving this bit of garbage can be dangerous.

So that was the lesson I learned from all this? The job of the garbage collector is to keep the program inside its heap-space and to try to be as silent about it as possible. If there is enough space there is no need to slow the program down by collecting. So it won’t.

Once I figured out what the problem was, the solution was quite simple (but not very elegant). I solved the problem by explicitly removing buffers that I know are no longer required from the GPU. Buffers that are potentially shared by different meshes are kept on the GPU until the backing CPU data gets garbage collected. But since these buffers don’t often changed that is not a problem.

It was quite an interesting insight in the workings of the garbage collector, and I think it also proves that you shouldn’t try to to be too smart around resources allocated out of sight of the garbage collector.

Java Game Programming

As you might already know, we use Java to build our game (+engine) in. While we wrote all engine code ourselves, we do use Ardor3D (in combination with LWJGL) for graphics and JBullet for physics (love them both).

Because Java is discredited quite a lot these days (mostly security issues), we wanted to write something about our experience using Java for game programming.

  • Is Java safe enough?

While there were numerous security issues found in Java the last few months (Java 8 even got postponed because of it), it won’t necessarily trouble you in using Java for games. The most problems found in Java all have to do in running Java Applets, not with running Java on desktop.
Of course, fewer people will install Java with all the commotion going on so it’s a little harder to make you game run out of the box. Luckily, it is pretty straight-forward to bundle the Java Virtual Machine with your game (as the good people over at Puppy Games already pointed out).

  • Is Java fast enough?

Short answer: yes! Longer answer: but like with every language, there are a few pitfalls… While it is unlikely that Java will ever reach the raw power of optimized C++ code, it does offer speed that comes very close while keeping its readability (of course, opinions will differ) and safeness (no overflow/pointer errors). On top of that, memory leaks are way easier to prevent.
On the down side, there is a warm-up time. It takes a while for the Just-In-Time compiler to compile the most used methods from byte-code. We work around this by running our menu in the game world so once you take the controls everything runs smooth. And while having a garbage collection makes 95% of working with objects a lot easier, it isn’t perfect either. This brings us to our next topic:

  • The Garbage Collector

Having a garbage collector at your side can be a real blessing when writing a game. You’ll never (well almost) have to worry about running out of memory because of memory leaks. But it comes at a price. As you know, unlike languages as C++ and C#, Java does not have structs -you cannot allocate simple groups of primitives on the stack and pass them by value. Therefore, even simple data like the typical Vector3 (basically 3 floats) have to be presented with objects. While the garbage collector normally does a great job cleaning things up for you, having this insane amount of garbage every frame will definitely lead to slowdowns.
The Java virtual machine will try to clear most garbage with minor garbage collections. That are small garbage collections that will hardly take any time (and will not slow your game down). If you leave so much garbage that the minor garbage collections cannot keep up, Java will do a major garbage collections to prevent your application from running out of memory. These major collections will make your game stutter every few seconds (killing for any game with fast-paced game-play).
The way to solve this is making sure the JVM can clean up all garbage with minor garbage collections. So, how do you do this? It’s simple; do not leave too much garbage!
We solved this by pooling our small/most-used objects. But how do you make sure that your temporary objects are always returned to the pool? We do this with the auto-closable interface in Java 7. We use (abuse?) the new “try()” functionality in the same way you would use “using()” in c# like this:

try(Vector3 temp = Vector3.getTemp())
… Some calculations with the temp vector …

And in the Close() of the Vector3 we put the temp vector back in the Vector3 pool of the current Thread. You will need a object pool per thread. Normally this is done using the ThreadLocal class but since we spawn our own worker threads there is no need for it; we get the temp object directly from the pool of the current Thread.
Of course, when pooling objects, it really helps to place some tactical asserts in the classes to make sure it doesn’t get altered while inside the pool (you should not hold references to those objects outside the try{} clause.
Also, it might help triggering a major garbage collection just after switching levels by calling System.gc(). This make sure all/most old data from the previous level is cleared up.
Most of the time, you only have a handful of methods that really spawn a lot of garbage, so grab the Java Visual Virtual Machine from the JDK to (easily!) find these methods and optimize those. Don’t go insane on trying to optimize all code, concentrate on the places where it matters.

  • UPDATE: Escape analysis

As Adrian Papari points out below, Java uses escape analysis to automatically place objects on the stack or replaces them with simple primitives (and thus preventing allocations & garbage). But for our game Caromble! pooling of the Vector3 at some critical places definitely was a must to get constant performance. To get to the bottom of this, I made a test app inspired by this example but made it a bit more complicated to resemble actual game code more closely. Still I made sure that only Vector3 classes where used (no string allocations etc). The results can be seen in the image:

Capture of Java Visual VM Top: escape analysis disabled Bottom: escape analysis enabled (default)

Capture of Java Visual VM
Top: escape analysis disabled
Bottom: escape analysis enabled (default)
The blue lines on the left show cpu time spend on garbage collection

As you can see, with escape analysis disabled the garbage collector was doing minor garbage collections the whole time (taking up quite a lot of cpu time) but still used heap memory was increasing still. Once it the it would have run out of heap space, a major collection would have kicked in to clean thing up.
As for the example with escape analysis, it runs a whole lot better, hardly spending time on garbage collection. But what is interesting to see is that it doesn’t keep the heap space constant! While it does take care of not allocating most Vector3’s on the heap, it does not optimize away all allocations of the Vector3 class and still some garbage collection is taking place.
This explains why in some extreme cases like game programming object pooling might still be useful.

  • DirectBuffers

While the JVM garbage collector is pretty good in handling large amount of garbage from objects, be very careful with native/direct buffers (you need those to communicate with OpenGL). Leaving a lot of direct buffers lying around will lead to major slowdowns pretty soon that can last several seconds! So be sure to reuse direct buffers (mostly mesh data) as much as possible.

To conclude, if you take note to the above and work your way around it, Java is a great language to write your game in. But yeah, like every other language, getting real time performance will always take some effort.