Android goes for 3D !!! 3D UI for android... a journey of.....

Wouldn’t it be cool to have a 3D UI in your Android™ application? I’m Erik Hellman, Lead Software Architect for Android development at Sony Ericsson, and I’ll explain how to make this possible. In this article, I’ll describe how to incorporate 3D with Android’s UI API and OpenGL ES 2.0 graphics, and I provide a bitmap code example that can be used as input to a texture in OpenGL ES 2.0.

Being a game developer for Android is getting easier every day. The devices are getting high-end GPUs, loads of memory and have nice multi-touch displays. There are also several game engines announcing their support for Android, making the time-to-market for game developers much less than it used to be. Creating advanced games for Android no longer requires a huge project involving hundreds of people and spanning over 2 years or more.

While game engines can help us create advanced 3D games, it is not as easy to create an application that uses 3D graphics to enhance the user experience. The engines are focused towards gaming and usually lack a tight integration with the existing Android API and services.

In this post, I will explain how you can combine the best from Android’s UI APIs with hardware accelerated OpenGL ES 2.0 3D graphics. At Droidcon 2011 in Berlin, I presented a session on this topic and showed some demos of how 3D graphics can enhance your applications.

  

  

  

 

 

Visitors in the Sony Ericsson booth at droidcon 2011 in Berlin, Germany.

The main problem with doing 3D graphics in Android applications is that the Android API for UI (android.widget.* and android.view.*) has no solution for manipulating the View objects in 3D. OpenGL ES 2.0 on the other hand can do 3D very well, but has no knowledge about Android’s touch and input APIs, lacks layout managers and is really bad at rendering content such as text. The first thing I’ll show is how you can combine the two technologies in the best possible way.

All user-interface components in Android are based on the View class. Regardless if you show a button, text or a list of objects, you will use one of the many sub-classes of View. When a View, and all its children, is drawn there are two things happening. All Views in the hierarchy are first measured and then the layout manager in Android positions them as specified. This is done by a call to the measure and layout method in the View class. All this usually happens behind the scenes in standard Android 2D UI, but if you wish to use your Android View in a 3D scene you need to do this by yourself.

Once measure() and layout() have been called we can draw the View using the draw() method. This method takes a Canvas as a parameter. The Canvas, in turn, can be created to draw on a Bitmap. This is where we tie it all together in a nice utility method:

public static Bitmap drawViewToBitmap(View view, Bitmap bitmap) {

   bitmap.eraseColor(Color.TRANSPARENT);

   int bitmapWidth = bitmap.getWidth();

   int bitmapHeight = bitmap.getHeight();

   Canvas canvas = new Canvas(bitmap);

   int measureWidth = View.MeasureSpec.makeMeasureSpec(bitmapWidth, View.MeasureSpec.EXACTLY);

   int measuredHeight = View.MeasureSpec.makeMeasureSpec(bitmapHeight, View.MeasureSpec.EXACTLY);

   view.measure(measureWidth, measuredHeight);

   view.layout(0, 0, bitmapWidth, bitmapHeight);

   view.draw(canvas);

   return bitmap;

}

The above method lets you reuse the same Bitmap instance and thus reduces the amount of memory allocations you have to do (which takes time).

The Bitmap that we just drew our View to can be used as input to a texture in OpenGL ES 2.0 that can be placed on a 3D model (or, in the simplest case two triangles forming a flat quad).

For examples of how to create a simple OpenGL ES 2.0 application where we use bitmaps as textures, I suggest looking at the samples in the official Android SDK.

This method mainly solves the problem of mixing Android UI content with OpenGL ES 2.0 scenes. There is still a lot you need to build yourself, such a scene graph, handling user input and animations. However, you now have enough to start experimenting and I’ll be posting a follow up on this where we look at how to build a simple scene graph and handling user input.

Note: Droidcon Berlin is a two day barcamp and conference that explores all aspects of developing for Android mobile devices. Happening in Berlin at the Urania Berlin on March 23 and 24, droidcon Berlin will host workshops on Android apps and game development. Over 1200 people are participating in this conference.