This is the third installation of my Flex Mobile series.  In my first post I talked about multitouch, and my second post I dug in on how to control the soft keyboard.  This post goes into detail on performance, specifically caching.

Flash applications tend to use a lot more vector graphics than other UI platforms. This is a great thing for designers who can directly use all their path art and graphics.  However, on mobile devices it increases the challenge to build high performing applications.

Fortunately, there is a feature of the Flash platform called cacheAsBitmap (and its newer sibling, cacheAsBitmapMatrix) that lets you speed up rendering performance at the expense of memory.

Cache as Bitmap

CacheAsBitmap is a boolean property of DisplayObject, and by extension all the visual elements you use in Flash and Flex includings Sprites and UIComponents, have access to this variable. When set to true, each time the DisplayObject or one of its children changes it will take a snapshot of the current state and save it to an offscreen buffer. Then for future rendering operations it will redraw off the saved offscreen buffer, which can be orders of magnitude faster for a complicated portion of the scene.

To enable cacheAsBitmap on a DisplayObject you would do the following:

cacheAsBitmap = true;

Flex UIComponents have a cache policy that will automatically enable cacheAsBitmap based on a heuristic. You can override this behavior and force cacheAsBitmap to be enabled by doing the following:

cachePolicy = UIComponentCachePolicy.ON;

While cacheAsBitmap is a very powerful tool for optimizing the redraw of your application, it is a double-edged sword if not used properly. A full size screen buffer is kept and refreshed for each DisplayObject with cacheAsBitmap set to true, which can consume a lot of device memory or exhaust the limited GPU memory if you are running in graphics accelerated mode.

Also, if you have an object that updates frequently or has a transform applied, then cacheAsBitmap will simply slow down your application with unnecessary buffering operations.

Cache as Bitmap Matrix

CacheAsBitmapMatrix is also a property on DisplayObject, and works together with cacheAsBitmap. For cacheAsBitmapMatrix to have any effect cacheAsBitmap must also be turned on.

CacheAsBitmap does not work when a transformation, such as a rotation or a skew, is applied to the object. The reason for this is that applying such a transformation to a saved bitmap produces scaling artifacts that would degrade the appearance of the final image. Therefore, if you would like to have caching applied to objects with a transform applied, Flash requires that you also specify a transformation matrix for the bitmap that is stored in the cacheAsBitmapMatrix property.

For most purposes, setting cacheAsBitmapMatrix to the identify matrix will do what you expect. The offscreen bitmap will be saved in the untransformed position and any subsequent transforms on the DisplayObject will be applied to that bitmap. The following code shows how to set cacheAsBitmapMatrix to the identify transform:

cacheAsBitmap = true;
cacheAsBitmapMatrix = new Matrix();

Tip: If you plan on setting cacheAsBitmapMatrix on multiple objects, you can reuse the same matrix to get rid of the cost of the matrix creation.

The downside to this is that the final image may show some slight aliasing, especially if the image is enlarged or straight lines are rotated. To account for this, you can specify a transform matrix that scales the image up prior to buffering it. Similarly, if you know that the final graphic will always be rendered at a reduced size you can specify a transform matrix that scales down the buffered image to save on memory usage.

If you are using cacheAsBitmapMatrix to scale the image size down you need to be careful that you never show the DisplayObject at the original size. The following figure shows an example of what happens if you set a cache matrix that reduces and rotates the image first, and then try to render the object at its original size:

Notice that the final image has quite a bit of aliasing from being scaled up. Even though you are displaying it with a one-to-one transform from the original, Flash will upscale the cached version resulting in a low fidelity image.

The optimal use of cacheAsBitmapMatrix is to set it slightly larger than the expected transform so you have enough pixel information to produce high quality transformed images.

Flash Mobile Bench

The Flash Mobile Bench is a simple application that lets you test the affect of different settings on the performance of your deployed mobile application.

The functionality that it lets you test includes the following:

• Addition of a large number of shapes to the display list
• Animation speed of a simple x/y translation
• Animation speed of a simple clockwise rotation
• Impact of cacheAsBitmap on performance
• Impact of cacheAsBitmapMatrix on performance
• Impact of the automatic Flex cache heuristic on performance

The code that updates the cache behavior of the shape group is shown below:

private var identityMatrix:Matrix = new Matrix();

private function cacheOff():void {
  shapeGroup.cachePolicy = UIComponentCachePolicy.OFF;
}

private function cacheAuto():void {
  shapeGroup.cachePolicy = UIComponentCachePolicy.AUTO;
}

private function cacheAsBitmapX():void {
  shapeGroup.cachePolicy = UIComponentCachePolicy.ON;
  shapeGroup.cacheAsBitmapMatrix = null;
}

private function cacheAsBitmapMatrixX():void {
  shapeGroup.cachePolicy = UIComponentCachePolicy.ON;
  shapeGroup.cacheAsBitmapMatrix = identityMatrix;
}

Even though we have only one instance of an object to apply the cacheAsBitmapMatrix on, we follow the best practice of reusing a common identity matrix to avoid extra memory and garbage collection overhead.

Upon running the Flash Mobile Bench, you will immediately see the FPS counter max out on your given device. Click on the buttons to add some shapes to the scene, set the cache to your desired setting, and see how your device performs. The following figure shows the Flash Mobile Bench application running on a Motorola Droid 2 with 300 circles rendered using cacheAsBitmapMatrix:

How does the performance of your device compare?

This is part 2 of my Flex Mobile series.  Please see my first post for information on getting started.

When using the text components, the Android soft keyboard will automatically trigger upon focus as you would expect. However, sometimes you need finer grained control over when the soft keyboard gets triggered and what happens when it gets activated.

The soft keyboard in Flex is controlled by the application focus. When a component that has the needsSoftKeyboard property set to true is given the focus, the soft keyboard will come to the front and the stage will scroll so that the selected component is visible. When that component loses focus, the soft keyboard will disappear and the stage will return to its normal position.

With the understanding of the focus, you can control the soft keyboard by doing the following:

• To show the soft keyboard declaratively – Set needsSoftKeyboard to true for your component
• To show the soft keyboard programmatically – Call requestSoftKeyboard() on a component that already has needsSoftKeyboard set.
• To hide the soft keyboard – Call setFocus() on a component that does not have needsSoftKeyboard set.

This works fine for components that do not normally trigger the soft keyboard; however, for components that automatically raise the keyboard, setting needsSoftKeyboard to false has no effect. A workaround to prevent the keyboard from popping up on these components is to listen for the activating event and suppressing it with code like the following:

<fx:Script>
  <![CDATA[
    private function preventActivate(event:SoftKeyboardEvent):void {
      event.preventDefault();
    }
  ]]>
</fx:Script>
<s:TextArea text="I am a text component, but have no keyboard?"
  softKeyboardActivating="preventActivate(event)"/>

This code catches the softKeyboardActivating event on the TextArea component and suppresses the default action of raising the soft keyboard.

In addition to getting events on activation, you can also catch softKeyboardActivate and softKeyboardDeactivate events in order to perform actions based on the soft keyboard status.

The following is the full code listing for a soft keyboard sample application that demonstrates all these techniques used together to take complete control over the soft keyboard.

<?xml version="1.0" encoding="utf-8"?>
<s:Application xmlns:fx="http://ns.adobe.com/mxml/2009"
         xmlns:s="library://ns.adobe.com/flex/spark"
         splashScreenImage="@Embed('ProAndroidFlash400.png')">
  <fx:Script>
    <![CDATA[
      [Bindable]
      private var state:String;

      [Bindable]
      private var type:String;

      private function handleActivating(event:SoftKeyboardEvent):void {
        state = "Activating...";
        type = event.triggerType;
      }

      private function handleActivate(event:SoftKeyboardEvent):void {
        state = "Active";
        type = event.triggerType;
      }

      private function handleDeactivate(event:SoftKeyboardEvent):void {
        state = "Deactive";
        type = event.triggerType;
      }

      private function preventActivate(event:SoftKeyboardEvent):void {
        event.preventDefault();
      }
    ]]>
  </fx:Script>
  <s:VGroup left="20" top="20" right="20" gap="15"
        softKeyboardActivating="handleActivating(event)"
        softKeyboardActivate="handleActivate(event)"
        softKeyboardDeactivate="handleDeactivate(event)">
    <s:HGroup>
      <s:Label text="Keyboard State: " fontWeight="bold"/>
      <s:Label text="{state}"/>
    </s:HGroup>
    <s:HGroup>
      <s:Label text="Trigger Type: " fontWeight="bold"/>
      <s:Label text="{type}"/>
    </s:HGroup>
    <s:Button id="needy" label="I Need the Keyboard" needsSoftKeyboard="true" emphasized="true"/>
    <s:TextArea text="I am a text component, but have no keyboard?"
          softKeyboardActivating="preventActivate(event)"/>
    <s:HGroup width="100%" gap="15">
      <s:Button label="Hide Keyboard" click="{setFocus()}" width="50%"/>
      <s:Button label="Show Keyboard" click="{needy.requestSoftKeyboard()}" width="50%"/>
    </s:HGroup>
  </s:VGroup>
</s:Application>

This code creates several controls and attaches actions to them so that you can hide and show the soft keyboard at will, as well as see the current soft keyboard state as reported by the events that trickle up. When you run the application it should look like this:

Notice that the TextArea control, which normally triggers the soft keyboard no longer brings it up, while the highlighted button immediately raises the soft keyboard whenever it gets focus. The two buttons at the bottom to show and hide the keyboard merely play focus tricks to get Flash to show and hide the keyboard at will.

You can use the same techniques in your own application to take full control over the soft keyboard.

We did a double header meeting in the North and South bay to kick off the Flash On group. It was a lot of work to coordinate and present back-to-back meetings, but it all came together. A big thanks to my co-presenter Oswald Campesator, my co-coordinators Keith Sutton and Justin Webb, and also, Nick Turner, from Plug and Play’s Mobile Meetup, who did an outstanding job on Thursday evening.

Here is what some of our new members had to say:
Tony Constantinides
“ Great meetup and very informative. Many good issues were raised at the meeting by developers which will lead to a followup meetup which will be hands-on hopefully. With Mobile nothing beats hands on with the fun devices! The possibilities of Android development with TV, tablets and mobile seem endless! ”

Drew Dara-Abrams
“ A useful introductory presentation and discussion. The mix of formal presentation and informal question and discussion worked well. ”

.

Aaron Tong
“ This was a great meetup! Lets have more of the same! ”

.

As promised, here is the presentation that Oswald and I gave (skip to page 30 for the links):

If you haven’t already, sign up for the Flash On meetup group to get informed of upcoming events:
http://www.meetup.com/flashon/

I am pleased to announce the Flash On… user group that I am kicking off together with Keith Sutton, Oswald Campesato, and Justin Webb.  The focus is Flash on consumer devices from Mobile to Tablet to TV.

Oswald and I will be doing the inaugural presentation on Flash mobile technologies this evening.  You can catch the live stream on Adobe Connect here:

http://experts.na3.acrobat.com/flashondevices/
(Stream starts at 7PM PST!)

For those of you who haven’t been following the Flash Mobile headlines, there have been a lot of great announcements that make this platform worth developing for:

Mobile

• With the AIR 2.5 release, Android devices are fully supported
• Apple has relaxed their license to allow Flash-based applications in the App Store
• Similar announcements have come from other vendors such as Palm, Windows 7, and others

TV

• Google TV prominently features Flash support
• Adobe also announced AIR support for Samsung devices such as Smart TVs and Blu-ray Players

Tablet

• Blackberry announced Adobe AIR support for their Playbook Tablet

Here is an excerpt from the Adobe Max 2010 keynote that shows off the Blackberry Playbook Tablet running Flash:

When put together, Flash is well poised to become the defacto standard for building rich user experiences across different screens.

We will cover all this and more in our presentation tonight.  As usual, we will have high production values for the talk with side-by-side presenter video and slides plus a chat area to ask questions.  I hope to see you there!

I haven’t talked much about Flash technology on my blog, but we use quite a bit of Flash/Flex for developing enterprise apps at my day job.  With the Open Screen Project from Adobe making Flash available on mobile and embedded devices, Flash has become a viable cross-platform toolkit fulfilling a lot of what I hoped JavaFX Mobile would become.

Yesterday at Silicon Valley Code Camp I did a talk on Flash Android development to a packed room.  About half the audience were Flash/Flex users, with a smaller, but very vocal, contingent of Android developers.  The goal of the talk was to help get folks off the ground with Flash Mobile development using the Android SDK in combination with Flash CS5 or Flash Builder 4.

The examples for the talk came from the upcoming Pro Android Flash book that I am writing for Apress together with Oswald Campesato and Dean Iverson.  This book will be coming out around Spring 2011, but there is already quite a lot of good content that we have finished.  The responses I got from attendees of the talk were extremely positive, but check out the presentation and see for yourself:

Download the PDF

As I continue working on the book, I plan to increase the coverage on Flash and Flex Mobile in this blog.  It is a slight shift, but consistent with my philosophy around promoting rich client technologies, and won’t decrease my focus on JavaFX.  Hopefully you find some value in this as a technology that integrates well with Java and opens up some new mobile deployment capabilities.

In case you missed the big event last week, I have finished post-processing and uploading the video.  We took the quality up a notch by getting a direct screen capture from the presenter laptop.  This means that you will not only get crystal clear slides, but also full-screen demos and a nice tight head-shot of the presenter.  This moves our video setup firmly up from a Level 4 to a premium Level 1 operation as detailed in Stephan Janssen’s blog.

Without further ado, here is the Parleys version of Hinkmond’s JavaFX Mobile Dojo talk:

I got a lot of requests for just the slides last time, so I am also making them available here:

Finally, a quick plug for our next SvJugFx event.  We will be doing a double feature with folks from the Java Store and JFrog Artifactory presenting back-to-back.  Even if you plan to attend online, make sure to sign-up here:
http://www.svjugfx.org/calendar/12559455/

I was the very first person to buy an HTC Diamond at JavaOne.  (Jacob Lehrbaum probably thought he was about to be mugged as I stalked him into the Java Store.)  It worked out great for my presentations, but I had to tip-toe around some issues that showed up only on applications deployed to the phone.

However, the latest JavaFX 1.2 EA release is ready for primetime!  The installation was a breeze and all of the JavaFX applications I have tried on it so far have worked great.  This includes:

• Hello Earthrise – Starter application from the Pro JavaFX book
• DrawJFX – Draw on your phone using the touchscreen, also from the Pro JavaFX book
• Nabaztag widget – I demonstrated this at JavaOne; now is your chance to impress your co-workers with silly bunny tricks

Hello Earthrise on the HTC Diamond

Here is what to like about the latest JavaFX Mobile release running on the HTC Diamond:

• Excellent resolution – It takes advantage of the native 480×640 resolution of the Diamond HTC device, allowing for some very detailed graphics.
• Improved performance – Startup time and application performance are orders of magnitude faster than with JavaFX 1.1.
• Some cool demo apps – TwitterFX Mobile takes the cake as the most interesting and useful app, and is bundled with the JavaFX 1.2 EA Release. Kudos to Liang Zhu and Jungeun Woo on the port and Steven Herod on inventing TwitterFX!
• Available for download – Unlike previous JavaFX Mobile releases, which were Sun internal or pre-installed on a device, this is a free download from the JavaFX website.  While it is only officially supported on the HTC Diamond and LG Incite, it reportedly works on other Windows Mobile devices, such as the XPeria X1.

If you have a Windows Mobile 6 or 6.1 device available, it is definitely worth giving this early access release a try.  Who knows, your JavaFX applications may already be mobile ready!

Through a generous invitation by Sun, I had the opportunity to attend the Mobile, Media & eMbedded Developer Days (M3DD) Conference down in Santa Clara.

There was definitely some very cool stuff going on, most notably the upcoming JavaFX Mobile release.  Is it real?  Well, here is a screenshot of JavaFX running on a real device:

JavaFX Mobile Running at M3DD

The official release is due out in February, which will include full support for deploying JavaFX applications that use the common profile to mobile devices.  The JavaFX Mobile team has been super busy cranking out the last few bits for the upcoming release, but were at the conference in force, armed with a myriad of devices to show the platform capabilities.

Also of note was a cool presentation by Ariel Levin showing mobile widgets deployed to a consumer portal written in LWUIT.  Right now their technology is not interoperable with JavaFX Mobile, but they definitely have support of JavaFX widgets in their future roadmap.

Mobile Widgets Written in LWUIT

Although, I have to give it up to Ken Gilmer of Bug Labs for coolest demo.  While it is not a mobile phone, the BUG has dozens of modules that allow low level access to motion sensors, cameras, GPSs, and other sensors.  Ken also has invented a new demo style called redo programming where he did a fully functional demo with real code exclusively using the undo buffer in Eclipse!

Ken Gilmer Gets the BUG

This was all great motivation to do some more work on deploying WidgetFX widgets to mobile devices.  More on this once JavaFX Mobile is released, but look forward to some exciting widget action in your pocket!