Topic: Available Memory and Flash Lite memory leak

If you've programmed for Flash Lite 2.x, you're probably familiar with this command, which returns remaining available heap memory:
fscommand2('GetFreePlayerMemory')

On mobile devices based on BREW or Symbian, the total memory available to the Flash Lite application typically starts at 2 MB, and goes down as the program runs. 
On Chumby (at least the one I got my hands on) the memory starts at 8 MB, which is great.

However, the above command doesn't seem to return accurate values on the Chumby.  I did a test where I had it load a jpg when I press a button, and every few milliseconds I update the display of free available memory.  The problem is that the free memory display didn't change, even after the (large) JPG loaded.  Is this a known issue?

It's important to know and follow the memory situation during development of Flash Lite applications.  On mobile devices, this is a BIG issue.  It's even a bigger issue because the Flash Lite player from Adobe seems to have some memory management problems, and their garbage collector doesn't always work.  (If you only create desktop/browser apps, then you wouldn't know or care about this.)

I would very much appreciate it if the person at Chumby Industries who ported Flash Lite to Linux commented on the GetFreePlayerMemory command. 
Is it supposed to work properly? 
Are you aware of the memory management problems in Adobe's code, and were you able to spot and fix any of them?  (One problem area is their XML parser.)

Re: Available Memory and Flash Lite memory leak

OK, this is a complicated topic.  There are substantial differences between how the chumby manages the FlashLite memory vs the way other devices do.

Basically, the way normal FlashLite players work is that the player maintains an internal heap from which is allocates all memory.  The player requests memory from the operating system in blocks of 32K - the calculation that is typically done to respond to the GetMaxAvailablePlayerMemory is to subtract the number of blocks allocated from the memory allocated, and multiply by 32K.  Sometimes, if a new Flash allocation happend to fit in the blocks already allocated, the value of GetMaxAvailablePlayerMemory will not change.

On a chumby, for some rather obscure technical reasons, some, if not most, of the memory is allocated out of the normal application heap, though malloc() and free() (actually through an intermediate class to handle leak detection, etc).  However, there's still some allocated through the normal mechanism.  At the moment, memory allocated out of the malloc()/free() method is not accounted for in the GetMaxAvailablePlayerMemory statistic - this is a bug.

It's possible that we may go back to the "official" FlashLite method for allocating memory, if we can resolve the particular issues that caused us to do it this way in the first place, in which case FlashLite would be reporting the "correct" values.

In reality, our player, and the chumby overall, does other things with memory that make it rather difficult to know exactly how much is available at any given time.  In a new player we're working on, there's also a dynamically resizing HTTP cache and other functionality to improve the overall performance and reduce the network load.

As to leaks, chumbys now run for weeks cycling through thousands of widgets without crashing due to memory leaks.  However, this is because all of the widgets published so far explicitly avoid the situations that typically cause leaks - it's possible to have leaks not be a problem if the developer understands why they happen and how to avoid them.  The Control Panel, and a huge fraction of our widgets, use the XML object and do not leak.

In the production units, it's very likely that each widget will be started with a clean player instance, and its memory usage far more closely monitored than in the current prototype units.  However, I think you should be able to rely on 8MB or so for your movies.

Re: Available Memory and Flash Lite memory leak

Thank you for that detailed answer.  That explains why I didn't see the available memory decreasing when I loaded the JPG.

I noticed something in the "What's new?" section here:
http://www.chumby.com/corporate/update

It says "upgraded Flash player to Flash Lite 2.1" and the next line says "fixed memory leak" !!

So I have questions about this: 

1. Did you get new source code from Adobe for Flash Lite 2.1? 
2. Will this allow developers to use things like (xml) socket communication, not available in Flash Lite 2.0?
3. Most importantly, did you notice (as I have) the memory leak in Adobe's code, and is that where you fixed it?  Did you share the fix with Adobe so they don't ship Flash Lite 3.0 with the same bug? 

Thanks again for your time.

Re: Available Memory and Flash Lite memory leak

flashguru wrote:

It says "upgraded Flash player to Flash Lite 2.1" and the next line says "fixed memory leak" !!

The memory leak was in our implementation of the HTTP network stack, not Adobe's code

1. Did you get new source code from Adobe for Flash Lite 2.1? 
2. Will this allow developers to use things like (xml) socket communication, not available in Flash Lite 2.0?
3. Most importantly, did you notice (as I have) the memory leak in Adobe's code, and is that where you fixed it?  Did you share the fix with Adobe so they don't ship Flash Lite 3.0 with the same bug?

1) Yes - we actually received 2.1 source code just prior to Foo, however, we did not have the necessary time to port and adequately test it in that release.  We needed to address a number of Linux issues, primarily related to the wifi driver support and new Katamari hardware before the firmware 128 release.
2) We have not yet added the necessary additional support for XML sockets - it's high on our list of things to implement, and will likely be available as an update.
3) We are not aware of a memory leak in Adobe's code.  We've run our player,with the chumby in normal operation, with memory leak detection enabled for days with no increase in memory usage.

If you have specific movies that generate leaks, I'd be willing to run our detector against them to see what might be happening, however, I'd need the FLAs.

This is not to say that our widgets have never had leaks - however, all of them were addressable in Actionscript, not the player.  The biggest offender is the assignment of XML objects to local variables.

5 (edited by flashguru 2007-03-05 15:19:47)

Re: Available Memory and Flash Lite memory leak

Here's an example of a problem releasing memory.  It's not exactly a memory leak - I couldn't put together such a concise example of that.  However, it does show that there's a problem reclaiming memory after an object is deleted.

This is something anyone who has installed the Flash Lite extension for Flash Pro 8 can try in the emulator, set to emulate a generic device.  No FLA is needed: simply create a blank FLA, set the publish settings to Flash Lite 2.x / Actionscript 2, and paste this code on the first and only frame.

When you run the test, it creates a string and displays the available memory left.  When you press any key, it parses the string as XML, and then deletes the XML object.  Even though the XML object is deleted, the available memory still drops by a few hundred KB, and does not go back up again. I also tried setting the XML object to null or undefined, with the same result. 

I ran this test on actual Nokia devices and saw similar behavior to that of the emulator, even after waiting several minutes for the garbage collector to kick in.  I wanted to try something similar on the Chumby, but as explained in this thread, it doesn't yet give an accurate indication of remaining memory available.

Stage.scaleMode = "noScale";
fscommand2( "FullScreen", true );

var xmlObj:XML = new XML();  // not a local variable

// create an XML string to parse
var xmlString:String;
xmlString = "<Data>";
for(var j:Number=0;j<1000;j++) {
    xmlString += "<Node>0123456789012345678901234567890123456789</Node>";
}
xmlString += "</Data>";

this.createTextField("mem", 1, 0,0,176,24);  // TextField for available memory display
this.createTextField("out", 2, 0,24,176,180); // TextField for other messages

this.onEnterFrame = function() {
  this["mem"].text = "Available memory: " + fscommand2('GetFreePlayerMemory') + " KB";
}

this["out"].text = "Press any key to parse XML\n";

Key.addListener(this);  // on Chumby you'd wait for a screen-touch instaed
onKeyDown = function() {
    xmlObj.parseXML(xmlString); 
    this["out"].text += xmlObj.firstChild.childNodes.length + " child nodes parsed\n";
    delete xmlObj;  // also tried setting xmlObj to null and undefined
}

p.s. This is more of a question for Adobe than for the Chumby folks, so I'd completely understand if you're too busy to look into it.  However, I've found memory management to be a big issue in data-driven Flash Lite applications, so it could be a possible issue for Chumby apps some day.

Re: Available Memory and Flash Lite memory leak

I'll give it a try and see what happens in our player with leak detection on.  Thanks for posting the code!

Do you happen to know the _version of these players?

7 (edited by flashguru 2007-03-05 15:42:02)

Re: Available Memory and Flash Lite memory leak

If you’re referring to the version information I can read in Actionscript via System.capabilities.version, then in the Flash Lite emulator within Flash Professional 8 it is "FL 7,2,14,6", and on my actual Nokia device, it is "FL 7,2,100,20"

I don’t think this issue is related to a specific build of Flash Lite.  It’s interesting that it even shows up in the emulator.  I really hope I’m simply doing something wrong, and there is a way to force the freeing of memory.

Anyone who has Flash Professional 8 can try the above code and see the same thing. What I’m looking for is a way to free the memory that’s taken up during the "parseXML"  command in the sample code above.

Re: Available Memory and Flash Lite memory leak

I guess the first thing I'd try is creating a new XML object and assigning it to the variable xmlObj in the keyDown handler - ie, not reusing the same XMLObject.

It's actually a sort-of good thing your "delete" didn't work, otherwise the parse on the next keyDown would have failed.

9 (edited by flashguru 2007-03-05 19:26:19)

Re: Available Memory and Flash Lite memory leak

Duane wrote:

It's actually a sort-of good thing your "delete" didn't work, otherwise the parse on the next keyDown would have failed.

I am aware of that.  My goal was to create a test that lets you click a key once and see the problem, not to create a real application. 
BTW, the "delete" does work, in the sense that it deletes the reference to the object.  Therefore, subsequent keyDown events do not successfully parse any XML, as you can verify.  The problem is that the garbage collector doesn’t free the memory, as you can see if you try running this code.  The program updates the display of available memory with each onEnterFrame (yes, I know it’s not efficient to update the textfield on every frame unless it changes, but again, that’s not the point of this test.)

Duane wrote:

I guess the first thing I'd try is creating a new XML object and assigning it to the variable xmlObj in the keyDown handler - ie, not reusing the same XMLObject.

I tried that, and several other combinations.  The result is the same - the available memory decreases and never goes back up.

I really appreciate your time, and that of anyone else who might reply.  However, since the code is available for anyone to test, it would be nice if someone actually tried their proposed fix before suggesting it.  It should be easy for anyone who’s got Flash 8 with the Flash Lite extension on their desktop to try it.

Re: Available Memory and Flash Lite memory leak

I posted the FLA containing the above sample code here:
http://www.mediamax.com/flashguru/Hoste … rseMem.fla

Hopefully, this makes it even easier try and see the problem I am referring to.  You just need Flash 8 Pro with the Flash Lite extension installed to run this.

If anyone finds a way to release the Flash Lite memory allocated in this sample application, please do share your findings.

Re: Available Memory and Flash Lite memory leak

It turns out others have had the same memory problems with Flash Lite, without resolution:
http://store2.adobe.com/cfusion/webforu … erthread=y

After doing more tests, and thanks to Duane’s first response above, I think I understand what's going on.  I'll describe it here, in case it helps someone in the future.

First, Flash Lite seems to have a 2-tiered memory allocation scheme.  The first tier allocates memory in 32 KB chunks from the Operating System, as Duane described, and NEVER GIVES IT BACK to the OS, as long as the Flash Lite application is running.  The second tier allocates memory that's been reserved from the OS, for use by Actionscript.  When an Actionscript object is deleted, and properly garbage collected, it goes back into the pool that's available for other Actionscript objects, but it is not returned to the OS.

The command fscommand2("getFreePlayerMemory") only tells the Actionscript developer how much memory can still be reserved from the OS, even though more memory may be available for Actionscript objects, due to other objects being deleted already.  Therefore, the value returned by fscommand2("getFreePlayerMemory") only decreases, and it will generally not go up again even if you properly delete all objects,.

The other key factor is that the garbage collector runs only once every 60 seconds, even though there is no way to detect this from the Actionscript, because we cannot read the fact that there is more memory available. 

Therefore, if you try this:

1. Load a JPG or Large XML.
2. Delete/Remove it.
3. Load the JPG/Large XML again.
4. Delete/Remove it again.

You will notice the value returned by fscommand2("getFreePlayerMemory") drops (approximately) twice as much as if you try this:

1. Load a JPG or Large XML.
2. Delete/Remove it.
3. *** Wait at least 60 seconds (use setInterval)  ***
4. Load the JPG/Large XML again.
5. Delete/Remove it again.

In the second case the garbage collector works in between the load operations, and the memory can be reused.