Topic: Simulating a read/write filesystem on chumby one

Have you every wanted to try putting some software into /usr/bin but you don't want to modify your firmware until you know it works?

Here's a cool hacker trick - the chumby one has the unionfs filesystem service built into the firmware - that means you can create and use a union mount on your device.

Here's the trick:

1) (insert a USB dongle, preferably formatted as ext3)
2) mkdir /mnt/usb/overlay
3) chmod 777 /mnt/usb/overlay
4) mkdir /mnt/overlay
5) mount -t unionfs -o dirs=/mnt/usb/overlay:/=ro unionfs /mnt/overlay
6) chroot /mnt/overlay

At this point, you'll find yourself in what appears to be a writable version of the chumby firmware - you can add, edit and remove files in directories that *appear* to be in the firmware.  Your changes are actually recorded in /mnt/usb/overlay.

To get out, simply exit, then "umount /mnt/overlay"

There are some caveats - some parts of the filesystem such as /proc and /sys are not available.  It's also possible to "bind" /dev to this filesystem with a few extra commands.

Thanks to help from ChumbyLurker, I've built an overlay that has a native gcc toolchain, with everything in the right place - no mucking with PATH, etc. - so one can actually compile and run C and C++ programs on the device itself.

Note that unionfs allows the chaining of more than two file systems - with some cleverness, you could create a read/write overlay on top of a read-only tools overlay, on top of the firmware, use the tools to build something, then remount without the tools overlay.  This would let you use the results of the build without polluting it with the build tools.  Pretty cool.

None of this is black magic - it's a pretty common trick with Live CD distributions of Linux.

Re: Simulating a read/write filesystem on chumby one

This is great Duane,

Could you post a few notes on getting the toolchain setup?

I believe doing compiles on the Chumby itself would be a big help for me.  I setup scratchbox according to the wiki and everything works great using the wiki examples.  However, ever time I try something new it's a struggle.  Maybe it's the PATHs, maybe missing dependencies, it seems like doing it on the Chumby will be easier for me.

Re: Simulating a read/write filesystem on chumby one

Well, this really should be considered a bit of a toy at the moment - the device may not have enough memory to build very large projects.  At the moment, the toochain dongle has just the compiler - no "make", etc.   One would really want something on par with Debian's "build-essential", along with more complete implementations of Perl.

I'll try to post a dongle image with what we have so far - perhaps others can take it further.

Re: Simulating a read/write filesystem on chumby one

I've uploaded the overlay image for native GCC here.  Just unpack it to a USB dongle, or /mnt/storage - it should create a directory called "overlay" and use it per the unionfs instructions above.

Your USB dongle should be formatted as ext3, or some other similar filesystem or the symlinks won't be properly created.  VFAT won't do the trick.  Since the C1 has mkfs.ext3, you can do this from the device.

5 (edited by rleyden 2010-02-07 19:10:35)

Re: Simulating a read/write filesystem on chumby one

Thanks Duane,

So far, so good.
As to whether this will be truly easier for me than Scratchbox, I guess that's doubtful (as you probably suspected).  I've actually made good progress on leaning Scratchbox recently.  (My most frequent problems were due to a misunderstanding of what "configure" was doing, rather than path problems).  My latest project "fswebcam" went relatively smoothly in Scratchbox after I fixed configure complaints about "PNG support not found in GD".  (Hopefully, more about fswebcam later.)

Just as a test, I tried building fswebcam natively.  I expected similar complaints about GD but instead got

chumby:/mnt/usb-5d4c7bb6-79bf-45ac-93c3-4da021de597f/fswebcam-20091224# ./configure --prefix=/usr
checking for gcc... gcc
checking for C compiler default output file name... a.out
checking whether the C compiler works... yes
checking whether we are cross compiling... no
checking for suffix of executables... 
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking how to run the C preprocessor... /lib/cpp
configure: error: in `/mnt/usb/fswebcam-20091224':
configure: error: C preprocessor "/lib/cpp" fails sanity check
See `config.log' for more details.

I don't mean to hijack your wonderful thread on overlays, it's just that I have no idea where to check cpp's sanity or lack there of.  Config.log gives only a few clues;

....  error: linux/limits.h: No such file or directory
 ....#error "You need a ISO C conforming compiler to use the glibc headers"

Again, thanks for the toy.  (You say toy like it's a bad thing smile)

Re: Simulating a read/write filesystem on chumby one

I hate to be an ungrateful whiner, but where is make?

chumby:/mnt/usb-5d4c7bb6-79bf-45ac-93c3-4da021de597f/overlay/fswebcam-20091224# make
/bin/ash: make: not found
chumby:/mnt/usb-5d4c7bb6-79bf-45ac-93c3-4da021de597f/overlay/fswebcam-20091224# find / -name make
find: /mnt/overlay: Operation not permitted
find: /sys: No such file or directory
find: /proc: No such file or directory

Re: Simulating a read/write filesystem on chumby one

As I mentioned earlier, there isn't one.  Unfortunately, you need "make" to make "make" from source.

However, I think I included "make" in this set of cross-compiled Flash utiilities, so you may just have to unpack that one and copy make into /usr/bin in the overlay.

Re: Simulating a read/write filesystem on chumby one

Sorry, I ignored and forgot your warnings in my burst of enthusiasm.

The make you provided in the flash utilities does the trick and should keep me occupied for a while.

Re: Simulating a read/write filesystem on chumby one

Seems like we may need kernel headers for this to be truly usefully (limits.h and that stuff).  In theory we should just be able to grab the chumby kernel source, extract to the overlay, and go.  I'll have to give it a spin later.

Re: Simulating a read/write filesystem on chumby one

I just tried this on my Chumby Classic and it failed. Is unionfs missing from the old firmware?  If so, can you add it in the future?

On a related note, next time you release a new firmware image, it would be hugely useful if there was an empty /opt directory in the root filing system onto which we could mount things.  There is a huge body of binaries compiled for the ARM from Debian sources (for the NSLU2 and the like), available as handy packages, which expect to exist in /opt. I have a hacked cramfs image with a /opt that I can mount these onto but I can't easily pass distributions on to others without getting them to re-flash their Chumby, which is a MUCH harder sell than giving them a USB stick and saying that they can just plug it in.

Re: Simulating a read/write filesystem on chumby one

The Chumby classic does not support unionfs.  You can see what filesystems are supported by running "cat /proc/filesystems".

As for the empty /opt directory.  Wouldn't that be possible if unionfs is enabled in the next Chumby release?

One last thing, you should be able to get this working without flashing a custom cramfs using a USB stick by having a debugchumby script that could load a kernel module for unionfs, then create an /opt directory in your mounted overlay.

I might try building unionfs for the chumby classic if I get time in the next few days, as this would be helpful for me as well.

Linux Guy - Occasional Chumby Hacker

Re: Simulating a read/write filesystem on chumby one

Unfortunately I need access to /proc for what I'm doing so unionfs doesn't work for me.  It also adds a whole extra layer of indirection for all file access for anything that uses the unified filesystem, and I don't need R/W access to the root filesystem.

Unionfs is clearly valuable for many applications but, in much the same way that many linux workstations run a locked down root FS and then NFS mount extra stuff on /opt, I think many Chumby users' needs can be covered by having a /opt onto which extra stuff can be mounted. Furthermore, it only costs 20 bytes in the cramfs image! :-)