1 Topic by dougg3 (edited by dougg3 2013-12-01 21:41:32)

  • Registered: 2013-11-30
  • Posts: 36

Topic: Newer kernel for silvermoon

Hey everyone,

I've been playing around a bit with trying to bring my Chumby 8 up to a recent kernel version. I soldered on the console header and went to work testing things.

I found all the stock boot partitions confusing, so I changed my SD card to only have two partitions: the bootloader partition and a main (currently barebones) root filesystem that runs a test Qt program. I've also patched u-boot's boot script to load the kernel from my main root filesystem partition.

I thought getting a newer kernel running would be easy, but it turns out that a bunch of stuff in the mainline kernel for the PXA168 is broken. With the help of a bunch of random patches I found and some time spent debugging, I have many things working on my Chumby 8 on Linux 3.13-rc1:

  • Kernel actually runs -- it turns out the L2 cache needs to be disabled early in the boot process or newer Linux versions will crash randomly and stuff -- then Linux can turn it back on when it's ready for it

  • USB works

  • LCD works. Still need to make the overlay frame buffer work too, but that's not a high priority

  • The internal boot SD card works and I can read/write it

  • I think the internal SDIO wifi module is sort of working, but it has issues -- it has a timeout error during the driver load unless libertas debugging output is turned on, not sure if it's a libertas problem or a PXA168 SDIO controller problem

  • I'm currently booting to an NFS root over a USB ethernet adapter, but it should boot to a root filesystem on the SD card now too

  • Shutdown and reboot commands work correctly

Still needs to be done:

  • Port touchscreen driver and other custom Chumby stuff to the new kernel

  • Get sound output working -- this may be a hefty task, I don't see anything obvious in the mainline kernel for it

  • Fix small issues I listed in the "working" stuff above

  • Make the built-in USB SD/CF/etc. reader work? Not tested yet, maybe it already works

  • There's probably more that I haven't thought of...

Has anyone else done any work on a newer kernel? On a related note, has anyone else actually recompiled u-boot from scratch for this machine? I've downloaded all of the tools from the chumby source code page, but there are so many crazy things to put together to make the PXA168 boot from an SD card partition and I haven't been able to figure out all of the tools needed to create a new bootloader partition from scratch.

I'd definitely be interested in working together with people on getting the rest of the stuff I listed above working...this device is so cool and I think it deserves to get some modern Linux support! I also have some falconwings...maybe that can be the next "new kernel" project?

I'd love to share my work so far if anyone is interested in kernel hacking. I'll put the patches up somewhere soon. If you're interested, I'd definitely recommend saving your stock microSD card and getting a new one to use just for the kernel hacking stuff so you can go back to the known working stuff if needed.

  • From: Oceanside, CA
  • Registered: 2008-04-17
  • Posts: 650

Re: Newer kernel for silvermoon

I'm interested in your kernel work, but I'm equally interested in the uboot stuff.  I am always afraid to mess with bootloaders for fear of bricking things.  Can you give more details about the bootloader work, changes, etc. you've done and how others can do them safely without fear of bricking their devices?

Linux Guy - Occasional Chumby Hacker

Materdaddy's Website

  • Registered: 2013-11-30
  • Posts: 36

Re: Newer kernel for silvermoon

Absolutely! First of all, these Chumby devices are essentially unbrickable as long as you have a good bootable SD card. I got another microSD card and cloned my original card's contents to it using dd. Now I do all my work on the new card. I have the original card somewhere safe, so if I absolutely destroy something on the filesystem, I can always go back to the original card. Alternatively, you could copy the original card's contents to a file using dd, and then if it ever gets screwed up, you can restore its original contents using dd. Of course, either route you pick, it's going to involve opening up the Chumby and temporarily pulling out the card so you can work with it on a desktop computer (preferably a desktop computer running Linux).

I guess it would technically be possible to fry something on the board if you drove something at 3.3V which expects 1.8V or something like that, but I haven't run into anything like that yet.

This particular device has a PXA166, but it's essentially the same thing as a PXA168 except for running at a lower clock rate.

The processor starts up as follows (this is also documented in Marvell's Armada 16x Boot ROM Reference Manual):

  1. The power is switched on, and the PXA166's internal boot ROM (built into the chip, cannot be modified by anyone) runs

  2. The internal boot ROM probes attached devices to look for a bootable image

  3. As soon as it finds a device containing a bootable image, it loads it into RAM and runs it

The Chumby 8 appears to have been designed to boot from two possible devices: the microSD card or SPI flash. The pads for an SPI flash chip are there, but it's not populated. So it all depends on the microSD card containing a good boot image.

In order to be bootable, the first partition of the card has to contain some data in a particular format that the PXA166's boot ROM recognizes. This data can contain a bunch of different sections that do things such as configure the SDRAM's timings or load a bootloader. I know for sure that u-boot lives somewhere in that partition, and I think there is an earlier bootloader in there too (whatever prints the messages "Hello world, I'm the most incredibly annoying boot process you'll ever meet!" and "levelling up").

The only real hacking I have done to u-boot so far is to change its startup script. At the offset 0x2c370 of partition 1 of the SD card, you can find the boot script that u-boot runs. Each command is separated by a NULL (0) byte, then when two NULL bytes are found in a row that signals the end of the boot command list. I deleted a bunch of the stuff in there that figures out the current boot partition, displays pictures, etc. and modified it to load the kernel from partition 2. I also added some extra commands that set different boot args for booting from NFS. (Deleting the original Chumby logo and rfsA/rfsB selection stuff from the script gave me room to do this)

Below that, at 0x2c97d, there are a few default environment variables. I changed this so bootdelay is 1 which gives me a moment to interrupt the boot process when it starts.

It's great to be able to modify the binary, but I want to figure out how to put in a new compiled u-boot instead of hacking the existing binary. The problem is the boot partition is assembled from a bunch of different parts and pieces, and I'm not sure how to fit it all together. I don't even know if the source code or tools to assemble it all are still available. Marvell is no help because it seems you have to sign an NDA just to get access to anything, and I don't think they want to talk to individuals.

Anyway, if that can be figured out, it would open up the door to being able to recompile u-boot. I know how to compile u-boot already, but I don't know how to mix it in with the rest of the various sections in the boot partition. All of the source code that Chumby provided may be useful...aspenix-1.0.tgz on Chumby's source code page looks interesting, for example. I think I found some stuff for OpenEmbedded that may be useful too, but I can't remember.

Hope this helps a bit :-)

  • Registered: 2012-02-26
  • Posts: 40

Re: Newer kernel for silvermoon

I've been wanting to work on the same thing for a while now, and I'd be delighted to see what you've come up with and try to help out. Do you have your modified kernel posted anywhere? (If not, any chance you can stick it up on Github or something?)

As far as the boot process goes, the first-stage bootloader lives on a serial flash chip on the back side of the C8 board. It's not documented (it may be under NDA), but I believe it's basically just implementing something similar to the imx233 (Falconwing) bootloader. If you aren't already familiar with that, reading up on it may help explain what's going on.

  • Duane
  • Duane
  • Blue Octy
  • Offline
  • From: San Diego, CA
  • Registered: 2006-08-24
  • Posts: 8,211

Re: Newer kernel for silvermoon

I believe that both the PXA168 and IMX233 required proprietary, signed boot loaders for DRM etc (the IMX233 was designed for MP3 players). 

As I recall, what we did is build just enough of a boot loader to launch an unsigned loader.

I think there was another issue with the PXA168 where the silicon had a bug that made it fail to boot off SD as advertised, so we added a small bit of flash with another loader that fixed that.

Duane's Website

  • Registered: 2013-11-30
  • Posts: 36

Re: Newer kernel for silvermoon

Howdy! I don't have the modified kernel posted anywhere yet. I'm trying to figure out a good way to post it and keep it up to date as newer mainline kernels come out. I'm pretty dumb when it comes to git, so this might be a learning experience :-) Github sounds good to me. Right now I basically have a few patches that go against 3.13-rc1 (which will probably also work fine against newer rc versions of 3.13 too). I will definitely figure out a way to share it soon, I promise.

Thanks for the tidbits about the bootloader, guys! The talk of the serial flash chip is what confuses me. I saw a post mentioning it earlier, but I couldn't find the chip. I believe the Insignia Infocast 8" has an almost identical PCB, which I checked out at bunnie's blog:

http://bunniestudios.com/blog/images/in … b_back.jpg

I see the spot near the top middle for the boot SPINOR, but my Chumby 8 does not have that part populated. I also don't see any other boot devices in the "silvermoon_OEM_ref_v3.pdf" schematic. Is there a different flash chip I'm not seeing? Also, when I look at the SD card, it looks like it's structured according to what the PXA168 expects to find when booting directly from an SD card. That's good to know about the silicon bug issue with the SD boot, thanks. Unless there's another flash chip somewhere on a later revision of the Chumby 8 PCB, I'm puzzled about where the flash chip might be though...

  • Duane
  • Duane
  • Blue Octy
  • Offline
  • From: San Diego, CA
  • Registered: 2006-08-24
  • Posts: 8,211

Re: Newer kernel for silvermoon

It's possible this bug was only in certain revs of the PXA168 silicon - the Insignia Infocast 8 was released before the Chumby 8, so it may have been addressed somewhere along the line, either before or during the C8 product lifespan.

It's been several years now - I don't remember all the details.

Duane's Website

  • Registered: 2012-02-26
  • Posts: 40

Re: Newer kernel for silvermoon

Waitaminute, the C8 boots differently from the Infocast 8? That would certainly explain why mine acted like I'd bricked it when I tried using my Infocast image on it (after stupidly wiping the card that it came with)!

Sorry to get a little off-topic here, but: Where can I get a "clean" C8 image from?

  • Duane
  • Duane
  • Blue Octy
  • Offline
  • From: San Diego, CA
  • Registered: 2006-08-24
  • Posts: 8,211

Re: Newer kernel for silvermoon

Yes, the C8 and I8 are not identical.  I'm not sure I have a raw C8 SD image posted anywhere - if someone can make one, I can put it up on files.

Duane's Website

  • Registered: 2013-11-30
  • Posts: 36

Re: Newer kernel for silvermoon

I would definitely be willing to post an image of my card. Isn't it true that the main big partition that ends up mounted at /mnt/root (I think) can be omitted because it is automatically recreated if it's missing?

  • From: Oceanside, CA
  • Registered: 2008-04-17
  • Posts: 650

Re: Newer kernel for silvermoon

I finally purchased a C8 that I've never booted... if nobody else gets something working, let me know what partitions are needed, or any other details and I can likely get a C8 image that you can post.

Linux Guy - Occasional Chumby Hacker

Materdaddy's Website

  • Registered: 2012-02-26
  • Posts: 40

Re: Newer kernel for silvermoon

An image of the whole card would be awesome, but the most important parts are going to be the partition table and the first, small partition (assuming the layout is similar to the I8).

  • From: Dallas, Tx
  • Registered: 2011-01-08
  • Posts: 1,262

Re: Newer kernel for silvermoon

I've uploaded a dd of the sd card for a c8 that I'm tinkering with trying to get the screen to behave on. it's at:

http://files.chumby.com/resources/chumb … .2.img.zip 266M

created with dd if=/dev/sdc of=rom-silvermoon-1.8.2.img bs=1024k, so presumably, the reverse should give you a duplicate.

lemme know if you run into any problem with it.

Cleaning up any loose bits and bytes.
  • Registered: 2013-11-30
  • Posts: 36

Re: Newer kernel for silvermoon

Wow, thank you diamaunt for posting an SD card image! I have something to share too. Here is my initial posting of the patches so far:

https://github.com/dougg3/chumby8-linux-patches

If you patch a stock linux 3.13-rc3 against these patches, it should work. I know this is just the kernel and I haven't provided any u-boot instructions. I'll try to get to that soon. For anyone feeling like screwing around, you have to run the "setid" command in u-boot to make sure that Linux gets the correct new-style processor ID. Then you can load the compiled kernel zImage from another partition on the SD card using the "ext2load" command. Here's the command I use in u-boot:

setid
mmc rescan 0
setenv bootargs root=/dev/nfs nfsroot=192.168.1.155:/chumbyroot ip=192.168.1.32:192.168.1.155:192.168.1.1:255.255.255.0:::off console=ttyS0,115200 mem=128M uart_dma init=/linuxrc sysrq_always_enabled ro rootwait earlyprintk
ext2load mmc 0:2 ${default_load_addr} /boot/vmlinuz.new
bootz ${default_load_addr}

That is loading from an NFS root filesystem, it can be simplified if you're booting from a root filesystem on your SD card. It probably won't work with the standard Chumby root filesystem, not sure on that. I have my own simple rootfs with some simple busybox stuff. Maybe I'll create a simple rootfs to share too, but for now I'm mainly just focusing on getting the kernel working.

I'm not sure if some of the stuff like uart_dma and sysrq_always_enabled matter. For now I'm just copying them from the original chumby boot command. I'm too lazy to look up what they do right now :-)

  • Registered: 2012-02-26
  • Posts: 40

Re: Newer kernel for silvermoon

Awesome. That SD image seems to work, and I'll see about putting together a proper Git repo from those patches. Will also see if I can remember how to trim down the boot partition. (I figured out how to get it down from 9 MB to 256 KB before, but I don't remember what I did anymore...)

Watch this space:

https://github.com/duskwuff/silvermoon-linux

16 Reply by Materdaddy (edited by Materdaddy 2013-12-14 07:59:54)

  • From: Oceanside, CA
  • Registered: 2008-04-17
  • Posts: 650

Re: Newer kernel for silvermoon

I had a linux repo a while ago that contained chumby patches... maybe we can merge them and keep everything together:

https://github.com/Materdaddy/linux-2.6

I'll have to look and see what Linus did for the move to 3.0 and the new tree to see if they can be merged together for complete history.

See here for branches:
https://github.com/Materdaddy/linux-2.6/branches

Linux Guy - Occasional Chumby Hacker

Materdaddy's Website

17 Reply by dusk (edited by dusk 2013-12-14 15:30:11)

  • Registered: 2012-02-26
  • Posts: 40

Re: Newer kernel for silvermoon

dougg3, I've integrated your patches into the "silvermoon" branch:

https://github.com/duskwuff/silvermoon-linux

I've confirmed that this builds and runs on my I8! The SD reader appears to work perfectly, for what it's worth. I'll take a crack at either getting the touchscreen driver ported over, or setting silvermoon up as a new machine type (instead of replacing aspenite).

  • Registered: 2013-11-30
  • Posts: 36

Re: Newer kernel for silvermoon

Awesome dusk, thanks! I was a little worried about the I8 because there are some #ifdefs in the Chumby kernel and I didn't know if the kernel needed to be set up any differently for that case.

I'm thinking my next task will be getting the overlay framebuffer working (usually there's a /dev/fb0 and /dev/fb1, and fb1 is on top of fb0). If you see any weird graphical corruption on the framebuffer, it may be because the overlay buffer is currently pointing out into random memory.

I'd also like to figure out why the SDHC wifi card acts up when debugging output is turned off...

Now that you have the kernel in github, that's great. I'll try my best to do "pull request" type stuff to your project now. Trying to learn how to use git!

19 Reply by dusk (edited by dusk 2013-12-17 00:26:59)

  • Registered: 2012-02-26
  • Posts: 40

Re: Newer kernel for silvermoon

I've pushed a commit to make Silvermoon its own platform (again): https://github.com/duskwuff/silvermoon- … 985aba1f5b

As noted in the commit notes, you'll need to set a different machid now.

I'm taking a detour now to try and get the backlight working properly, ideally WITHOUT hacking the mfp driver to export mfp_config like the Chumby kernel did. (Because that's just kind of gross.) This shouldn't be too tough, but I'm kind of learning about platform devices along the way so I may take a bit to figure it all out.

  • Registered: 2013-11-30
  • Posts: 36

Re: Newer kernel for silvermoon

For anyone else who needs to patch the bootloader to work with the new machine ID in dusk's latest commit, here's what I changed in the bootloader partition of the SD card to make it send the new machid (2222) to the kernel:

At offset 0x1D848, there should be these four bytes (displayed in hex):

25 07 00 00

This is the little endian number 0x00000725, or 1829. To change it to 2222, replace the four bytes with:

AE 08 00 00

If for some reason it's different in your bootloader, just use a hex editor to search for the hex string 25 07 00 00. That was the only instance of that particular combination of bytes in my boot partition.

Note that after doing this, the older Chumby 2.6.28 kernel will no longer boot unless you modify its machid to use 2222 as well. To make the original 2.6.28 kernel use the the new machid, it's probably easiest to just recompile it and change the file arch/arm/tools/mach-types. The new and old IDs are already in there; just comment out the old one and uncomment the new one.

Good call on deciding to use the number from the ARM registry. I was just too lazy because I didn't want to patch u-boot, but now you gave me a good reason to find it and patch it smile

  • Registered: 2012-02-26
  • Posts: 40

Re: Newer kernel for silvermoon

That's one approach! Another one is to add "set machid 8ae" to your bootcmd.

I'm still working on the backlight. pwm_backlight looks like it should be able to take care of this, but I'm still trying to figure out how to get the PWM device to show up and work properly.

22 Reply by dougg3 (edited by dougg3 2013-12-22 16:42:59)

  • Registered: 2013-11-30
  • Posts: 36

Re: Newer kernel for silvermoon

Ah, nice! I was looking for a command but couldn't find one. Thanks for sharing the other (easier) way to do it smile

  • dman
  • New member
  • Offline
  • Registered: 2011-02-25
  • Posts: 4

Re: Newer kernel for silvermoon

Wow, I'm so glad I stumbled across this new thread. I have some time off during the holidays and decided to dust off my Infocast 8 devices and do some hacking. I had a nice debian chroot environment working last year but then other things took priority. Once I get caught up on things I hope to do a write up and share my experiences.

I also stumbled across this thread (from 2012) regarding the boot process:
http://forum.chumby.com/viewtopic.php?id=7793
See post #4 from Bunnie himself.

Also, member 'dusk' made some great progress and shared some useful info in that thread as well.

  • Registered: 2014-07-10
  • Posts: 1

Re: Newer kernel for silvermoon

dougg3 wrote:

It's great to be able to modify the binary, but I want to figure out how to put in a new compiled u-boot instead of hacking the existing binary. The problem is the boot partition is assembled from a bunch of different parts and pieces, and I'm not sure how to fit it all together. I don't even know if the source code or tools to assemble it all are still available. Marvell is no help because it seems you have to sign an NDA just to get access to anything, and I don't think they want to talk to individuals.

Anyway, if that can be figured out, it would open up the door to being able to recompile u-boot. I know how to compile u-boot already, but I don't know how to mix it in with the rest of the various sections in the boot partition. All of the source code that Chumby provided may be useful...aspenix-1.0.tgz on Chumby's source code page looks interesting, for example. I think I found some stuff for OpenEmbedded that may be useful too, but I can't remember.

Hope this helps a bit :-)


Could you tell me how to  recompile u-boot of chumby 8(silvermoon), I see "include $(TOPDIR)/chumby.mk" in the config.mk ,but I can't find the needed file.  thanks!

  • Registered: 2013-11-30
  • Posts: 36

Re: Newer kernel for silvermoon

Naturally, I said in my post that I know how to do it, but I didn't write down directions anywhere. Go figure :-P One of these days I'll learn.

It looks like I applied the patches here:
https://github.com/zdw/chumby-oe/tree/m … oon-200907

I don't have anything more specific than that, sorry!

BTW, I haven't forgotten about this project. My Chumby 8 is still sitting here open. I just have gotten majorly sidetracked. I do still want to get this thing fully working with the mainline kernel eventually. I just have to find the time...