OK, I've got the system booting off a custom image now, and I'm running into some new and exciting problems. The system boots up properly, but starts misbehaving severely a few seconds after everything appears settled. Here's the kernel log:
[ 0.000000] Linux version 2.6.28-M2.1.8.12 (builder@stormbuild) (gcc version 4.3.2 (Sourcery G++ Lite 2008q3-72) ) #1 Thu Dec 9 13:04:27 PST 2010
[ 0.000000] CPU: Marvell Mohawk [56158400] revision 0 (ARMv5TE), cr=04053977
[ 0.000000] CPU: VIVT data cache, VIVT instruction cache
[ 0.000000] Machine: PXA168-based Chumby Silvermoon platform
[ 0.000000] Ignoring unrecognised tag 0x54410008
[ 0.000000] Memory policy: ECC disabled, Data cache writeback
[ 0.000000] Built 1 zonelists in Zone order, mobility grouping off. Total pages: 32512
[ 0.000000] Kernel command line: root=/dev/mmcblk0p3 rootfstype=ext3 rootwait console=ttyS0,115200 mem=128M uart_dma sysrq_always_enabled ro
[ 0.000000] debug: sysrq always enabled.
[ 0.000000] PID hash table entries: 512 (order: 9, 2048 bytes)
[ 0.000088] Console: colour dummy device 80x30
[ 0.000106] ram_console: buffer 00000000, invalid size 0, datasize -12
[ 0.000185] Dentry cache hash table entries: 16384 (order: 4, 65536 bytes)
[ 0.000480] Inode-cache hash table entries: 8192 (order: 3, 32768 bytes)
[ 0.006779] Memory: 128MB = 128MB total
[ 0.006802] Memory: 123904KB available (3852K code, 934K data, 1060K init)
[ 0.006894] SLUB: Genslabs=12, HWalign=32, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
[ 0.006917] Calibrating delay loop... 796.26 BogoMIPS (lpj=3981312)
[ 0.220326] Mount-cache hash table entries: 512
[ 0.220657] CPU: Testing write buffer coherency: ok
[ 0.223463] net_namespace: 292 bytes
[ 0.223663] NET: Registered protocol family 16
[ 0.224156] silvermoon.c - silvermoon_init():701 - Just so you know, the CPU type is 0x56158400
[ 0.224269] pxa168_mfp_set_fastio_drive config changed to 2000
[ 0.224521] silvermoon_init() - attempting to add SSP2 for touchscreen
[ 0.224536] silvermoon_ssp_init() adding ssp2
[ 0.224587] silvermoon_ssp_init() exit
[ 0.224637] silvermoon_init() - adding u2h ehci usb hub
[ 0.224808] silvermoon_init() - adding MMC3 init[2], wifi MMC1
[ 0.225198] platform_driver_register
[ 0.228064] SCSI subsystem initialized
[ 0.228192] usbcore: registered new interface driver usbfs
[ 0.228291] usbcore: registered new interface driver hub
[ 0.228438] usbcore: registered new device driver usb
[ 0.251046] NET: Registered protocol family 2
[ 0.340433] IP route cache hash table entries: 1024 (order: 0, 4096 bytes)
[ 0.340630] TCP established hash table entries: 4096 (order: 3, 32768 bytes)
[ 0.340720] TCP bind hash table entries: 4096 (order: 2, 16384 bytes)
[ 0.340789] TCP: Hash tables configured (established 4096 bind 4096)
[ 0.340805] TCP reno registered
[ 0.370443] NET: Registered protocol family 1
[ 0.381516] Installing knfsd (copyright (C) 1996 okir@monad.swb.de).
[ 0.382606] NTFS driver 2.1.29 [Flags: R/W].
[ 0.383053] fuse init (API version 7.10)
[ 0.383663] msgmni has been set to 242
[ 0.384723] alg: No test for stdrng (krng)
[ 0.384784] io scheduler noop registered
[ 0.384797] io scheduler anticipatory registered
[ 0.384810] io scheduler deadline registered
[ 0.384929] io scheduler cfq registered (default)
[ 0.392852] pxa168fb_ovly.c - pxa168fb_probe():2075 - Just pointed fb0 at 0112b000
[ 0.396325] pxa168fb_set_par() warning: viewport x/y zoom were 0
[ 0.411260] Console: switching to colour frame buffer device 100x75
[ 0.424037] pxa168fb_ovly: frame buffer device was loaded to /dev/fb0 <Ovly-silvermoon>.
[ 0.428864] pxa2xx-uart.0: ttyS0 at MMIO 0xd4017000 (irq = 27) is a FFUART
[ 0.428897] console [ttyS0] enabled
[ 0.747069] pxa2xx-uart.2: ttyS2 at MMIO 0xd4026000 (irq = 29) is a STUART
[ 0.754366] brd: module loaded
[ 0.758669] loop: module loaded
[ 0.761927] logger: created 64K log 'log_main'
[ 0.766424] logger: created 256K log 'log_events'
[ 0.771218] logger: created 64K log 'log_radio'
[ 0.775821] usbcore: registered new interface driver cdc_ether
[ 0.781755] Driver 'sd' needs updating - please use bus_type methods
[ 0.788213] usbmon: debugfs is not available
[ 0.792515] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[ 0.799205] u2h regbase 0xc883a000 phybase 0xc883e000 irq 51
[ 0.806684] pxau2h-ehci pxau2h-ehci: Marvell PXA SOC EHCI Host Controller
[ 0.813679] pxau2h-ehci pxau2h-ehci: new USB bus registered, assigned bus number 1
[ 0.821488] pxau2h-ehci pxau2h-ehci: irq 51, io base 0x0883a000
[ 0.840384] pxau2h-ehci pxau2h-ehci: USB 2.0 started, EHCI 1.00
[ 0.846638] usb usb1: configuration #1 chosen from 1 choice
[ 0.852426] hub 1-0:1.0: USB hub found
[ 0.856208] hub 1-0:1.0: 1 port detected
[ 0.860721] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002
[ 0.867514] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 0.874785] usb usb1: Product: Marvell PXA SOC EHCI Host Controller
[ 0.881083] usb usb1: Manufacturer: Linux 2.6.28-M2.1.8.12 ehci_hcd
[ 0.887346] usb usb1: SerialNumber: pxau2h-ehci
[ 0.891917] [ehci-pxau2h.c] Silvermoon media insertion / removal detection enabled
[ 0.899577] Initializing USB Mass Storage driver...
[ 0.904589] usbcore: registered new interface driver usb-storage
[ 0.910632] USB Mass Storage support registered.
[ 0.915327] usbcore: registered new interface driver libusual
[ 0.921244] u2o regbase 0xc8842000 phybase 0xc8846000 irq 44
[ 0.928018] mice: PS/2 mouse device common for all mice
[ 0.933745] mmp-rtc mmp-rtc: rtc core: registered mmp-rtc as rtc0
[ 0.940143] mmc0: SDHCI controller on MMC [pxa-sdh.0] using ADMA
[ 0.946363] mmc1: SDHCI controller on MMC [pxa-sdh.1] using ADMA
[ 0.952567] mmc2: SDHCI controller on MMC [pxa-sdh.2] using ADMA
[ 0.958621] pxa-sdh: Secure Digital Host Controller Interface driver
[ 0.965014] pxa-sdh: Copyright(c) Pierre Ossman
[ 0.970462] usbcore: registered new interface driver usbhid
[ 0.976042] usbhid: v2.6:USB HID core driver
[ 0.980446] Advanced Linux Sound Architecture Driver Version 1.0.18rc3.
[ 0.987063] ALSA device list:
[ 0.990027] No soundcards found.
[ 0.993457] oprofile: using timer interrupt.
[ 0.998071] TCP cubic registered
[ 1.001348] NET: Registered protocol family 17
[ 1.006411] RPC: Registered udp transport module.
[ 1.011161] RPC: Registered tcp transport module.
[ 1.015863] XScale iWMMXt coprocessor detected.
[ 1.020824] pxa168fb.c - pxa168fb_probe():1268 - Changing LCD_SPU_DMA_CTRL1 from 0x20020081 -> 0x2002ff81
[ 1.042912] pxa168fb.c - pxa168fb_probe():1328 - Changing LCD_SPU_DMA_CTRL0 from 0x08401111 -> 0x08441111
[ 1.052520] pxa168fb.c - pxa168fb_probe():1331 - Changing LCD_SPU_DMA_CTRL1 from 0x2002ff81 -> 0x2002ff81
[ 1.062335] pxa168fb: frame buffer device was loaded to /dev/fb1 <Base-silvermoon>.
[ 1.069994] silvermoon_lcd_power() ON
[ 1.073837] mmp-rtc mmp-rtc: setting system clock to 1970-01-01 00:00:00 UTC (0)
[ 1.081448] Waiting for root device /dev/mmcblk0p3...
[ 1.275752] usb 1-1: new high speed USB device using pxau2h-ehci and address 2
[ 1.681514] usb 1-1: configuration #1 chosen from 1 choice
[ 1.731239] hub 1-1:1.0: USB hub found
[ 1.779046] hub 1-1:1.0: 4 ports detected
[ 1.872137] usb 1-1: New USB device found, idVendor=058f, idProduct=6254
[ 1.878843] usb 1-1: New USB device strings: Mfr=0, Product=1, SerialNumber=0
[ 1.886028] usb 1-1: Product: USB2.0Hub
[ 2.465019] usb 1-1.3: new high speed USB device using pxau2h-ehci and address 3
[ 3.003575] usb 1-1.3: configuration #1 chosen from 1 choice
[ 3.054193] usb 1-1.3: New USB device found, idVendor=0846, idProduct=1040
[ 3.061117] usb 1-1.3: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[ 3.068425] usb 1-1.3: Product: NETGEAR FA120 Adapter
[ 3.073546] usb 1-1.3: Manufacturer: NETGEAR
[ 3.299017] usb 1-1.4: new high speed USB device using pxau2h-ehci and address 4
[ 3.881826] usb 1-1.4: configuration #1 chosen from 1 choice
[ 3.932895] mmc1: new SDIO card at address 0001
[ 3.937753] scsi0 : SCSI emulation for USB Mass Storage devices
[ 3.944745] usb 1-1.4: New USB device found, idVendor=058f, idProduct=6366
[ 3.951734] usb 1-1.4: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 3.959049] usb 1-1.4: Product: Mass Storage Device
[ 3.963958] usb 1-1.4: Manufacturer: Generic
[ 3.968222] usb 1-1.4: SerialNumber: 058F0O1111B1
[ 4.944017] mmc2: new SD card at address d3ba
[ 4.948700] mmcblk0: mmc2:d3ba SU01G 942 MiB
[ 4.953263] mmcblk0: p1 p2 p3 p4
[ 5.055612] kjournald starting. Commit interval 5 seconds
[ 5.061185] EXT3-fs: mounted filesystem with ordered data mode.
[ 5.067138] VFS: Mounted root (ext3 filesystem) readonly.
[ 5.072854] Freeing init memory: 1060K
<system boots up...>
[ 8.894964] Internal error: Oops: 0 [#1]
[ 8.898885] Modules linked in: asix
[ 8.902370] CPU: 0 Not tainted (2.6.28-M2.1.8.12 #1)
[ 8.907680] PC is at 0x0
[ 8.910215] LR is at enqueue_task+0x1c/0x28
[ 8.914404] pc : [<00000000>] lr : [<c0120598>] psr: 20000093
[ 8.914415] sp : c7fdfeb8 ip : c7fdfed0 fp : c7fdfecc
[ 8.925892] r10: 00000000 r9 : 00000003 r8 : c04d5d90
[ 8.931119] r7 : 00000000 r6 : c04d78d8 r5 : c0500e60 r4 : c04d78d8
[ 8.937647] r3 : c03ecac8 r2 : 00000001 r1 : c04d78d8 r0 : c0500e60
[ 8.944175] Flags: nzCv IRQs off FIQs on Mode SVC_32 ISA ARM Segment kernel
[ 8.951571] Control: 0405397f Table: 07c3c018 DAC: 00000017
[ 8.957315] Process khelper (pid: 6, stack limit = 0xc7fde270)
[ 8.963149] Stack: (0xc7fdfeb8 to 0xc7fe0000)
[ 8.967504] fea0: c0500e60 c0500e60
[ 8.975760] fec0: c7fdfee4 c7fdfed0 c0120690 c012058c 00000001 c0500e60 c7fdff0c c7fdfee8
[ 8.984015] fee0: c0121a74 c0120678 c7fdff6c 20000093 c04d5d3c c04d5d84 00000001 00000001
[ 8.992270] ff00: c7fdff1c c7fdff10 c0121ad8 c0121a1c c7fdff4c c7fdff20 c0120c20 c0121ad4
[ 9.000525] ff20: 00000000 60000013 c7c9c240 c0135164 c7ff40a0 00000000 00000000 00000000
[ 9.008780] ff40: c7fdff6c c7fdff50 c0120d4c c0120bf8 00000000 c0135164 c7ff40a0 00000000
[ 9.017036] ff60: c7fdff84 c7fdff70 c01351c0 c0120d1c c7ff40a0 c7fde000 c7fdffa4 c7fdff88
[ 9.025291] ff80: c0135740 c0135174 c7ff40a8 c7fdffb8 c7fccfa0 c7ff40a0 c7fdffdc c7fdffa8
[ 9.033546] ffa0: c0135e10 c0135698 c7fdffdc 00000000 c7fccfa0 c0139644 c7fdffb8 c7fdffb8
[ 9.041801] ffc0: c7ff40a0 c0135d64 00000000 00000000 c7fdfff4 c7fdffe0 c01391e0 c0135d74
[ 9.050056] ffe0: 00000000 00000000 00000000 c7fdfff8 c0128834 c013919c 00000000 00000000
[ 9.058311] Backtrace:
[ 9.060753] [<c012057c>] (enqueue_task+0x0/0x28) from [<c0120690>] (activate_task+0x28/0x34)
[ 9.069207] r5:c0500e60 r4:c0500e60
[ 9.072773] [<c0120668>] (activate_task+0x0/0x34) from [<c0121a74>] (try_to_wake_up+0x68/0xb8)
[ 9.081403] r5:c0500e60 r4:00000001
[ 9.084969] [<c0121a0c>] (try_to_wake_up+0x0/0xb8) from [<c0121ad8>] (default_wake_function+0x14/0x18)
[ 9.094294] r7:00000001 r6:00000001 r5:c04d5d84 r4:c04d5d3c
[ 9.099952] [<c0121ac4>] (default_wake_function+0x0/0x18) from [<c0120c20>] (__wake_up_common+0x38/0x7c)
[ 9.109449] [<c0120be8>] (__wake_up_common+0x0/0x7c) from [<c0120d4c>] (complete+0x40/0x4c)
[ 9.117818] [<c0120d0c>] (complete+0x0/0x4c) from [<c01351c0>] (__call_usermodehelper+0x5c/0x70)
[ 9.126628] r4:00000000
[ 9.129154] [<c0135164>] (__call_usermodehelper+0x0/0x70) from [<c0135740>] (run_workqueue+0xb8/0x144)
[ 9.138477] r5:c7fde000 r4:c7ff40a0
[ 9.142053] [<c0135688>] (run_workqueue+0x0/0x144) from [<c0135e10>] (worker_thread+0xac/0xc0)
[ 9.150673] r7:c7ff40a0 r6:c7fccfa0 r5:c7fdffb8 r4:c7ff40a8
[ 9.156342] [<c0135d64>] (worker_thread+0x0/0xc0) from [<c01391e0>] (kthread+0x54/0x80)
[ 9.164362] r7:00000000 r6:00000000 r5:c0135d64 r4:c7ff40a0
[ 9.170030] [<c013918c>] (kthread+0x0/0x80) from [<c0128834>] (do_exit+0x0/0x6bc)
[ 9.177530] r5:00000000 r4:00000000
[ 9.181097] Code: bad PC value.
[ 9.184507] ---[ end trace 2509e2451b9817e6 ]---
[ 9.198853] bad: scheduling from the idle thread!
[ 9.203638] [<c03e9840>] (dump_stack+0x0/0x14) from [<c01225fc>] (dequeue_task_idle+0x24/0x38)
[ 9.212325] [<c01225d8>] (dequeue_task_idle+0x0/0x38) from [<c0120620>] (dequeue_task+0x7c/0x88)
[ 9.221170] [<c01205a4>] (dequeue_task+0x0/0x88) from [<c01206c4>] (deactivate_task+0x28/0x38)
[ 9.229805] r7:c04d8560 r6:c04d78d8 r5:c04d7a48 r4:c0500e60
[ 9.235498] [<c012069c>] (deactivate_task+0x0/0x38) from [<c03e9ea0>] (schedule+0x10c/0x328)
[ 9.244024] r5:c04d7a48 r4:c01133bc
[ 9.247626] [<c03e9d94>] (schedule+0x0/0x328) from [<c01133b0>] (cpu_idle+0x50/0x5c)
[ 9.255420] [<c0113360>] (cpu_idle+0x0/0x5c) from [<c03e86e0>] (rest_init+0x58/0x6c)
[ 9.263208] r5:c04ff9cc r4:c0544d60
[ 9.266783] [<c03e8688>] (rest_init+0x0/0x6c) from [<c0008b10>] (start_kernel+0x280/0x2d8)
[ 9.275091] [<c0008890>] (start_kernel+0x0/0x2d8) from [<00008034>] (0x8034)
[ 9.284743] bad: scheduling from the idle thread!
[ 9.289459] [<c03e9840>] (dump_stack+0x0/0x14) from [<c01225fc>] (dequeue_task_idle+0x24/0x38)
[ 9.298156] [<c01225d8>] (dequeue_task_idle+0x0/0x38) from [<c0120620>] (dequeue_task+0x7c/0x88)
[ 9.306993] [<c01205a4>] (dequeue_task+0x0/0x88) from [<c01206c4>] (deactivate_task+0x28/0x38)
[ 9.315651] r7:c0724000 r6:c04d78d8 r5:c04d7a48 r4:c0500e60
[ 9.321358] [<c012069c>] (deactivate_task+0x0/0x38) from [<c03e9ea0>] (schedule+0x10c/0x328)
[ 9.329814] r5:c04d7a48 r4:c0102300
[ 9.333416] [<c03e9d94>] (schedule+0x0/0x328) from [<c01133b0>] (cpu_idle+0x50/0x5c)
[ 9.341204] [<c0113360>] (cpu_idle+0x0/0x5c) from [<c03e86e0>] (rest_init+0x58/0x6c)
[ 9.348972] r5:c04ff9cc r4:c0544d60
[ 9.352573] [<c03e8688>] (rest_init+0x0/0x6c) from [<c0008b10>] (start_kernel+0x280/0x2d8)
It goes on line this indefinitely... behind the constant spew on the serial console, the system is running, but performs poorly. The issue is intermittent; sometimes it won't happen, booting up with init=/bin/sh doesn't trigger it, but subsequently exec'ing /sbin/init does unless you wait a few seconds, which has me wondering if it's some sort of hardware race condition...
Any ideas?