1 (edited by BarryMead 2011-04-05 16:32:55)

Topic: Mismanaged Source Tree

It is really too bad that the Chumby Source Tree was never properly maintained when the Hacker Board was created.  The Chumby Hacker board is competitively priced, and could have been a serious option for embedded systems engineers, if chumby had treated it as more than just a toy.  It's hardware is superior to many other commercial embedded systems, but doesn't have the professional level of source code management required to promote reliable commercial use.  I had hoped that it would be reliable with the existing drivers, and included modules, but such is not the case.  I can run the same program 4 times, and sometimes it will run perfectly, and other times it will crash with unfixable errors in the USB serial port, drivers.   Over and Over I keep running into the same problem.  The source code on the published web site does not match up with the version of the code in the binary rom falconwing image.  This makes it essentially impossible to correct problems, and maintain a reliable code base.  If the developers of the Chumby Hacker Board would simply use any of the many available Version Control systems (CVS, git, Subversion, mercurial) to create stable releases of the full source tree to match each binary code version, then none of these problems would be plaguing the Chumby Community.  Most of the complaints, and problem reports on this forum can be traced back to this single bad managerial decision.  Now that I realize the scope of the codebase inconsistencies,  I have to scrap my entire investment in Chumby, and switch to a more reliable, and professional embedded system platform.  Such a shame!  I really fell in love with the Chumby hardware, and it's price performance matrix is better than any other embedded platform for my needs.  But without supported, reliable maintainable software it might as well be made of stone.

If you can locate all of the original source code team, and all of the original code fragments, I can show you how to set up a
version tracking repository that will solve all of these issues, and would be willing to donate some time to help Chumby fix this problem for good.  But if there is no interest from management on down, then there is no way to save the Chumby Hacker Board from a doomed fate of neglected support and obscurity.

Re: Mismanaged Source Tree

Hi BarryMead,

Sorry about the frustration you've encountered with the chumby hacker board. When we created the board, it was in response to the demand by makers, tinkerers and hobbyists to get a platform to play with in an arduino-like fashion. Thus, the focus was more on rapid usability and weekend projects, so the board shipped with a compiler pre-installed so you could do small projects very quickly. However, a few people have picked it up and have tried to do sophisticated embedded development and ran into the source management problems you have cited. Part of the problem is that chumby is tuned for use as a consumer product, so it comes with a fairly hefty pre-load of software that is largely irrelevant to developers, or inaccessible (such as Flash -- we can only distribute in binary form, and only within a final product, not in developer images), and these get in the way of what you're trying to do.

In response to this, we're creating an alternate, developer-friendly image, the "chumby starter image". This is a stripped-down build that is reliable, simple, and extendable. It doesn't do much on its own, but the build environment comes with package management tools to allow you to rapidly add exactly what you want. We are using OpenEmbedded as the build engine, you can download instructions here for setting it up yourself:

http://wiki.chumby.com/index.php/Buildi … %28Beta%29

This uses git to manage the source code. Again, what you get in the starter image is *not* a chumby as you may be familiar with in a consumer product, but a geared-for-embedded-development core BSP.

Furthermore, we are planning on releasing some combination of an AMI (EC2 cloud image) and/or a VMware image that you can download. Since the initial build and checkout takes several hours, and can be finicky depending on what release of linux you're building from, these "pre-built" images will allow you to very rapidly start from a "sane" base and extend/add your apps without a lot of waiting and frustration. As each build takes a few hours to run, it may be a few days before these are validated and available, but please check the wiki page referenced above for updates.

Sorry about your frustration, and hopefully this helps clean things up a bit.

7BAA 2E53 01C1 DCFF 497B  E7F0 9699 A303 78F0 D9B9

Re: Mismanaged Source Tree

This response is better than I could have ever dreamed of.  I don't know how to to express enough thanks for your timely and professional handling of this matter.  Wow!!!!

Re: Mismanaged Source Tree

Thank you Bunnie!  A couple of years ago I found some posts about OE and did some builds, but was not satisfied with the results and never looked into OE much more. I'll play with OE again over the next couple weeks.

Linux Guy - Occasional Chumby Hacker

Re: Mismanaged Source Tree

Will the "chumby starter image" be available as an sd card image for download?
-dan

Re: Mismanaged Source Tree

There should be a link to an SD card image generated using OE.  It's rather sparse, but it's an example of an SD image built entirely using the available OE overlay.

The image is at the bottom of the Writing images to disk section.

Re: Mismanaged Source Tree

Instructions on how to fire up a pre-built OE instance are now posted.

http://wiki.chumby.com/index.php/Quickstarting_OE

You will need to create an amazon EC2 account to do this, but it may save you some headache!

It's a bit hard to fully vet the instructions because I don't have a second EC2 account to test this on, so if someone does find issues, please do let me know.

7BAA 2E53 01C1 DCFF 497B  E7F0 9699 A303 78F0 D9B9

Re: Mismanaged Source Tree

For those who do not want to create an EC2 account and would rather download a VMware image, you can grab one at http://files.chumby.com/hacks/openembedded.zip

The file is about 8GB in size.

7BAA 2E53 01C1 DCFF 497B  E7F0 9699 A303 78F0 D9B9

Re: Mismanaged Source Tree

bunnie wrote:

For those who do not want to create an EC2 account and would rather download a VMware image, you can grab one at http://files.chumby.com/hacks/openembedded.zip

The file is about 8GB in size.

Do you have login information for this image?

Re: Mismanaged Source Tree

Try username: user, and password: user

Cheers,
Matt

Re: Mismanaged Source Tree

bennettm wrote:

Try username: user, and password: user

Cheers,
Matt

Jep, that's it. Thanks!

12 (edited by BarryMead 2011-04-29 08:42:51)

Re: Mismanaged Source Tree

I downloaded the openembedded.zip file unzipped it and tried to open the generated directory with the free vmware player on my debian AMD-64 linux machine with the linux-64-bit vmware player program.   That proved unsuccessful, as it claimed that "openembedded.vmx" is not a valid virtual machine configuration file.  If the free vmware player is not the correct VMware product to utilize this zip file does anyone know which one is.  When I looked at VMware's web site there were dozens of VMware products and I couldn't tell from the descriptions which one would be the best candidate to utilize this file. 
I suspect that the zip file is invalid, because when I unzip it most of the internal files have (0) ZERO length.  I tried unzipping it in linux and the linux unzip utility complained that the file was invalid.  When I unzipped the file in Windows7 it unzipped, but most of files have zero length.  I will try downloading the zip file again to see if I get the same result.

I was able to compile the chumby starter image, and the task-sdk-native image with the other instructions on an ubuntu 10.04 32-bit machine, however.  I built several packages and recreated the full gcc toolchain and libraries on the target platform with the generated ipk packages.  I have almost everything working that the original chumby falconwing image had plus full source code modification capability.   I still haven't figured out how to build specialized device driver modules for the kernel, or found a way to leave the kernel source around after the bitbake process to permit "make modules" commands to operate.  If anyone has any suggestions on this issue, please feel free to chime in here.  I also don't know how to instruct bitbake to make an "ON-PLATFORM" linux kernel source ipk package, and I don't know if the existing on-platform image created for the falconwing system is completly compatible with the angstrom build tree, so I haven't tried to use that tar.gz file on the angstrom platform.  I am reading the bitbake manual and beginning to make sense of the complex recipe synthesis process.  If anyone knows how to create a local recipe for the rtl8192cu wifi driver or the pl-2303 usb-serial driver I would greatly appreciate hearing from you also.  Thanks Everyone (especially Bunny) for keeping the OPEN in open hardware/software ALIVE!

Re: Mismanaged Source Tree

I didn't boot the VM yet with vmware player, but it extracted fine.  The zip I downloaded has md5sum c708f2216ddb88063f420c0ad5a2878f.  I suspect something happened with your download.

Linux Guy - Occasional Chumby Hacker

14 (edited by BarryMead 2011-04-29 17:40:15)

Re: Mismanaged Source Tree

Materdaddy:  When you extracted your zip file did it contain 15 zero length files and only 5 files with actual contents in them?
I downloaded the zip file twice and that is what I found.  I couldn't even unzip it in linux, but it unzipped in windows.
The md5sum of my file is c708f2216ddb88063f420c0ad5a2878f which is identical to yours.    I tried to open the unzipped file with both the linux 64 version of VMware player and with the Windows 7 version of VMware player, and got the same result both times VMware player says that the "openembedded.vmx is not a valid virtual machine configuration file".  Which is not surprising considering that it is one of the files with zero length.  I suspect that the zip file was built incorrectly or from a
bad virtual machine image in the first place.  I would be very surprised if anyone has gotten this file to open with any VMware
product since most of the files in the zip archive are empty (zero length).  Why would the files be in the zip file if they didn't contain any data?  There must be an error in the process of how the zip file was made.  Either a permission error or some other mistake allowed the file names to be added to the zip archive, but not the file contents.  (That would be my guess).

Re: Mismanaged Source Tree

What version of linux are you using?  With md5sum c708f2216ddb88063f420c0ad5a2878f on the zip, I can extract files with unzip version 6.00 from the "unzip" package in ubuntu which is InfoZIP.

I end up with the following md5sums/files/sizes:

edf7cb60638554b79966dbf6d7d86382 8.5K openembedded.vmwarevm/openembedded.nvram
4675728abad7835bc100ff4feea53510 1.7G openembedded.vmwarevm/openembedded-s001.vmdk
e3984c159f3165d7e9d939185bbe4ae3 2.0G openembedded.vmwarevm/openembedded-s002.vmdk
3418ee5d9ef48cfbcb9be0ecdfb9b93c 2.0G openembedded.vmwarevm/openembedded-s003.vmdk
40c055c675fec9b1cd3617f94dd804fc 2.0G openembedded.vmwarevm/openembedded-s004.vmdk
c54c18f99afc0f7fbac9d0d4ac507a4e 2.0G openembedded.vmwarevm/openembedded-s005.vmdk
a81aef1a035019e57b7acb593d484819 2.0G openembedded.vmwarevm/openembedded-s006.vmdk
9319596f5ae6b5b1f5aae7467dd2c83e 1.9G openembedded.vmwarevm/openembedded-s007.vmdk
9203242cd0d4658b9765819b7be7c67c 1.9G openembedded.vmwarevm/openembedded-s008.vmdk
a5a6a971a1aa67c74e2ded67c0b507af 1.9G openembedded.vmwarevm/openembedded-s009.vmdk
0e8128a6efe60b573e8a95f6e37a43c2 754M openembedded.vmwarevm/openembedded-s010.vmdk
09ed65ca603b0e91225ac31cc8ddee43 64K openembedded.vmwarevm/openembedded-s011.vmdk
8f6b09e0ec9cb6d8f03a4d43eae253cd 944 openembedded.vmwarevm/openembedded.vmdk
d41d8cd98f00b204e9800998ecf8427e 0 openembedded.vmwarevm/openembedded.vmsd
764af451cb5a0c6642201d00450dd23e 2.9K openembedded.vmwarevm/openembedded.vmx
01f7dcd80444eec0dba03abf54a75e1c 267 openembedded.vmwarevm/openembedded.vmxf
7f5ff3e5c8f1630cdce66dd3454f37f5 512 openembedded.vmwarevm/openembedded.vmx.lck/M32559.lck
d41d8cd98f00b204e9800998ecf8427e 0 openembedded.vmwarevm/quicklook-cache.png
fb8524899202b3f0361d8d0bf006673e 237K openembedded.vmwarevm/vmware-0.log
11fb77b07e95ebd3767a5c63e94f3f3c 162K openembedded.vmwarevm/vmware-1.log
fcd40dc035139444f3c5b82e13873f8c 248K openembedded.vmwarevm/vmware.log

I'm downloading vmware player now to see if I can boot the image.

Linux Guy - Occasional Chumby Hacker

Re: Mismanaged Source Tree

Boots fine.  I reconfigured some settings for the VM image, and vmware prompted me on a few others.
* I lowered the RAM & CPUs - I can't afford a "bunnie" machine wink
* The VM was being used - lock files must be present
* It detected the image had been moved or copied - some configuration paths changed
* An IDE device the VM was configured with wasn't present - my system is different than bunnie's
* My host kernel didn't have a kernel parameter set that made multi-core machines run slowly

Once booted, I had to change the network interface.  The VM was configured to get dhcp on eth0 and for me eth1 was the vmware bridge device, there was no eth0.

Other than that, things look to be working.  Again, since your zip has the same md5sum, I suspect your problem has nothing to do with the VM, or your download, but the tools you're using to extract it.

File tells me it's a zip that needs at least "1.0 to extract".  Maybe you have a zip that doesn't understand a newer zip compression algorithm?  Maybe you're using samba or FAT32 or something crappy that doesn't handle anything bigger than a 4Gb file properly?

Linux Guy - Occasional Chumby Hacker

17 (edited by BarryMead 2011-05-01 00:17:22)

Re: Mismanaged Source Tree

Cool:   I am glad it works,  I will need to set up a new computer with a bigger hard drive to be able to try it out with the newer ubuntu unzip program.  WHICH VERSION OF UBUNTU are you using 10.04 (Lucid Lynx), 10.10 (Maverick Meercat), or 11.04 (Natty Narwhal)?   Tomorrow, I will move a 200G hard drive over to an older 32-bit machine and load up one of the ubuntu versions.  I was using the unzip program from Debian Lenny version 5.06  (unzip version 5.52), and It couldn't handle the zip file (claimed it was an invalid file and wouldn't unzip it at all).  When windows 7 tries to unzip it it gives the 15 empty files, so I guess it just needed the newer unzip to work.  What kind of (memory/cpu count) did Bunnie's machine have?  The filesystem I am using is reiserfs, and it handles huge files just fine.  It was the unzip program that was the problem.  The funny thing is that the big files were the ones that were not zero length. Only the smaller files had the problem.  I will plan on installing Ubuntu 10.04 unless you get back to me and tell me that you are using a newer one on your machine.  Thanks,  Barry

Re: Mismanaged Source Tree

MaterDaddy:  I found the 6.0 version of Info zip for Windows and put it on my big machine and unzipped the file with that program and it worked just fine.  Thanks for your accurate reporting of the unzip version that put me on the right track without having to build up another machine and install ubuntu on it.  Barry

Re: Mismanaged Source Tree

Materdaddy:  I got the virtual machine running, but as soon as I allowed it to load the vmware tools for linux, the network interface eth0 inside the virtual machine vanished, and I haven't been able to get it back since.  I read dozens of forum posts in the vmware forums, and lots of people are having the same or similar problems, but none of the solutions that they propose work in my case.   Without network connectivity from inside of the virtual machine, the bit bake stuff won't work, because it reaches out all over the web for files.   I was hoping to be able to run this stuff on my fastest machine with the vmware image, but I guess I am stuck running it on a slow spare computer.  I can't really complain, because I have a working solution, (except for being able to compile the kernel modules), but I believe that solution will soon present itself, because I know at least someone has already done it.  I just have to find one of those who has and pick their brain a little.

Re: Mismanaged Source Tree

BarryMead wrote:

Materdaddy:  I got the virtual machine running, but as soon as I allowed it to load the vmware tools for linux, the network interface eth0 inside the virtual machine vanished, and I haven't been able to get it back since.  I read dozens of forum posts in the vmware forums, and lots of people are having the same or similar problems, but none of the solutions that they propose work in my case.   Without network connectivity from inside of the virtual machine, the bit bake stuff won't work, because it reaches out all over the web for files.   I was hoping to be able to run this stuff on my fastest machine with the vmware image, but I guess I am stuck running it on a slow spare computer.  I can't really complain, because I have a working solution, (except for being able to compile the kernel modules), but I believe that solution will soon present itself, because I know at least someone has already done it.  I just have to find one of those who has and pick their brain a little.

Glad to hear you got it to unzip.  For the missing network interface, more than likely your network interface is no longer eth0 and is now eth1.  What do you get with "ifconfig -a"?  If that shows you an eth1 that isn't up, simply edit /etc/network/interfaces and change "eth0" with "eth1" in the 2 places it exists.  Then restart network (/etc/init.d/networking restart)

Linux Guy - Occasional Chumby Hacker

21 (edited by BarryMead 2011-05-01 13:08:14)

Re: Mismanaged Source Tree

Materdaddy:  Thanks,  that fixed it and was much simpler than all of the other suggestions that I gleaned out of the hundreds of vmware forum postings I read about the loss of internet inside the virtual machine.  Now I get to see if my fast machine can really speed up the bitbake process.  My quad core AMD 3.4Ghz machine with 8G RAM, and SataIII 1-Terabyte hard drive really made short work out of running the virtual ubuntu bitbake.  It ran the "bitbake task-sdk-native" in less than a half hour, where it took nearly 5 hours to run on my old celeron 3.2 Ghz 32-bit single core machine.  Both were run after a successful run of the basic bitbake process "bitbake chumby-starter-image", so the task-sdk-native build didn't have to download the whole world, only the files that were extra for making the task-sdk-native stuff.

Thanks for helping me get this option working.  Gratefully,  Barry

Re: Mismanaged Source Tree

No problem, glad you got it up and running and everything's going much faster for you.

Linux Guy - Occasional Chumby Hacker

Re: Mismanaged Source Tree

BarryMead and MaterDaddy, which VMware tool are you using to run openembedded.vmx ?

When I add openembedded.vmx to my VMware inventory, it crashes the VMware Host Agent service.
Even after changing openembedded.vmx parameters to memsize = "512", numvcpus = "2" and deleting the shared drive references, and deleting the .lck directory.

I'm using  VMware Infrastructure Web Access Version 2.0.0 Build 128374 with VMware Server Version 2.0.2 Build 203138 on Win XP.

Re: Mismanaged Source Tree

VMware Player 3.1.4 build-385536 on Ubuntu 10.10

Linux Guy - Occasional Chumby Hacker

Re: Mismanaged Source Tree

Materdaddy wrote:

VMware Player 3.1.4 build-385536 on Ubuntu 10.10

Much Thanks...up & running quickly.