Topic: using openembedded and opkg

I've installed openembedded on my development system and built a chumby starter image as per the instructions posted by bunnie (many thanks btw). I've also built various SDK's and lots of packages, and copied the boot image to a uSD card, along with all the packages to another partition on the same card. I can plug the card into my chumby hacker board and it boots up just fine. I've mounted the partition with all the packages that I want to install and now I've run into a problem.

There doesn't seem to be a simple way to use opkg to install packages straight from a directory. If they were organised in a package cache, it could be done with the --cache <directory> option, but naively applying this to various points in the directory structures created by bitbake doesn't seem to work, and no amount of searching for documentation on the internet has turned up any hints either.

So, I've downloaded the source code from opkg from http://code.google.com/p/opkg/ and after I've caught up on sleep I'm going to start reading it for clues. However I'm hoping that some kind soul will be able to point out a simple solution before I resort to doing this.

Does anyone know how to create an opkg cache without a network connection?

Re: using openembedded and opkg

Well I think I've got the next step figured out. Looking through the source code for opkg it seemed that it supports local repositories, similar to debian. I replaced all the files in /etc/opkg with a single opkg.conf file containing the following lines:

##########
dest root /
lists_dir ext /var/lib/opkg
arch all 1
arch any 6
arch noarch 11
arch arm 16
arch armv4 21
arch armv4t 26
arch armv5te 31
arch chumby-falconwing 36

src/gz base file:/media/ipk/armv5te
src/gz chumby-falconwing file:/media/ipk/chumby-falconwing
src/gz no-arch file:/media/ipk/all

option tmp_dir /var/lib/opkg/tmp
##########

With the partition containing the subdirectories of output-angstrom-.9/deploy/eglibc/ipk mounted on /media/ipk I can now issue commands like 'opkg update' and 'opkg install gcc' and get something to happen with no cache option required.

Onwards and upwards...

3 (edited by infurl 2011-04-21 13:56:13)

Re: using openembedded and opkg

Now I'm happy to report that networking is working. I bought one of those nifty little ultra-mini-nano-usb-wifi adaptors from DealExtreme http://goo.gl/fCVHp downloaded the 8192cu.so driver for it from http://goo.gl/b1lWn and installed it as per the instructions on Lady Ada's website http://goo.gl/9aRYQ as follows:

opkg install task-proper-tools
install -p -m 644 8192cu.ko /lib/modules/2.6.28-chumby/kernel/drivers/net/
depmod -a 2.6.28-chumby

The chumby-starter-image doesn't actually have the install utility installed, nor is it included in its instance of busybox, hence it is necessary to install task-proper-tools first to get it. If you need to use iwlist you will also need to install wireless-tools.

To get the network going I edited /etc/network/interfaces to contain the following:

auto lo wlan0
iface lo inet loopback
iface wlan0 inet dhcp
    wpa-driver wext
    wpa-conf /etc/wpa-supplicant.conf

And I then created /etc/wpa-supplicant.conf with:

network={
    ssid="office"
    scan_ssid=1
    key_mgmt=WPA-PSK
    psk="pass-phrase-in-quotes" or hexadecimal-number-without-quotes
}

The final steps to conceal the secrets in the configuration file and bring up the network are:

chmod 0600 /etc/wpa-supplicant.conf
opkg install wpa-supplicant
reboot

4 (edited by BarryMead 2011-04-23 20:11:04)

Re: using openembedded and opkg

infurl You wrote "I've also built various SDK's and lots of packages"  Did these packages build automatically following bunny's instructions, or did you have to take additional steps to build these packages?  I am interested in building up the whole toolchain gcc glibc standard libraries, VIM, gdb and all the stuff needed to do standard C programming and compilation on the platform itself.  If you have achieved that goal, could you share a little more about how you accomplished that step.

I will tell you where I am at currently.  I followed the instructions that bunny provided and ran the bitbake process (took almost 24 hours to run)  Then it built a successfully bootable angstrom rom image that I checked in the chumby hacker board.  It also made many ipk packages in the chumby-oe/output-angstrom-.9/deploy/eglibc/ipk/armv5te/ directory.  I didn't see gcc in that directory, only libgcc.
I did see another directory with ipk packages, but it wasn't clear to me if these packages were compiled to run on the arm
target cpu or an i686 cpu.  Some of the files contain the string i686 in the filename.  That directory was:
/chumby-oe/output-angstrom-.9/pstage/angstromlibc/  and all of these ipk packages begin with the word "staging-" which makes me suspect that they are used to stage between the i686 and the target and may not be ready to run on the
target itself.  Do you know any more about this?  Did you instruct bitbake to build the toolchain, or did you somehow get the
adafruit chumby toolchain to run on the Angstrom target system.   Do you know how to use the cross compiler to compile a new kernel module for a previously unsupported device that will run natively on the target platform now that  this whole cross compiling development platform is set up on my UBUNTU 10.04 LTS linux computer?    I assume that I can figure this out by studying the details of what bitbake does and doing hours of analysis on the makefiles and scripts that bunny's instructions say to run, but if you already did it you could save me a ton of time, by sharing how.

You said above that you were able to use the adafruit 8192cu.ko file to get the Deal Extreme WiFi module working on the Angstrom kernel.  I thought that modules compiled for the falconwing kernel image wouldn't work with the Angsgrom kernel, because the linkages to the kernel internals would be at different addresses.  If it works I am glad, bacause that saves time and trouble, but I am not sure why it works, unless all of the kernel module intefaces are through interrupts or something. 

Thanks for any light that you can shed on this large grey mystery,   Barry Mead

Re: using openembedded and opkg

Hi Barry, like you I'm still working on the problem of figuring out just which recipes to run to get the right image for my application, and it is a time consuming process. I'll continue to publish whatever I can discover, though I've been trying to gather up enough results to write coherent articles rather than just publish random snippets.

One such snippet that would probably help you in the short term is that you can build task-sdk-native and install the resulting ipks to get something like the falcolnwing toolchain on the chumby starter image. The ipk which contains the C compiler and associated tools is called binutils.

Like you I was surprised when the precompiled 8192cu.do module worked with the kernel that I had built but I suppose that my new kernel is being built from the same source as the kernel against which the precompiled module was built. Nevertheless I won't be completely happy until I've been able to build the module myself. There is a discussion about it at the following link to the Gumstix support forum:

http://old.nabble.com/Building-driver-m … 00785.html

I'm planning to set up and maintain a Chumby Hacker Board repository at my website http://infurl.net to save everyone else the trouble of having to compile everything from scratch, however it has taken a lot of effort so far and I'm not nearly finished yet. I'll keep the forum posted about my progress on that front.

6 (edited by BarryMead 2011-04-24 05:46:12)

Re: using openembedded and opkg

infurl,  Thanks for responding to my questions so promptly.  I read that gumstix support forum link you listed, and it looks like they put the full kernel source on the target platform, and did the usual kernel and kernel module builds from there.  That is how I was thinking of doing it myself, but didn't know how to get a target platform up to that level of completeness.  I don't know if the exact package list options that they used in that example would be perfect for what we need to do, but it looks like it would be pretty close.  For my application I want to be able to build the same wifi module as you, and also the usbserial and usb host stuff.  I found some bugs in the original chumby usbserial module, and need to be able to recompile it from source to fix those bugs.  It seems that if you have a usbserial device attached that is constantly outputting text, and you frequently open and close that device, it doesn't gracefully open. or close the device, and eventually generates a buffer
overflow in the kernel module's I/O buffers.   This behavior is unacceptable for my application, so I need to fix the driver.
Even flushing the buffers, before closing, and after opening doesn't prevent the bug, and there is no mechanism to use hardware, or software flow control to suspend the flow of characters from the device.  The driver needs to be corrected so that it prevents buffer overflow whenever the device isn't opened.  I don't know why they assumed that
serial devices automatically go quiet when you close them, that isn't a valid assumption for writing  a dependable driver.
Since the bug only occurs occasionally, it is probably only causing the overflow if the buffer is near the wrapping point
when the open or close occurs.  I need to study the source for this driver to see where the logic error is.

Re: using openembedded and opkg

I followed the info at

http://wiki.chumby.com/index.php/Buildi … enEmbedded

to install the OE files and built the default chumby-starter-image. I dd the img over to a 8gb micro sd card and tried it in my c1. It started to boot but after a few minutes it seem to lock up. The last time I tried it I left it alone for about 30min but nothing happened.

Don't shoot me for asking but will this run on a chumby one or was it meant only for the hacker board?
If not is there any info on how to build a base OE install?
I'm using Ubuntu10.10 if that makes any difference.

Here's a link for pics and more info on what I've done so far.
http://forums.parallax.com/entry.php?21 … g-a-Chumby
-dan

Re: using openembedded and opkg

The hardware for the C1 and hacker board are NOT identical, only similar, so I don't think you can just use one ROM image on the other.  If however you customized the bitbake build and recipies to reflect the differences you could build a new ROM image for the C1 just like the hacker board.  Keep in mind that the OE builds are designed to talk to a console plugged into the 3.3V serial port on the chumby and by default don't include any graphics rendering.  There are OE build options that let you include
XORG, and QT and if you specify the graphics parameters correctly they would be able to render to the chumby's touch screen.

To make a long story short, your assumption was right these instructions are only for the hacker board.

Re: using openembedded and opkg

Thanks, that explains why it seemed like it locked up. It was expecting a serial connection-something I didn't have.
It probably didn't have ssh or wifi running either.
-dan

Re: using openembedded and opkg

The hacker board uses the MX23 processor, and the C1 uses the MX21 processor.  If you made the necessary adjustments, and reran bitbake, you could make a C1 image that wouldn't crash.

Re: using openembedded and opkg

The different processors would explain why it didn't work but,
I was looking at the specs for the Chumby one and the hacker board.
hacker board
http://www.adafruit.com/index.php?cPath … cts_id=278
chumby one
http://wiki.chumby.com/index.php/Main_P … by_devices

According to those pages the hacker board and chumby one use the same IMX233 processor, does that mean those pages need to be updated?
-dan

Re: using openembedded and opkg

Dan:

I stand corrected, I was looking at an older schematic, and it said the processor was the MX21.  I downloaded this schematic several months ago from a site that said it was the schematic for the chumby one, but when I googled for chumby one schematics today, I found a newer schematic that says the processor is the MX233-169 pin BGA cpu the same as is on the
the hacker board.  You could review the schematics for the C1 and hacker borard in detail to see what the differences are, and I am sure you could find out why the programs are not compatible, or you could download a pre-built rom image from others who have run bitbake to see if they do run.  That would tell you if your process is broken.  Here are the new chumby one schematics:
http://www.adafruit.com/blog/2009/11/29 … ware-docs/

And here are the schematics for the hacker board:
http://wiki.chumby.com/index.php/Chumby … board_beta

And you can download a good rom image here : http://wiki.chumby.com/index.php/Buildi … ded_(Beta)

Click on the very bottom link after it says "An example ROM image can be downloaded from":

Re: using openembedded and opkg

Dan:  Above you wrote..."I'm using Ubuntu10.10 if that makes any difference."

Bunny was very specific that it needed to be Ubuntu 10.04.  I found a web site that shows what needs to be done to get open embedded working for other distributions.  Perhaps that site will help you find out where the Ubuntu 10.10 went wrong.
http://wiki.openembedded.net/index.php/OEandYourDistro

Hope this helps,  Barry

Re: using openembedded and opkg

Dan:  I thought of one other thing that might help you.  To use the Angstrom kernel made by bitbake and bunny's instructions, you need to have a console terminal attached to J105 of the Chumby One main cpu board.  The cable you need to attache is a modified FTDI  TTL-232R-3V3 cable.  You need to move some pins around on the end of the cable so that only TX RX, and GND are connected to the right pins.  If you don't have one of these modified cables attached, to a desktop computer running minicom (linux) or hyperterm (windows) then you won't see anything when the new linux boots.

Re: using openembedded and opkg

I've set up a repository of compiled packages for the Chumby Hacker Board on my web server. You can add the following sources to your opkg configuration if you want to use them. Directory listing is currently disabled but you can find out which packages are present in each directory using "opkg update; opkg list" or by downloading the Packages.gz file manually. I'll endeavor to set up a file browser of some sort to make it easier to find things at a later date.

http://infurl.net/ipk/all
http://infurl.net/ipk/armv5te
http://infurl.net/ipk/chumby
http://infurl.net/ipk/chumby-falconwing
http://infurl.net/ipk/x86_64
http://infurl.net/ipk/x86_64-armv5te-sdk

It takes my workstation about 12 hours to run "bitbake -k world" to build all these packages from scratch, and it requires around 170 gigabytes of disk space for working and temporary files. I'll try to keep the repository up to date if it is proving useful to the community and it doesn't become prohibitively expensive to maintain.

There are currently nearly 22 thousand packages there, and there are another 2 thousand that don't build successfully yet. There are probably a lot of them that aren't needed for CHB and over time it should be possible to weed them out to streamline the whole process. Please post any suggestions or problem reports here so I can address them as quickly as possible.

16 (edited by BarryMead 2011-04-26 03:46:11)

Re: using openembedded and opkg

infurl:  Do any sections of your current repository contain the linux-source linux-headers, and linux-firmware modules?
From my reading of that gumstix forum link where they were able to compile kernel modules, we will need these.  Was that how you read it too? 

Were you able to compile kernel modules, for the hacker board yet?  Did you do it another way?

I followed your advice and reran bitbake with the  task-sdk-native target, and it ran much faster this time since it didn't need to download the whole world.  I was able to put the whole ipk directory tree on a usb stick, and mount it under /media as you did.  Then I configured the opkg.conf file as you suggested, and was able to install the gcc glibc libc6 binutils and several other
dependancies needed to make the toolchain work.  Now I can compile native c programs on the new target.  All I need to learn now is how to add individual modules to the bitbake recipie files and learn how to compile kernel modules and I will have just about everything I need to make a fully archiveable, maintainable source tree for the system I am using.  When all the pieces fit together I will breathe a sigh of relief that it cannot get lost.

I used a spare computer I had kicking around to do the bitbake stuff on, because I typically don't use ubuntu (normally debian) So I threw ubuntu onto an old 2.8Ghz celeron computer and ran the bitbake stuff over there.  It only has a 60G hard drive so I can't do the whole world on that computer like you did.  My main computer has a 1TB hard drive, but I have debian-AMD64  loaded on that machine, and I knew that bunny's instructions were only for 32-bit ubuntu-10.04 (Lucid Lynx) and I wasn't about to lose all my data on my main development machine just to try out bitbake.  Have you played with that
VM-Ware image that they talked about?  I downloaded it, but don't currently have VM-ware installed so I haven't tried it.

Thanks for paving the frontier,  Barry

17 (edited by BarryMead 2011-04-26 13:53:55)

Re: using openembedded and opkg

infurl:   Have you tried the "rootstock" command that they mentioned in that gumstix forum posting?

sudo rootstock --fqdn overo --login ubuntu --password temppwd --imagesize 2G --seed linux-image-omap,gcc,g++,make,build-essential,openssh-server,nano,apt,usbutils,linux-kernel-headers,linux-firmware,wireless-tools,wpasupplicant,zip,linux-source-2.6.35 --components "main universe multiverse" --dist maverick --serial ttyS2

I haven't researched the "rootstock" command yet to see how it interacts with/affects  bitbake and the whole cross compile/build system.  I guess I need to roll up my sleeves and dig into another unknown area of mystery, so that I can make some sense out of all this stuff.  I also couldn't figure out what he named his rtl8192cu bitbake recipe file, or where he placed it in the massive tree of bitbake recipe files.  I have yet to unravel the complexities of the bitbake dependency and automatic nesting system.  I know it is very sophisticated to be able to do what it does, but, that also means the it is very easy to screw up and create giant messes if you make a mistake.  Have you been reading up on the fine points of bitbake recipe files and how to make new ones that have a minimal impact on the rest of the build?

Re: using openembedded and opkg

Hi Barry, I haven't had time to follow up on the missing packages yet, but like you I am anxious to begin compiling device drivers. I've been doing everything on a Debian-AMD64 system (Squeeze) and have not encountered any problems that I'm aware of. Of course, that may be the reason that there are about 2000 recipes that don't compile yet.

Re: using openembedded and opkg

I did a little more reading on roostock and bitbake, and from what I read it seems that roostock is the OLD way to do embedded systems and now they use bitbake.  So there will be limitations in using rootstock that have since been fixed with bitbake.  I dont' know if they require a completely separate build tree or not yet, I bet they do.  Anyway that leaves me with some confusion about that gumstix posting, because he said that he wrote a "bitbake" recipe, but he was using rootsock, so I am a little confused now.  One thing I did read in the bitbake manual is that there is a separate place for "LOCAL" recipes to help users compile individual packages without messing up the whole recipe nesting tree.  That is a relief.   When I dig more into it tomorrow, I will concentrate more time on this "LOCAL' recipe area and see if we can put the kernel module stuff there.

Take Care,  Barry