1 Topic by zurk (edited by zurk 2012-05-01 10:08:38)

  • Registered: 2010-05-10
  • Posts: 222

Topic: Octopus - Download every single chumby widget and the entire catalog.

I've decided to release this rather than hang on to it. Its a simple method of downloading every single widget including all support files from the entire chumby network library.

Download it here including all source code : http://sourceforge.net/projects/zurk/fi … p/download

Octopus -- A chumby.com full catalog downloader.
by
Zurk
----

Download version 7 of java from http://java.sun.com and install it.

Create a new writeable directory and then go to a dos or unix prompt.

Type C:\<java path>\java octopus on Windows and the program will run.
or /<java path>/java octopus on UNIX and the program will run.
It will create 4 directories and download all XML files, widget SWFs and thumbnails.

You may pipe the output to a text file to examine it later. e.g.
/java/bin/java octopus > octopusoutput.txt

You can then open the output in a text editor to view which files were written where.

Note : This program requires network access and uses the chumby network heavily. Run it only once to back up all the widgets!

  • Registered: 2010-08-13
  • Posts: 89

Re: Octopus - Download every single chumby widget and the entire catalog.

Ok thank you, this will help alot, now can I connect my chumby to the localhost of my computer and use that as a "chumby server"?

  • Registered: 2010-05-10
  • Posts: 222

Re: Octopus - Download every single chumby widget and the entire catalog.

sorta. its actually much easier to dump octopus on the chumby itself and use lighthttpd on the chumby to serve the files. you can do it with a PC webserver too provided it supports aliases.
you need to make sure you alias the correct directory paths to match the stage 1-4 generated by octopus and use a hosts file to point the xml.chumby.com and s3movies.chumby.com to the local webserver.
note that no HTML files are downloaded by octopus so you cant go in there with a web browser and see which widgets are where. the text output will tell you what is downloaded where and the chumby can read the widgets and thumbnails etc.

  • Registered: 2012-04-20
  • Posts: 20

Re: Octopus - Download every single chumby widget and the entire catalog.

How big is the entire haul? Since this potentially could be a bandwidth drain on chumby.com can someone put a copy of the haul up somewhere on the intertubes ? I would do it myself but I am still having issues running octopus; totaly my fault I think as I have an older ver of java (1.6) as Minecraft likes it better than 1.7...and Bob forbid that the kids cant play Minecraft.... Another thought, maybe set up a torrent of it to spread the load. If I can get a copy of the haul I would seed it for a while.

5 Reply by servalan (edited by servalan 2012-04-23 08:03:33)

  • From: Australia
  • Registered: 2008-11-12
  • Posts: 104

Re: Octopus - Download every single chumby widget and the entire catalog.

Hmmm... doing something dumb here as it isn't working - 64bit w7 (tried cygwin env. & cmd)

$ java octopus
Exception in thread "main" java.lang.UnsupportedClassVersionError: octopus : Unsupported major.minor
version 51.0
        at java.lang.ClassLoader.defineClass1(Native Method)
        at java.lang.ClassLoader.defineClassCond(Unknown Source)
        at java.lang.ClassLoader.defineClass(Unknown Source)
        at java.security.SecureClassLoader.defineClass(Unknown Source)
        at java.net.URLClassLoader.defineClass(Unknown Source)
        at java.net.URLClassLoader.access$000(Unknown Source)
        at java.net.URLClassLoader$1.run(Unknown Source)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(Unknown Source)
        at java.lang.ClassLoader.loadClass(Unknown Source)
        at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
        at java.lang.ClassLoader.loadClass(Unknown Source)
Could not find the main class: octopus.  Program will exit.

adding full paths to the command made no diff.

$ java -version
java version "1.6.0_31"
Java(TM) SE Runtime Environment (build 1.6.0_31-b05)
Java HotSpot(TM) Client VM (build 20.6-b01, mixed mode)

(edit: we need 1.7 java to run this???)

  • Registered: 2010-05-10
  • Posts: 222

Re: Octopus - Download every single chumby widget and the entire catalog.

hmm..try with version 7
or recompile with version 6.
This is what i have :
/java/bin/java -version
java version "1.7.0_02"
Java(TM) SE Runtime Environment (build 1.7.0_02-b13)
Java HotSpot(TM) 64-Bit Server VM (build 22.0-b10, mixed mode)

Entire thing is around 130MB but i dont believe its legal to distribute.

  • cb4
  • Member
  • Offline
  • Registered: 2011-03-15
  • Posts: 20

Re: Octopus - Download every single chumby widget and the entire catalog.

I ran octopus and it generated stage1...stage4 folders and my output.txt file.  The total download size was around 114mb uncompressed, currently on my winxp box.  Does that sound right?

Wondering how exactly I can copy this onto one of my chumbys, then use it to serve the other chumbys in the house.  I currently have 2 C1s and a Sony Dash.

Otherwise, what's the best way to identify which widgets are what so I can run them with the Zurk offline fw.

I have the 4 stage folders and the output text file compressed into a RAR for a 101mb download, but I'm thinking it can't be "legally" distributed.

  • Registered: 2010-05-10
  • Posts: 222

Re: Octopus - Download every single chumby widget and the entire catalog.

As long as it got to the Completed line its got everything.
copy them to directories with aliases pointing to full paths. look at the lighthttpd config file for the alias command syntax. you need to modify your hosts file in the offline firmware to point to one of the chumbys ip addresses. best way to identify widgets is to look at the xml definition file for the widget concerned. look in the stage 2 directory which has the index files with the filenames belonging to the widgets and a text description.

  • Duane
  • Duane
  • Blue Octy
  • Offline
  • From: San Diego, CA
  • Registered: 2006-08-24
  • Posts: 8,200

Re: Octopus - Download every single chumby widget and the entire catalog.

cb4 wrote:

...but I'm thinking it can't be "legally" distributed.

That is correct - you must obtain permission from the widget authors.

Duane's Website

10 Reply by servalan (edited by servalan 2012-04-23 14:23:27)

  • From: Australia
  • Registered: 2008-11-12
  • Posts: 104

Re: Octopus - Download every single chumby widget and the entire catalog.

zurk wrote:

hmm..try with version 7.

k - might want to fix your lead post to say 'download version 7 of java from oracle - it's a dev. rel. only - so not at java.com: http://www.oracle.com/technetwork/java/javase/downloads/index-jsp-138363.html#javasejdk' wink

  • Registered: 2012-04-20
  • Posts: 20

Re: Octopus - Download every single chumby widget and the entire catalog.

Finally got it working, had to open a windows vm instance and installed java 7 on it. Works . Thanks. I will now go off and say a few Hail Linus full of opens....:)-

12 Reply by chumbyfly (edited by chumbyfly 2012-04-24 08:48:25)

  • Registered: 2010-12-15
  • Posts: 28

Re: Octopus - Download every single chumby widget and the entire catalog.

Hi Zurk,

   I have a few questions that I would like your clarifications please:
   1. Why do we need to modify your hosts file in the offline firmware to point to one of the chumbys ip addresses?
   2. How come the Completed line said Stage 1,2,3,4 and 5? There isn't any Stage 5 folder? 
   3. How do you use the output/results of octopus?

CF

zurk wrote:

As long as it got to the Completed line its got everything.
copy them to directories with aliases pointing to full paths. look at the lighthttpd config file for the alias command syntax. you need to modify your hosts file in the offline firmware to point to one of the chumbys ip addresses. best way to identify widgets is to look at the xml definition file for the widget concerned. look in the stage 2 directory which has the index files with the filenames belonging to the widgets and a text description.

  • Registered: 2010-05-10
  • Posts: 222

Re: Octopus - Download every single chumby widget and the entire catalog.

1. so the chumby will know where to load the widgets from ? effectively youre substituting chumby.com with your own local chumby ip address.
2. typo. there was a stage 5 but it was merged into stage 4.
3. look at the URLs it generates and look in the stage2 folder. read the files in the stage 2 folder which tell you what widget/thumbnail/xml snippet/template corresponds to which hex value. you can use a text editor for searching to the widget you want. the downloaded files will be inthe stage 3/4 folders. bit convoluted but then its a 100-ish line program which simply downloads everything it sees. not exactly user friendly. i might clean it up when i get time.

14 Reply by Xav

  • Xav
  • Xav
  • Chum
  • Offline
  • From: UK
  • Registered: 2012-04-26
  • Posts: 25

Re: Octopus - Download every single chumby widget and the entire catalog.

zurk wrote:

hmm..try with version 7
or recompile with version 6.

I'm using Ubuntu with Java 1.6 here. Can you tell me how I would go about recompiling? I'm pretty tech savvy and very comfortable with a command line, but I'm not a Java developer.

Xav's Website

  • From: Austin, TX
  • Registered: 2009-05-07
  • Posts: 471

Re: Octopus - Download every single chumby widget and the entire catalog.

Just run "javac" on the octopus.java file included, that will rebuild it.

unwiredben's Website

16 Reply by Xav

  • Xav
  • Xav
  • Chum
  • Offline
  • From: UK
  • Registered: 2012-04-26
  • Posts: 25

Re: Octopus - Download every single chumby widget and the entire catalog.

unwiredben wrote:

Just run "javac" on the octopus.java file included, that will rebuild it.

Thanks! It looks like it's working, but given that my internet connection runs over a bit of wet string shared amongst several people I won't get to the final line of output for a while...

Xav's Website

  • From: Tulsa, OK
  • Registered: 2008-08-04
  • Posts: 657

Re: Octopus - Download every single chumby widget and the entire catalog.

Hello zurk,

I'm all for a free chumby.  I myself have torn down the CP and started an alpha build of an alternative, but stopped short at my abilities when it came to manipulating the actual device firmware.  That being said, I can't say I'm particularly thrilled that my widgets are running wild outside of the chumby ecosystem, no matter what the state of said system may be.  A couple of my widgets use server resources that I personally manage.  This puts me in a precarious situation now in regards to keeping those widgets running.

While I believe that there is a solution for moving from the chumby servers, this type of "looting the store" leaves me feeling a little queasy.  Obviously I don't have the power to stop this, but I can certainly share my opinion as a developer.

However, I do applaud your (and others) efforts on localizing the system.  We all benefit by the hard work of the community to keep chumby screens lighting up to see another day.

Best,
Cb (develephant)

Get -> Chumbyland | Joice | Word Up | Newsdom

cbreeze's Website

  • Registered: 2010-05-10
  • Posts: 222

Re: Octopus - Download every single chumby widget and the entire catalog.

well develephant -- i have good news for you. in a few weeks octopus will no longer work. i'm sure you will be thrilled at that news.
you will also be thrilled at the fact that chumby.com will also not work in that same timeframe. and you will have no users using your crappy widgets.  no more pesky idiots using your precious software eh ? awesome.
I also have some more news which will thrill the hell out of you -- you can block your widgets from running on the few chumbies left after the dust settles. just simply do a check for chumby.com = 127.0.0.1 on your widget start and YOU YES YOU can block your widgets from being run by all the looters using the offline firmware. no doubt such scum should be punished severely. recommend you also add a random flash overwriting routine to your widgets to make sure their chumbies never work again.

  • Duane
  • Duane
  • Blue Octy
  • Offline
  • From: San Diego, CA
  • Registered: 2006-08-24
  • Posts: 8,200

Re: Octopus - Download every single chumby widget and the entire catalog.

Zurk, with all due respect, you're being unnecessarily rude to one of the most generous third party contributors to the Chumby community.

cbreeze has made a quite reasonable statement regarding the possible personal implications of what you're doing.  He was comfortable with what Chumby was doing with his widgets, and he actually has a legal relationship with Chumby through the terms of service.  You have no such relationship with him, or the other widgets developers whose stuff you've copied, so I would personally encourage you to tread very lightly.

Whatever else you might think about what's going on with Chumby (and I have to say you're certainly reading a lot into a handful of articles), the widgets are not yours to do with simply as you please.  They are owned by their developers and they have the ultimate say, both morally and legally, as to what happens to them.

Duane's Website

20 Reply by gregf (edited by gregf 2012-04-26 17:13:36)

  • Registered: 2008-03-14
  • Posts: 45

Re: Octopus - Download every single chumby widget and the entire catalog.

I agree that we should have permission from the developers before copying the widget locally.
My classic has been my faithful clock next to my tv for some time.
In particular roboclock, binary clock, nixtie clock and other clock widgets.

Any ideas on how to get permission?
Legal issues aside, it's the right thing to do.
But I will also applaud the efforts of those who are trying to help users keep their chumbies going.

Some day little black lumpy friend will display it's last, but such is life.
I wish all of my electronic things worked as well as my chumby has.

  • Registered: 2011-03-15
  • Posts: 7

Re: Octopus - Download every single chumby widget and the entire catalog.

I have only developed a few widgets for the Chumby devices but I give my permission to copy them.

I have even posted the source code and instructions on how to write widgets which is available to the community at http://chumbydev.blogspot.com/. I do have one widget which is using my server however I have not found it to be a heavy load and I reserve the right to take it down at any time. With that said, I have posted the source code for the server piece as well and the widget can point to any server running the web service so it is not much of an issue.

  • Registered: 2010-05-10
  • Posts: 222

Re: Octopus - Download every single chumby widget and the entire catalog.

hey richteel -- if u get some time please look at the offline firmware. i have a text messaging widget similar to yours with the server hosted on the chumby itself. problem is the widget basically sucks. it could be waay better with a bit of help. fla is included so modify as required.
also your analog clock is waay better with picture background than the talking clock interface...except yours doesnt use the speech synth system -- if you have time to merge the talking clock with one of your widgets that would be cool too. any help is appreciated before the curtain goes down.

  • Registered: 2011-03-15
  • Posts: 7

Re: Octopus - Download every single chumby widget and the entire catalog.

Sorry but fla files are useless to me as I do not have Adobe tools to edit these and there are no freeware apps which will allow me to do anything with them. I wrote my widgets using ActionScript only using the freeware FlashDevelop application.
It would be best to look at my blog as I tried to explain what I had done and included all the source code. If you have specific questions you may ask and I will help if I can.

  • Registered: 2010-05-10
  • Posts: 222

Re: Octopus - Download every single chumby widget and the entire catalog.

.as files are also included. smile

25 Reply by Xav

  • Xav
  • Xav
  • Chum
  • Offline
  • From: UK
  • Registered: 2012-04-26
  • Posts: 25

Re: Octopus - Download every single chumby widget and the entire catalog.

Xav wrote:
unwiredben wrote:

Just run "javac" on the octopus.java file included, that will rebuild

Thanks! It looks like it's working, but given that my internet connection runs over a bit of wet string shared amongst several people I won't get to the final line of output for a while...


It died after I left it running last night, but re-running it today seems to have gone fine. Thanks again for the instructions on recompiling - certainly easier than installing a new copy of Java.


cbreeze, I do understand and appreciate your perspective on this. I am quite happy to continue running my Chumby using the stock firmware and the chumby.com servers for as long as I can. But I also don't want to be caught with my pants down when the servers finally go dark (as they will do one day - even if they get a stay of execution past the summer). Using octopus is simply a precautionary measure that I can make now, but won't be able to later. I would prefer that either the servers can be kept running, or that some other amicable solution can be found for the unchained chumbys, but I'd also like to be prepared for the worst.

Xav's Website