Topic: Using Chumby as a front-end (CAR)

I want use my Chumby-based Infocast8 as a front-end for my Car-puter. What I mean is I want to use the touchscreen and LCD to control the computer remotely (relay video and mouse control back and forth to the computer in the trunk.)

Originally I was looking into porting Android (any version really) on to it.  Unfortunately, porting to the device seems to be more difficult then I initially realized. So back to the drawing board.

Then I thought hey, all I really want is the touchscreen. The rest I can do with an externally with a PC over ethernet or over USB. The benefit is I'm not limited to using an embedded solution with limited RAM / Hardware.

What I would need:
  1. Linux will act as the host OS.
      - Preferably as stream-lined and stripped-down as possible.

  2. Some form of remote software such as VNC, etc..
      - Possibly lower level like bitmap and serial streaming?

  3. Any necessary tweaks to improve performance/framerates like buffers, drivers, etc...

Considering I don't plan to watch video (porn) while I'm operating my vehicle, my performance reqs are relatively low. The most time-sensitive use I can think of is GPS. Also, Audio is not needed since I can just run RCA from the PC to my AMP.

Any, suggestions where I should start? Anyone seen anything similar to what I'm describing?