The same is happening to me. I tried updating the firmware (Have 1.7.3 now), with no results.
Tested pinging the ntp server, and executing ntpdate to a couple of the time servers listed in /etc/ntp.conf, both run OK. Any other ideas?
chumby:/etc# ping south-america.pool.ntp.org
PING south-america.pool.ntp.org (200.89.75.198): 56 data bytes
64 bytes from 200.89.75.198: seq=0 ttl=52 time=375.3 ms
64 bytes from 200.89.75.198: seq=1 ttl=52 time=342.6 ms
64 bytes from 200.89.75.198: seq=2 ttl=52 time=948.1 ms
64 bytes from 200.89.75.198: seq=3 ttl=52 time=352.7 ms
64 bytes from 200.89.75.198: seq=4 ttl=52 time=307.4 ms
64 bytes from 200.89.75.198: seq=5 ttl=52 time=769.1 ms
64 bytes from 200.89.75.198: seq=6 ttl=52 time=833.5 ms
64 bytes from 200.89.75.198: seq=7 ttl=52 time=737.3 ms
64 bytes from 200.89.75.198: seq=8 ttl=52 time=461.6 ms
64 bytes from 200.89.75.198: seq=9 ttl=52 time=384.7 ms
64 bytes from 200.89.75.198: seq=10 ttl=52 time=448.9 ms
64 bytes from 200.89.75.198: seq=11 ttl=52 time=440.0 ms
--- south-america.pool.ntp.org ping statistics ---
12 packets transmitted, 12 packets received, 0% packet loss
round-trip min/avg/max = 307.4/533.4/948.1 ms
chumby:/etc# ping pool.ntp.org
PING pool.ntp.org (170.210.222.2): 56 data bytes
64 bytes from 170.210.222.2: seq=0 ttl=56 time=433.5 ms
64 bytes from 170.210.222.2: seq=1 ttl=56 time=82.1 ms
64 bytes from 170.210.222.2: seq=2 ttl=56 time=86.2 ms
64 bytes from 170.210.222.2: seq=3 ttl=56 time=97.1 ms
64 bytes from 170.210.222.2: seq=4 ttl=56 time=103.8 ms
64 bytes from 170.210.222.2: seq=5 ttl=56 time=223.7 ms
64 bytes from 170.210.222.2: seq=6 ttl=56 time=125.2 ms
64 bytes from 170.210.222.2: seq=7 ttl=56 time=57.0 ms
64 bytes from 170.210.222.2: seq=8 ttl=56 time=128.0 ms
64 bytes from 170.210.222.2: seq=9 ttl=56 time=123.2 ms
64 bytes from 170.210.222.2: seq=10 ttl=56 time=105.8 ms
64 bytes from 170.210.222.2: seq=11 ttl=56 time=82.3 ms
64 bytes from 170.210.222.2: seq=12 ttl=56 time=103.3 ms
--- pool.ntp.org ping statistics ---
13 packets transmitted, 13 packets received, 0% packet loss
round-trip min/avg/max = 57.0/134.7/433.5 ms
chumby:~# ntpdate pool.ntp.org
3 May 10:36:40 ntpdate[26701]: step time server 168.181.185.90 offset 755.484377 sec
chumby:/etc# ntpdate south-america.pool.ntp.org
3 May 10:39:33 ntpdate[26779]: step time server 201.217.3.85 offset 5.093600 sec
chumby:/etc# ntpdate pool.ntp.org
3 May 10:40:08 ntpdate[26780]: step time server 168.181.185.90 offset 0.994468 sec
chumby:/etc# ntpdate pool.ntp.org
3 May 10:41:00 ntpdate[26797]: step time server 168.181.185.90 offset 1.396551 sec