You'll pretty much lose all FM reception if you get rid of the antenna, however...
There are two ways to open the case:
1) Remove the four round stickers on the back that cover the screw holes, and unscrew the four screws, and slide the back out.
or..
2) With a small flathead screwdriver, gently pry the bottom piece away from the front plate.
Once open, this wire is soldered to the board, so you're need to snip it off. You *could* just curl it up inside the case if you think you might want to use it in the future.
If you don't have a battery, you can just curl it up in the battery compartment.
We really can't get anything without the antenna - it's required by the frequencies at which FM operates. Most small handheld players that don't have a retractable antenna use the headphone wires.