Personally, if nothing else than for the environment, I would turn it off.
There is an argument that if you are running a 24/7 Internet connection (like cable or DSL), then you are more vulnerable, but really you'd have to be running some kind of service, such as a web server, ftp server, mail server etc... for you to be really vulnerable.
However, some spyware, viruses etc.. do indeed act as some of the above, so if you are infected, your computer could be really busy during the night
I generally switch off my desktop computer, but I have 2 servers running 24/7. Every morning I get a report outlining the activity during the previous day. It's amazing how much the machines are being attacked.
For instance, part of this mornings logs contained:
authentication failure; logname= uid=0 euid=0 tty= ruser=
rhost=126.96.36.199 : 28496 Time(s)
check pass; user unknown: 28496 Time(s)
That was 28,496 attempts at guessing the password for my FTP server.
Every time I'm at this computer, I repeatedly get messages coming up saying "The remote computer has closed the connection". It is such a regular occurrence that I hardly notice them any more. This is simply somebody trying to take over my computer - again by guessing the password set on my VNC (remote desktop software which allows me to control my computer from a remote location) .
It's a battlefield out there and any time you can reduce your computer's exposure to the Internet, the better IMO.