[JSLTIGER]

1: Yes, although unless you're engaging in risky activities with your computer, you probably won't notice it. Most worms and such require a user's stupidity to be what initially launches them.
2: The speed increase comes as a result of the lack of CPU usage and memory usage incurred by Norton running background scans on your computer. Going back to number 1, this is a loss of a layer of protection.

In all honesty, eliminating Norton probably won't affect you too much. Norton allows you not to worry about viruses and the like by effectively putting your computer on auto-pilot. So long as you're willing to manually put in the time to run most of these freeware programs, you should be fine. Of course, you leave yourself a nice gaping hole in your e-security if you neglect to remember to do this.