You can ride all winter and enjoy it. Here's the secret, borrowed from Mark Twain: Clothes make the cyclist. As the thermometer drops, the number of layers of clothes you wear needs to increase.
The first place you're likely to notice the cold is in your fingers and arms. Get some full-fingered gloves for your pinkies. Put a long-sleeve shirt under your bike jersey or get a pair of arm warmers. Arm warmers are simply tubes of fabric that you pull on over your hands and arms. They cover from your wrists to your shoulders and turn a short sleeve shirt into a long one. If the weather warms up, as it often does on a fall day, you can peel them off and stuff them in your saddle bag or pockets.
Because your legs are constantly moving, they don't get as cold as quickly. When they do start to feel the chill, switch from shorts to knickers or wear knee warmers. Knee warmers are just like arm warmers except they cover your legs from just below the knee to mid-thigh. Like arm warmers, when things heat up you can slide them off and ride in shorts.
As the weather gets colder, you need to give more thought to the various layers of clothing. Start with a wicking layer such as polypropylene or lightweight wool. Wool is a great insulator but it makes a lot of people itch. If you're one of those, go for a synthetic fabric next to your skin. Over the wicking layer add an insulating layer such as a fleece vest or pullover. To block the wind, you might need a windproof layer. Be aware, though, that windproof layers prevent perspiration from evaporating and defeat, at least in part, the purpose of the wicking layer.
You might find that the full-fingered gloves you got for the fall are too thin. If so, get a pair of glove liners or even a pair of cross-country ski gloves. Just make sure your fingers are mobile enough to shift gears.
Humans lose 75% of their body heat through their heads. Bike helmets aren't designed for insulation so get a thin knit cap to wear under your helmet. Most bike shops carry them.
Don't forget your feet. If you use bike shoes, you'll need a pair of booties to keep your toes toasty. Even if you ride in regular shoes, get a pair or two of thick socks.
Riding in winter has challenges not present in other seasons. Sand and gravel laid down by snowplows and sanders make corners tricky. Turning on sand or gravel can slip a wheel out from under you faster than you can say "whoa, Nellie." Also be wary of unplowed bike lanes and ice buildup along the sides of the road.
A face mask, designed for skiers, is a good way to prevent a frostbitten nose or cheeks. If you choose to go bare-faced, check exposed skin often. Also, just because it's 40 degrees outside doesn't mean you can't get sunburned. With reflection from snow and ice you need to protect any exposed skin, and wear UV-protection sunglasses to protect your eyes.
Your bike needs attention, too. Take some extra care cleaning away the road grime.
Riding in winter can be a lot of fun. On a sunny winter day, the cold is invigorating. People ski, snowshoe and ice skate in the cold; there's no reason not to ride. A bonus: Come spring, you won't have to get into shape because you will have never gotten out of shape.