Cold weather running can be a real bummer if you are unprepared. The key to staying warm while out on a winter run is layers, layers, layers! How cold it actually is will help determine exactly what you need to wear, and how much.
The first thing to remember is that running warms you up quickly, especially your legs (so, in general, you will wear more up top than down below.) The exception to this is most commonly your extremities- runners' hands and feet/toes tend to get cold quickly so you might often find yourself running around in short sleeves, short shorts, but still in need of gloves and thick socks!
Okay, so let's start in the 60-degree temperature zone. It might feel a bit chilly when you first start, but you will warm up quickly so go for just a simple t-shirt and shorts. Another ten degree temperature drop into the 50's means you should add a long-sleeve base layer under your t-shirt, and long tights if you want (gloves might also be warranted). If it drops down into the forty degree zone, then you will definitely want tights and may need to add on a vest too. For thirty degree temps, you should consider layering on a jacket, and your head will need to be covered. (Heat is lost rapidly through the head!) If the temperature drops below freezing, you might consider doubling up on certain articles of clothing: two long-sleeve base layers, two pairs of socks, and protect your neck! Any exposed skin at those really low temperatures can quickly become a hazard.
If it gets really cold where you live, you can buy tiny thermal packets to keep in your pockets and for you to hold to help warm your hands. And, if possible, try to plan your training around sunlight: darkness often means it will be even colder. And if it really is just too cold, too dark, and too unbearable outside, (aka zero-degree temperatures or below) there is always the treadmill! (Which might require some additional motivation, found here!)