If it's warm you can run in the rain without a jacket and just get wet. But that's not an option in cold weather or on mountain trails where being soaked through can be dangerous with a hypothermia risk.
So what should you look for when you're being a waterproof jacket? It really depends on its use - road or trail, cold or warm weather, short runs or long.
Running fast on the roads generates a lot of sweat. A fully waterproof jacket isn't very breathable and if it's hot or you're running fast, you'll soon overheat.
Most of the time when you're trail running it's best to carry a fully waterproof jacket for safety. Just in case you get caught out. On roads you can get by with a water-resistant jacket.