Towards a lower place; towards what is below.
Gravity pulls everything downwards.
Towards something which is lower in order, smaller, inferior, etc.
After the rise in oil prices, the economy headed downwards.