That is correct, hot air does rise. However, thermal radiation knows no up or down, left or right. There is only one direction and that is hot to cold. This is the same principal with the sun and why you feel warmth as soon as the sun hits you. That is the thermal radiation from the sun warming your body.