If you have one close by, go to an Army surplus store. You can get some warm stuff (lots of wool) for cheap.
I particularly like the wool watch caps which keeps my head warm even when it's wet. Usually $2-$3 each. I have a drawer full of them.. (well, maybe 5.. ).