MacMillan of Rathdown wrote: “Humidity, or rather the lack of it, is a big factor.”

That’s exactly it. When the humidity is low, much higher temperatures are more bearable.
And the desert can also be cold at night.
Aside from those factors, heavy clothing also has its uses in the desert. The Long-Range Desert Group and others who used the tracks far south of the Mediterranean shores wore greatcoats and other clothing to wrap themselves up against the sand and the heat. Paradoxically it worked.
Regards,
Mike