First I tried cutting a section out of a non-crinkly plastic bag, to place between the plastic sections to seal it better. This seemed to work and I went back to bed. A lot quieter too. But no dice! I was still getting chilly breezes! On further investigation the cold air was coming out of the sides of the vent where it doesn't seal properly with the wall. Luckily, I knew exactly where I had some draught excluder hiding away, so used that to seal it properly. This time it worked like a charm, and the bag logo is quite funky through the slats of the vent too.
But, bah! DIY at 5:30 in the morning. Someone up there is having fun taunting me. *shakeyfist*