You can't fly kites on rainy days, remember," then asks Willie to lock up the tea shop, and walks out into the rain, as Sam mentions that the bench Hally sat on as he flew the kite said "Whites Only" but Hally was too excited to notice it, and that he can (figuratively) leave it at any time.