Can a indoor rabbit sleep outside?

hi, i have just brought a lion head mini rabbit, he stays indoors with the run of the flat when i am at home or just pop out for a short period of time. During the night or if i am out for a long period of time i put him back outside in his hutch. He has a large two story hutch with a snuggle and a rain proof cover to protect him from bad weather. Do you think this is to con fussing for him and will cause him stress? i am very concerned and am unsure of what to do please help.