Honestly, I think it depends on their own individual personality, how much you handled it before, and how well it took to handling. When I kept in the past, I could go weeks without holding my animals and they were as docile as they were when I handled them frequently. My royals can become a bit head shy, but other than that, I never saw signs of defensiveness, stressed at being handled again, or anything.
To the OP, you shouldn't have to worry about your animal. Go visit your brother, have fun, and don't worry about him having a hissy fit when you return.