That's correct. However, there is the option for those in buildings one and three to use the covered walkways between the buildings on the fourth and eighth floors and take the elevators.

Also, characters are allowed to move to a room and building that better suits their needs if they'd like.