Navigate to Manage > Devices > select the building from the drop-down menu. If your devices haven't been online in a while you'll see an option to "Remove all".
For all other devices, select it from the list > Edit > Delete.
How to delete a room display
Updated