These are some of the things when I build my “building” collection (by “building I mean anything from houses, semi permanent structures like police sheds, storefronts, not necessarily high rises”)- first I look for buildings that when photographed, look like they can be from “anywhere” (A Manila building photo looks like it could be something I shot in Taiwan.). Of course, that also depends on how you crop your photo, because a few inches more environmental detail, and it will reveal where I shot it.
Second I look for something in the scene that shows how man has altered the building or its environs, either through interventions like election posters, banners, signs, graffiti, or the ravages of time- streaks of repainted color over the original wall, cracks, grass starting to grow out of cracks.