In developing countries, building codes often are outdated or not enforced. As a result, a large portion of the housing stock is substandard and vulnerable to natural hazards and climate related events. Assessing housing quality is key to inform public policies and private investments. Standard assessment methods are typically carried out only on a sample / pilot basis due to its high costs or, when complete, tend to be obsolete due to the lack of compliance with recommended updating standards or not accessible to most users with the level of detail needed to take key policy or business decisions. Thus, we propose an evaluation framework that is cost-efficient for first capture and future updates, and is reliable at the block level. The framework complements existing work of using street view imagery combined with deep learning to automatically extract building information to assist the identification of housing characteristics. We then check its potential for scalability and higher level reliability. For that purpose, we create an index, which synthesises the highest possible level of granularity of data at the housing unit and at the household level at the block level, and assess whether the predictions made by our model could be used to approximate vulnerability conditions with a lower budget and in selected areas. Our results indicated that the predictions from the images are clearly correlated with the index.