Can AI-reviewed rooftop labels support solar planning?
A peer-reviewed Czech dataset pairs 5,359 aerial images with 7,402 human review outputs and one vision-language-model proposal per image. It exposes annotation variation, but every human task began from the AI proposal and the files lack engineering-grade geometry.
The short version
- What happened
- RoofeNet-Multi contains 5,359 building-centred Czech orthophoto tiles, 7,402 human review outputs and one matched Gemini proposal per image.
- Who it affects
- Homeowners, installers and local authorities should not treat AI-assisted roof boxes as a quote or planning decision. Their value today is in improving research on annotation uncertainty and review workflows.
- What remains unproven
- All images come from Czech buildings around fifteen cities, excluding Prague; rural and international architecture may differ.
Keep in mind
All images come from Czech buildings around fifteen cities, excluding Prague; rural and international architecture may differ.
Every human task started from the same model proposal, so anchoring or inherited omissions can inflate apparent human–AI agreement.
Most images have only one human output, and the 60-image manual check intentionally selected easy, high-agreement cases.
The dataset lacks per-tile georeferencing, roof polygons and independently validated tilt or energy-yield measurements. The authors declare no competing interests and report EU programme support.
Source analysis · 2 linked sources. Primary analysed source date: 24 October 2026. Earliest linked supporting source: 14 August 2026.
Read the full report · 4 min The evidence, methods, implications and limitations
The dataset preserves disagreement—but not an independent ground truth
A new peer-reviewed dataset makes an unusually useful limitation visible: people can interpret the same roof differently. RoofeNet-Multi contains 12,761 annotation sets for 5,359 building-centred aerial tiles—7,402 human review outputs plus one vision-language-model proposal for every image. Labels cover flat or pitched roof planes and obstructions such as chimneys, vents and skylights.
However, the final article states that every human task began from the image's model proposal, which the reviewer could edit, delete or extend. The human outputs are therefore not independent of the model or of one another. That design can speed review and reveal how people modify an AI starting point, but agreement between AI and people must not be read as independent proof that either located the roof correctly.[1][2]
The Czech sample is large but uneven
The team sampled buildings within roughly 15 kilometres of the fifteen most populous Czech cities other than Prague. Each 416-by-416-pixel tile came from the national orthophoto service. Ninety supervised university students produced the human outputs; 3,857 images received one human review and 1,502 received between two and nine. Instructors spot-checked work and sent 51 image cases back for correction, but there was no numeric gold-standard screening threshold.
The matched AI labels came from Gemini 2.5 Pro preview with non-zero sampling temperature. Human-to-human mean F1 was 0.733 for roof planes and 0.604 for obstructions across 6,012 directed comparisons covering the multiply reviewed images. Against an any-human-acceptable consensus, the model's mean F1 was 0.684 for roofs and 0.491 for obstructions across all 5,359 images. These are consistency measures under chosen matching rules, not absolute accuracy. A coarse topology check found about 2.0% of human and 2.2% of model obstruction boxes failed to overlap a roof box, but overlap alone cannot show that the box is correctly placed or classified.[1][2]
The distinction that matters
What the release can and cannot support
The same files serve different purposes, but only some claims are supported by their design.
Review
People edit one AI proposal per image
Captures revision behaviour, not independent labels
Compare
Match roof and obstruction boxes
Measures agreement under selected thresholds
Plan
Estimate installable PV area or output
Needs geometry, georeferencing and field validation
Useful for research, not a solar quotation
The release can support experiments that retain several plausible labels rather than collapsing ambiguity into one supposed truth. It also gives researchers a provenance-aware test bed for measuring what changes when humans start from an AI suggestion. That is relevant to public bodies or solar developers considering assisted mapping, because annotation shortcuts can carry the model's initial omissions into the reviewed record.
The files are not ready for engineering decisions. They use axis-aligned boxes rather than roof polygons, omit per-tile georeferencing and do not validate the model's tilt estimates against surveyed geometry. The only manual validation subset contained 60 deliberately easy, high-agreement images. Although model tilt categories often agreed with human flat-versus-pitched labels, both can share the same visual ambiguity and neither is a physical measurement. Reliable deployment would require an independent expert reference sample, geographic tests beyond Czech urban and peri-urban buildings, calibrated area and shade estimates, and comparison with site surveys.[1][2]