Floor plans: They may not be sexy, but 79% of home buyers we surveyed said they’re more likely to view a home if the listing includes a floor plan they like.* Floor plans provide a quick grasp of a home’s layout, the flow of the house, and room dimensions, so you can see if your furniture will fit.
New construction listings generally have floor plans. But most listings for existing homes — say a 1980’s 3-bedroom split level — do not. That was frustrating for Zillow employee Yuguang Li, who spent the first half of 2020 searching for his first home. “I’d look at a hand-picked set photos of a house, all the different views and wonder, ‘is this all real?’ If you don’t know the geometry of the house, it’s super hard to figure out if the space works for you.”
Zillow Applied Scientist Yuguang Li, in front of the home he purchased in 2020 while working to create floor plans with AI technology.
So Li was excited by the opportunity to help solve the problem. He’s an applied scientist on Zillow’s Rich Media Experience (RMX) team, which has figured out how to create dimensionally accurate floor plans without taking any physical measurements. They also figured out how to integrate floor plans with listing photos and 3D Tours, to give shoppers a more authentic experience. To do this, Li and other engineers have had to solve computer science problems that academics have been working on for the better part of a decade.
“At Zillow everything we do is about helping customers find a home they love so they can unlock the next chapter of their lives” says Josh Weisberg, vice president of Zillow Rich Media Experience. “That’s why we hire people who can push the limits of artificial intelligence — to create a digital shopping experience unmatched by any other real estate site.”
AI and floor plans
Like many computer scientists, growing up, Li wanted to build robots. His work at Zillow is not that far off the mark: He develops technology to help “robots,” or machines, see. Li’s an expert in computer vision, a type of artificial intelligence applied to visual data, including photographs and video.
In the context of floor plans, computer vision allows a machine-learning model to look at images of a room to detect its shape, the locations of windows and doors, and how all the rooms in a home relate to each other. Engineers “train” the model with a collection of example images, enabling the machine to generate a schematic drawing with measurements.
When computer vision is combined with another AI technology called deep learning, the machine decides room shape, dimensions and relative positions of rooms on its own. This automates the floor-plan production, which gets faster and more accurate as the machine gets “smarter” over time.
AI as integrator
AI-generated floor plans are now part of listings in 25 markets nationwide that also include Zillow 3D Home tours. The Ricoh 360 degree camera used with the Zillow 3D Home app is the scanning device that captures the media Zillow’s AI models use to generate the floor plans. The result: Shoppers can navigate a home using the floor plan as a map, instead of just swiping through photos. The floor plan has hotspot icons that shoppers click on to bring up a specific room photo or part of a 3D Tour.
Zillow floor plans have hotspot icons that shoppers click on to bring up 3D Tour segments of spaces they want to explore.
“To make it that simple, we’ve had to solve a lot of hard computer science problems, but what really matters is the home shopping experience,” says Josh Weisberg. “The integration of floor plans with photos and 3D Home tours gives people a more accurate sense of a home’s flow, and in some cases, the confidence to purchase a home sight unseen.”
For agents, the Zillow technology is a cost-effective way to showcase a listing. The 3D Home app is free. There is no added cost beyond what your photographer charges for the capture.
AI at scale
To achieve their goal to use computer vision and machine learning to automatically generate floor plans, Li and his colleagues broke the project into phases. Initially, human investment was high. Engineers built tools and a production pipeline to generate the floor plans. Dozens of people called annotators were hired to label room features in the 360 panoramas captured in tour creation.
Over two months, the crew collected annotations for about 10,000 panoramas. Scientists used this labeled data to train machine learning models. The results were analyzed against “ground truth” evaluation data to validate the accuracy of the models.
Examples of how room shapes, doors, openings and windows are defined and labeled on 360 panoramas.
It was an iterative process that led to an automated pipeline that could produce more floorplans, faster, with accuracy, and with fewer people. Today, the role of annotators has become more of a quality assurance function, in which people correct any mistakes in the models. “We’ve brought down the time and cost of floor plan production dramatically,” Weisberg says.
Li says it’s exciting to see what was once an academic research problem become a new way for shoppers to find the right home. He imagines a day not too far in the future when shoppers might even be able to set up a home search based on a desired floor plan. In the meantime, his team continues to harness the potential of artificial intelligence to transform home shopping experiences.
*Consumer Housing Trends Report, 2020
Related:
- Zillow floor plan training models
- Careers opportunities at Zillow
Watch this video to see how floor plans work with photos and 3D Home tours