Construction is a $1.3 trillion a year industry in the United States. But, despite this, it can feel like nothing has changed in the last century. “Construction is antiquated,” says Patrick Murphy, CEO at Togal. “People are used to doing things the same way. Nothing major has truly changed since my grandfather first entered the business."
Take the estimation – or “takeoff” – process as it’s called in the industry. First, the architect and developer comes up with the design. Then they give those blueprints to the general contractor who has to measure the area and perimeter of each room to calculate a bid. Then the contractor passes those same blueprints to subcontractors like plumbers or electricians, who start their own estimation process all over again.
“Takeoff’s are laborious and time consuming. And it’s a loss leader for general contractors,” says Togal President Patrick Hughes. “They don't get paid for this part of the process. They might put in twenty bids and they’re only going to win a handful of those.”
That’s about to change.
Togal has built the world’s fastest estimating takeoff software, powered by deep learning artificial intelligence models. Togal’s machine learning engine uses AIA measurement standards to automatically and accurately detect, label, and measure project spaces and objects located within those spaces in seconds – a process that used to take hours or days, be duplicated multiple times, and slow down the entire bidding process.
But, like many tech startups with deep industry knowledge and non-technical founders, finding the skilled talent to bring their vision to life was a bumpy road.
Finding the right partner
Togal CEO Patrick Murphy – a former congressman and Executive Vice President of Coastal Construction – had firsthand experience with the challenges plaguing the construction industry and knew that emerging technologies like machine learning could have a huge impact. But he also knew his team would have to find the talent to build the technology.
“When you start a company, you can go out and try to find full time developers,” says Murphy, “But skills like machine learning are very limited and people that are really good at it are probably already working at Google or Amazon.”
So Togal did what many companies in their position do: they met with various technical agencies and offshore partners who assured them they could build the technology Togal needed.
“We hired a group that we had met that we thought could solve the problems,” Murphy says, “and we spent time and money, but it turned out they weren’t making the progress they suggested they were. Then we were lucky enough to be introduced to Noah and Tribe.”
Industry knowledge meets technical skills
Tribe is a collective of engineers, researchers, and scientists from industry leaders in AI and machine learning. But, unlike talent marketplaces where every engineer is a lone agent and clients are left to fend for themselves, Tribe partners with clients to understand the problem they’re trying to solve and put together a team with the skills and experience to solve it.
“Tribe really took the time to work with us and understand the process and what it would take for us to penetrate it,” says Hughes.
“If you want to have real impact, you have to think beyond technical talent,” says Noah Gale, Tribe co-founder. “It’s also about strategy and process, understanding how to build teams of experts and deploy their skills in a way that’s going to build something really powerful and industry-changing.”
Together, Togal and the Tribe team identified that it wouldn’t be enough to solve their user’s pain point. They’d have to solve it in a certain way – in this case something incredibly simple and streamlined – in order to reach market adoption.
“We knew what it needed to be: upload your plans, hit these three buttons, and here’s your report,” says Hughes. “From the beginning, Tribe has been really helpful for us developing this platform in this way.”
Building ML from scratch
Once the scoping process was done, Tribe drew on the collective to find just the right talent for the role. Due to the nature of the challenge – drawing structured and useful information from construction drawings – they assembled a team of engineers who specialize in highly complex, frontend and backend computer vision use cases. This included machine learning, design, product management, and frontend/backend experts.
“Machine learning runs the gamut,” says Nathan, product manager on the project. “On one end there are machine learning APIs that Google or Microsoft have developed for computer vision, but then there’s what Togal is doing – building a machine learning model from the ground up. It’s a very, very challenging project.”
First, the team needed to create their own labeled data set. This required the engineers to develop an entire methodology and create a specialized application. This was a huge undertaking. “There are entire companies out there where they specialize in labeling data,” says Nathan.
The next consideration was the algorithm itself. “What type of neural networks do you use to learn?” Nathan says. “How do you make sense of a bunch of construction drawings? How do you understand where the boundaries are?”
The challenges of designing a working end-to-end algorithm that generalizes all kinds of floor plans required the engineers to use both state-of-the-art artificial intelligence models and heavy data engineering to capture the right information needed to train such models.
“There really is no common standard for these technical drawings,” says Frank, a deep learning expert on the Togal project. “In the end, our machine learning models have to emulate an actual architect and be able to interpret all kinds of floor plans.”
The AWS technology
- Specialized application for data labeling of walls and objects in floor plans
- Neural network for analyzing building layouts from floor plans
- Customer-facing application, easing the workflow of construction industry professionals
The project started with building CI/CD pipelines for further work. This involved GitHub Actions, which build Docker images, automatically upload them into Amazon ECR and automatically spin them up on Amazon ECS Fargate. This way Tribe was able to easily set up three environments: for development work, staging and production. For our DNS needs we have set up Route53. And AWS CloudFront was used as a CDN.
The Tribe team also relied heavily on Amazon RDS to provision easily scalable PostgreSQL databases on Amazon RDS.
Next, Tribe used custom EC2 P2 and P3 instances to train machine learning models: semantic segmentation, object detection and more. Once the models are trained they were able to quickly produce artifacts to deploy on AWS Sagemaker Inference Endpoints. This allowed Tribe to easily scale up the infrastructure once we saw the load increase.
Tribe also designed workloads which were easily parallelizable into standalone Lambda functions and deployed them on AWS Lambda. This allows the tea to handle spikes of more than 100 concurrent requests with significant computation running on each.
Later Tribe moved from SageMaker Inference endpoints to custom made ML inference running directly on ECS. With spiky workloads Tribe had to put each endpoint behind a EC2 LoadBalancer with AutoScaling enabled. This allowed the team to get similar behavior from AWS SageMake Inference Endpoints, but with more flexibility running custom code.
Another side effect of running within AWS was that we the team gets AWS CloudWatch integrations without hassle, mostly by just writing to stdout. The team used these logs heavily while debugging problems in the systems.
As the project matured, Tribe saw the need for improved security and so AWS WEF was a few clicks away, giving firewalls. On top of that the automated code scanning of containers on ECS ensures the team are never missing any critical vulnerabilities.
A simplified diagram of an initial set up is shown below:
Scaling the team
In machine learning projects, the actual machine learning piece is hugely important, but also only a single piece of the work. These projects are complex mixes of software engineering, research, data management, UX, and ops work, all of which need to be set up from a unique, machine learning-first perspective and require strategic leadership to execute successfully.
“As time went on, it became clear we needed dedicated technical leadership in place,” says Noah. “So we brought on a CTO.”
“AI is not IT,” says Oleksandr, the CTO on the Togal project. “Best practices for deploying AI projects are still in their infancy, so it’s important to rely on prior deployment experiences more than existing industry processes and stay nimble in the face of an ever-changing landscape.”
This kind of flexibility is the norm for a project of this size and scope. “The nature of machine learning is scientific and hypothesis driven,” says Nathan. “You have to change your frame of mind, from here’s a set scope of work with a deliverable to this is a problem that no one has ever tried to solve before.”
And that requires the right mixture of expertise and flexibility.
“For people in the construction industry who see this, it’s like sliced bread,” says Murphy. “It’s so obvious – of course that’s the way it should be.”
Simplifying a complex problem
Together, Tribe and Togal developed a product that’s poised to change the way the construction industry works. “Togal will be able to help make the planning and bidding processes more client efficient, less prone to waste and delays, and faster for all parties,” says Murphy.
The technology is game changing, but not just because it saves time and money at every stage of the process. It’s also the beginning of a complete mindset shift. Before Togal, there were other software options out there, but all were too manual in essence. The interface was time consuming because users would have to take a mouse and drag it to outline every single room. It was complicated, clunky, and hardly saved time over using a ruler or roller to measure printed plans..
By taking this complex problem and showing how streamlined it can be, Togal has – in effect – opened a door into the future. And the industry is taking notice.
“For people in the construction industry who see this, it’s like sliced bread,” says Murphy. “It’s so obvious – of course that’s the way it should be.”
Beyond that, the success of this project has acted as a proof point for how much opportunity there is within the construction industry for machine learning.
“Construction is one of the biggest green fields left that’s ripe for disruption by technology,” says Murphy. “It’s an industry dominated by a lot of traditional-minded people that reject technology and are resistant to change. There’s a lot of opportunity in the sales, the bidding, the estimating, and the actual construction itself – all of it.”
An ongoing partnership
“We’ve made tremendous progress,” says Murphy. “With Tribe, it’s been a partnership. We know Noah has our back as someone who’s built a tech company before. On one hand, an advisor and on the other, getting the right talent into the right role at the right time. The resources that Tribe has and the doors they can open – investments, new ideas, new ways of doing things – they’re at the cutting edge.”
While phase one of the project may be over, the partnership isn’t. “We see every project as an ongoing partnership,” says Noah. “Because it’s not just about building a product – though obviously we have incredibly talented people to do that – but about working with partners to build and execute on a strategic vision. We’ve been able to help clients solve industry problems that would have felt impossible even a few years ago. It’s really exciting stuff.”
“Tribe stepped into our company at a critical time to help us not only build out our machine learning, but also to act as a true advisor,” says Murphy.
