I think parsing a whole blueprint with monolithic models is really difficult, but the constrained object detection/semantic segmentation problems are significantly more tractable. You can chain those CV models with VLMs to do things like get scale right. I'm always interested in novel HCI paradigms like voice!
Absolutely! That's our end-goal, to remove the painful back-and-forth of permitting. Once you solve blueprint understanding, the possibilities are enormous, from spell-check to material efficiencies, etc.
Thanks for such a detailed question! We're focused on material quantity estimates right now. We're using pretty generic averages for costing as our primary users (suppliers) have a way better finger on the pulse of the market and typically change the unit price anyways. We'll work on better cost accuracy though, and are looking to integrate with RSMeans or building our own scrapers.
For the second question, it really is most accurate on "fully standardized" blueprints due to our training distribution. Will work on improving that as well!
I realize you're opening yourself up to criticism if you answer this truthfully, but since suppliers are your primary user (and therefore paying customers I assume?) your pitch is "supplier, you can do more with less (or no!) people when specifying material quantities, and in fractions of a second as opposed to the minutes/hours it takes today!" So, ultimately, if the suppliers fully trusted your solution they would need zero personnel determining quantities? And since your solution's annual licensing cost would likely be a fraction of the price of even a single individual that'd be pretty compelling to the supplier.
Best of luck with the business (and with getting to know the corp dev people at Autodesk/Procore/etc.--sorry, couldn't help myself!).
> f the suppliers fully trusted your solution they would need zero personnel determining quantities?
You'd still need people to check what actually got installed, so that you can bill for it. Like, there's only so much you can determine off the plans.
And what happens if (when) the plans are wrong or impractical?
My Dad worked in construction for his career, and I did briefly, and there's generally a lot of stuff that needs to be figured out on site due to physical or logistical constraints.
Sounds like this is just for homebuilding though, which is a much easier problem.
It seems like, with sufficient users entering data (including previous sales/purchases if desired), you could begin to train models to take into account local/regional factors etc. do you have any long term plans to collect user data in such a manner?
Separately, it seems like it would be incredibly useful to use your models in various embodied carbon estimation tooling and other decarbonization research streams. Have you thought about partnering with any academic researchers on this? If you are interested, let me know, as I can definitely connect you with a bunch of researchers who would be interested!
We don't really have plans on using user-data to train our models, since it's too much of a chicken-and-egg problem. Carbon estimating is something we had some clients talk about -- it uses the same material quantity estimation we're already doing, and is something we're really interested in (my heart is in impact-driven work). Would love the connections!
We've seen that suppliers do a ton of bidding and cost estimation, and are keen to be accurate (estimate too high and they lose to competitors). GCs and suppliers don't want to be too low because that potentially costs them tens of thousands of dollars.
We do supplying to contractors, and do takeoffs for single family homes. Not tracked, one off's for framing material. You mentioned your co-founder has built tract homes so my words might not apply to your target customer.
Here are things to consider:
Experienced builders don't care about the takeoffs on a big picture basis, the takeoffs are usually wrong, even if perfectly done. In our experience half of drawings we receive, are heavily revised by the order is approved (heavily revised meaning over 10% has changed). EWP, structural metal need to be accurate but framing lumber, and sheet good can be off on counts at the lift quantity (+-1 lift for an average house).
Suppliers aren't responsible for the takeoff so the amount the quote is negligible (see drawing revisions, and trades can misallocate the materials - This can't be reasonably traced). Over? The customer ends up paying less, under? The customer pays more. This has been universal where I am (Ontario, Canada).
A large minority of plans are missing key elements (like sheer walls), pointing out, and showing these differences would be a big value add for the consumer (contractors using the materials) by the supplier.
Good customers understand that lumber is a commodity, a lower price this week can flip next week, and they'll contact their preferred vendor about the differences.
There's always a preferred vendor.
Not great customers will shoot drawing off to multiple suppliers, causing them all to do the same takeoff, wasting time, and money, only to deal with the same issues above. They'll still go back to their preferred vendor to get the lowest price.
Summary of the above is: EWP, and structural metal are key items because they rarely change, framing lumber, and sheathing requirements change all the time. What you're looking at is helping suppliers capture the bad customers (which are often the biggest, to be clear), but saving suppliers the time handling them is great. Also, accuracy, and pricing isn't that important (with caveats).
This isn't a statistically significant sample size, consider it anecdotal.
I love the idea you guys are doing. I also think your name should have takeoff in it.
As a side point - sometimes the shady lumberyards do bid too low on purpose to win business. Then later have the contractor submit a change order. This often hurts their reputation unless the contractor is in on it to win a bid. The supplier doesn’t tend to lose money though as the bid is for the quantity of materials.
You could turn this into a selling point. As in, helping a contractor or competing supplier verify the takeoff.
Hey Patrick, this is rare, it does happen, but usually (almost always) it's that something was missed in the takeoff, the updated plan wasn't submitted to the yard, or the drawings were missing elements.
I agree with your point, having a second, impartial source is important to confirm the ballpark.
