Launch HN: Segments.ai (YC W21) – Build better datasets for image segmentation

[bertdb]

Hi HN!

We're Bert and Otto, founders of Segments.ai (https://segments.ai). Our platform helps computer vision teams build better datasets for image segmentation, an increasingly popular computer vision technique in the world of self-driving cars, autonomous robots, and AR/VR devices.

A large, curated dataset of labeled images is the first thing you need in any serious computer vision project. Building such datasets is a time-consuming endeavour, involving lots of manual labeling work. This is especially true for tasks like image segmentation, where every object and region in the image needs to be precisely annotated with a pixel-level segmentation mask. Manually segmenting a complex image can easily take up to an hour, even for experienced labelers. This leads to costs of tens to hundreds of thousands of dollars for labeling large datasets.

With Segments.ai, our goal is to make it easier, faster and cheaper to build such datasets. Our core product is a powerful labeling technology for image segmentation, with automation features powered by machine learning. We're constantly tweaking and A/B testing the UX to optimize for labeling speed, and see empirical speedups of 2x-10x for semantic, instance and panoptic segmentation labeling, compared to traditional labeling tools. Have a look at this video to see it in action: https://youtu.be/8u1XHU7ueqU

Furthermore, after you’ve labeled an initial dataset and trained a first ML model, you can upload your model predictions to our platform and use those as a starting point to label additional images. Our labeling technology makes it easy to correct the predictions, as opposed to labeling each image from scratch. We call this model-assisted labeling, and it allows you to obtain additional speedups by iterating quickly between data labeling and model training. More details in this video: https://youtu.be/sCbNp9EDtjE?t=42

Otto and I rolled into this space a year ago, after our PhDs in ML and computer vision. I did my PhD on Scene Understanding for Autonomous Platforms, and experienced the problems with collecting high-quality labeled datasets for image segmentation first-hand.

The market for generic labeling platforms and services is very crowded, and so with Segments.ai we’re going deep rather than broad: our focus is on image segmentation specifically, and we aim to be the best in it. We managed to carve out a niche, and have happy customers across a wide variety of industries: from pharmaceutical companies and automotive OEMs to robotics startups. Our bet is that image segmentation is a fast-growing niche.

The easiest way to try out our platform is by creating an account (https://segments.ai/join) and playing around with the example images.

We would love to hear your thoughts on what we've built!

Bert

[fourseventy]

There are like 4 different companies called segment or segments

[lnsp → fourseventy]

Agree, but at least their name is descriptive. A bit more originality would have done a lot of good here though.

[crubier]

Really cool! We actually planned to do something similar at Sterblue 2 years ago but never prioritized it because it’s too far from our primary scope. But we definitely see the use for this exact tool, it’s really nice. This approach of « smart labelling » is really perfect to quickly obtain high quality segmentation datasets. Would you offer your product frontend labelling component as a library ? That would be ideal for us, as labelling interacts with other stuff in our frontend. Having it in our product rather than on a separate platform would be ideal. Congrats on the launch!

[bertdb → crubier]

Thanks! Would be great to learn more about your use case, we'll be in touch with you.

[auraham]

Any recommendations for learning image segmentation for medical images? I would like to learn how to use a pre-trained Keras model like FCN or U-Net. However, most of the resources I've found so far are a bit harder to grasp. Right now, I am reading 'Deep Learning with PyTorch'. The second part of the book covers image segmentation in great detail, but sometimes is too dense. I am familiar with CNNs, convolutions, max pooling, and so on, but not with upscaling and skip connections.

[bertdb → auraham]

That book is great if you want to go in-depth! If you're a practitioner who wants to get to a trained model as quickly as possible, you're probably better of just following a tutorial. The official Keras tutorial on segmentation looks pretty good [1]. We also have a blog post with code samples on how to set up an image segmentation workflow with Segments.ai and Facebook's detectron2 framework [2].

[1] https://keras.io/examples/vision/oxford_pets_image_segmentat...

[2] https://segments.ai/blog/speed-up-image-segmentation-with-mo...

[auraham → bertdb]

Thanks, your tutorial seems great!

[iujjkfjdkkdkf → auraham]

If you haven't already, take a look at the detectron2 models and tutorials, especially the colab they have. It is a great way to get started.

https://github.com/facebookresearch/detectron2

[auraham → iujjkfjdkkdkf]

Thanks! I was not aware of that model.

[technologia]

Love what I’ve seen so far, I’ve tried superannotate but ended up opting to build my own AI assisted tools into label box, excited to potentially try this out.

[bertdb → technologia]

Looking forward to hearing your feedback when you give it a try!

[janhenr]

Hi Bert and Otto,

As a fellow belgian, I have been following you and segments closely. Congrats on being the first belgian YC company :).

From the beginning onwards I was wondering why you chose to put such an emphasis on segmentation labelling. Do you see this usecase as the Computer Vision application with the biggest (future) market or maybe the least saturated offering at the moment?

[bertdb → janhenr]

Thanks! The existing tools on the market for image segmentation are not very sophisticated, so it's a niche where we can immediately make a difference.

In a sense, image segmentation labels are strictly more informative than bounding box labels: you can trivially extract the containing bounding box from a segmentation mask. One big reason that segmentation labels are not used more often, is simply because they are too expensive. Labeling a bounding box requires only two clicks, while labeling a segmentation mask requires much more time with manual tools. We're trying to solve that problem.

In the future we want to dig even deeper into this problem, and expand our scope to video and 3D segmentation labeling. We believe there will be a huge need for such tools now that everyone is getting smartphones with Lidar and AR/VR capabilities in their pockets.

[p_papageorgiou]

Hey! Great idea... It's a big pain to label properly but is there something extra from something like labelbox.com?

[p_papageorgiou → p_papageorgiou]

Or aquariumlearning.com

[bertdb → p_papageorgiou]

Our biggest differentiator is our strong focus on image segmentation: we've put a lot of effort in creating a labeling interface that is optimized to speed up segmentation labeling, a task that is notoriously slow and expensive. Another thing we do differently is that we allow unlimited labeling for free in public datasets.

Acquarium is focused more on exploring and curating your data. It integrates with external labeling providers, like us.

[the_cat_kittles]

man... every time i label images i think to myself "why cant i use the model to guess ahead the regions and then just correct them"... such a no brainer. but your implementation based on the video is so much more elegant than i ever thought of. well done!

[ottodebals → the_cat_kittles]

Thanks - feel free to give it a try and let us know your suggestions!

[panabee]

hi there! this looks awesome. we need something like this but for image matting. is image matting on your roadmap?

[bertdb → panabee]

Happy to listen to what you need, feel free to shoot me an email.

[_7bxa]

How does this compare to scale.ai?

[bertdb → _7bxa]

Scale only provides labeling services and does not provide labeling software that you can use with your own in-house or dedicated workforce team. We offer both, and our labeling service is even a bit cheaper thanks to our focus on speeding up labeling with our technology.

[gkk]

Hi,

Excited to see you launching this! I agree on the basic premise: existing tools for segmentation labeling leave copious room for an improvement.

I just gave Segments a spin with an image data I work on at the moment. First impressions:

1. When trying to connect segments (by dragging), I seem to lose the original segment

2. Your model seems to be confused by noisy data that I happened to upload - it's a microscopy image. To a human eye it's quite clear what the areas of interest are.

[bertdb → gkk]

Thanks for your feedback!

1. If the segment you start dragging from is already selected, all the segments you drag through will get deselected, and vice versa.

2. Did you try changing the granularity of the segments by scrolling your mouse wheel? We've had good experiences with microscopic imagery before, happy to connect and dig a bit deeper.

[gkk → bertdb]

Thanks for a quick reply!

1. Oh, I see. I didn't guess that's the intended behaviour. I wonder if it's not too clever.

2. Yes, then segments get too "excited" about the background noise. I would be able to make it work but with loads of manual tweaking which is, as I understand, the pain Segments wants to alleviate.

[bertdb → gkk]

The segments you see on the screen are generated by our ML model. If your data is very noisy, our out-of-the-box model might not be the best fit. We can always improve performance by training a custom model for you on a small set of manually labeled data though.

[Riegerb]

Hey Bert & Otto. It's Brian from Labelbox.

Congrats on YC! I'm excited to see what you build next.

[panabee → Riegerb]

hi brian.

will labelbox offer something around image matting?

we need a more precise GT for our datasets.

image segmentation would help, but we would love to automate/outsource the whole image matting process.

thanks.

[bertdb → Riegerb]

Thanks Brian!