Request: Support Image segmentation

[Luk164]

Is there, or will there be a way to use ML.net for object instance segmentation in an image? Or do I have to use tensorflow wrapper directly for that?

[antoniovs1029]

Hi @Luk164 . Currently ML.NET doesn't provide any transformer to do object segmentation.

Still, we do provide tools to run pretrained tensorflow models or onnx models. If you have a pretrained model that can do image segmentation, then you'd be able to make it run on ML.NET with the right setup. Please, refer to our sample repository to see how to run Tensorflow or Onnx models. I believe there's no sample on segmentation, but I'd recommend this other two samples as they also show how to work with images:

Image classification with Tensorflow in ML.NET

Object detection with ONNX and ML.NET

I'll tag this issue as a feature request so that we can have it in mind during the planning of future features for ML.NET.

[Luk164]

@antoniovs1029 Thank you, I thought that would be the case but I wanted to make sure. So I will have to use sciSharp repositories directly, will probably use Keras as it is still a framework over tensorflow so it should make stuff easier. Any recommendations? I am pretty new to this.

[BernhardGlueck]

Just wanted to give this a thumbs up.
We are training diverse Mask-RCNN models with different coco format datasets in Python currently... and run them in ML.NET
It would be awesome to get rid of the only python part in our backend pipeline, by being able to train those models in ML.NET directly...

Mask-RCNN, or CenterMask/CentermaskLite would be awesome...

[edwardzxw]

Sorry for digging an old post, I'd like to ask whether theres' any update or plan for this feature yet?

[waltergrimmvie]

HI,

I would like to enquire if there are any updates on how to run image segmentation with a Yolo8 pretrained modell, eg coloration of a single butterfly in ML.NET

thanx