Search
Patexia Research
Issue Date Jul 29, 2021
Claim this application
PDF Unavailable

Patent Application - SYSTEM AND METHOD FOR IMAGE SEGMENTATION USING A JOINT DEEP LEARNING MODEL > Claims

  • 1. A method, comprising: receiving an input image;processing the input image with a deep learning model to generate a segmented image, wherein weights of the deep learning model are updated during training of the deep learning model according to a custom loss function determined based on segmented images output by the deep learning model, autoencoder output images generated by inputting the segmented images to a shape regularization network, and ground truth images; andpresenting the segmented image via a display unit.
    • 2. The method of claim 1, wherein the deep learning model comprises a triad of predictors including a shape model, a foreground model, and a background model, wherein respective weights of the triad of predictors are updated during joint training of the triad of predictors of the deep learning model according to a joint loss function including the custom loss function, wherein the input image is processed with the shape model of the deep learning model to generate the segmented image.
      • 3. The method of claim 2, further comprising processing the input image with the foreground model of the deep learning model to generate a foreground texture image, and processing the input image with the background model of the deep learning model to generate a background texture image.
        • 4. The method of claim 3, further comprising presenting one or more of the foreground texture image and the background texture image via the display unit.
        • 5. The method of claim 3, wherein processing the input image comprises generating at least one of the foreground texture image, the background texture image, and the segmented image using a parallel U-Net architecture comprising separate weights for each of the triad of predictors.
        • 6. The method of claim 3, wherein processing the input image comprises generating at least one of the foreground texture image, the background texture image, and the segmented image using a shared U-Net architecture comprising shared weights for the triad of predictors.
      • 7. The method of claim 2, wherein joint training of the triad of predictors of the deep learning model according to the joint loss function comprises simultaneously minimizing a foreground cost factor, a background cost factor, and a shape cost factor.
    • 8. The method of claim 1, wherein the shape regularization network comprises a convolutional de-noising autoencoder.
    • 9. The method of claim 1, wherein the custom loss function includes a projection error cost term based on a distance between a predicted shape and a shape space, a representation error cost term based on a distance between an encoded representation of a ground truth segmentation mask and an encoded representation of a predicted mask, and a Euclidean cost term based on a distance between the ground truth segmentation mask and the predicted mask.
  • 10. A system, comprising: an image acquisition unit configured to acquire an input image;a display unit;a deep learning unit comprising a deep learning model, wherein weights of the deep learning model are updated during training of the deep learning model according to a custom loss function determined based on segmented images output by the deep learning model, autoencoder output images generated by inputting the segmented images to a shape regularization network, and ground truth images, wherein the deep learning unit is configured to process the input image with the deep learning model to generate a segmented image; anda processor unit communicatively coupled to the deep learning unit and the display unit, wherein the processor unit is configured to present the segmented image via the display unit.
    • 11. The system of claim 10, wherein the deep learning model comprises a triad of predictors including a shape model, a foreground model, and a background model, wherein respective weights of the triad of predictors are updated during joint training of the triad of predictors of the deep learning model according to a joint loss function including the custom loss function, wherein the deep learning unit processes the input image with the shape model of the deep learning model to generate the segmented image.
      • 12. The system of claim 11, wherein the deep learning unit is further configured to: process the input image with the foreground model of the deep learning model to generate a foreground texture image; andprocess the input image with the background model of the deep learning model to generate a background texture image.
        • 13. The system of claim 12, wherein the processor unit is further configured to: present one or more of the foreground texture image and the background texture image via the display unit.
      • 14. The system of claim 11, wherein the deep learning unit is configured to perform the joint training of the triad of predictors of the deep learning model according to the joint loss function by simultaneously minimizing a foreground cost factor, a background cost factor, and a shape cost factor.
  • 15. A method, comprising: receiving an input image;obtaining a deep learning model comprising a triad of predictors including a shape model, a foreground model, and a background model, the deep learning model comprising a multi-channel fully convolutional neural network with the triad of predictors jointly trained according to a joint cost function;processing the input image with the shape model of the triad of predictors to generate a segmented image;processing the input image with the foreground model of the triad of predictors to generate a foreground texture image;processing the input image with the background model of the triad of predictors to generate a background texture image;presenting the segmented image via a display unit; andpresenting one or more of the foreground texture image and the background texture image via the display unit.
    • 16. The method of claim 15, wherein joint training of the triad of predictors of the deep learning model according to the joint loss function comprises simultaneously minimizing a foreground cost factor, a background cost factor, and a shape cost factor.
    • 17. The method of claim 15, wherein weights of the shape model of the deep learning model are updated during the joint training of the deep learning model according to a custom loss function for the shape model, the custom loss function determined based on segmented images output by the deep learning model, autoencoder output images generated by inputting the segmented images to a shape regularization network, and ground truth images.
      • 18. The method of claim 17, wherein the shape regularization network comprises a convolutional de-noising autoencoder.
      • 19. The method of claim 17, wherein the custom loss function includes a projection error cost term based on a distance between a predicted shape and a shape space, a representation error cost term based on a distance between an encoded representation of a ground truth segmentation mask and an encoded representation of a predicted mask, and a Euclidean cost term based on a distance between the ground truth segmentation mask and the predicted mask.
    • 20. The method of claim 15, wherein processing the input image comprises generating at least one of the foreground texture image, the background texture image, and the segmented image using a parallel U-Net architecture comprising separate weights for each of the triad of predictors, or using a shared U-Net architecture comprising shared weights for the triad of predictors.
Menu