Search
Patexia Research
Patent No. US 10839268
Issue Date Nov 17, 2020
Claim this patent
PDF Unavailable

Patent 10839268 - Artificial intelligence adversarial vulnerability audit tool > Claims

  • 1. A computer-implemented method for characterizing the robustness of a computer-vision machine learning model comprising: receiving an image with a known, first classification by the machine learning model;iteratively modifying the image using at least one perturbation algorithm and inputting the corresponding modified image into the machine learning model until the machine learning model outputs a second classification different from the first classification; andproviding data characterizing the modifications to the image that resulted in the machine learning model outputting the second classification;wherein the at least one perturbation algorithm uses gradient-based optimization to determine how to modify the image.
    • 2. The method of claim 1, wherein the providing data comprises at least one of: causing the data to be displayed in a graphical user interface, loading the data into memory, storing the data in physical persistence, or transmitting the data to a remote computing device.
    • 3. The method of claim 1, wherein the provided data characterizing the modifications explains how the modifying caused the machine learning to change its classification from the first classification to the second classification.
    • 4. The method of claim 1, wherein the at least one perturbation algorithm further uses reinforcement learning to determine how to modify the image.
    • 5. The method of claim 1, wherein the at least one perturbation algorithm further uses a random selection algorithm to determine how to modify the image.
    • 6. The method of claim 1, wherein the at least one perturbation algorithm further uses a pattern matching algorithm to determine how to modify the image.
      • 7. The method of claim 6, wherein the pattern matching uses an Apriori algorithm.
    • 8. The method of claim 1, wherein the iterative modifying of the image simulates an attack on the machine learning model.
      • 9. The method of claim 8, wherein the attack simulates a change in an environmental condition of a sensor obtaining the image.
      • 10. The method of claim 8, wherein the attack simulates a change in a position of a sensor obtaining the image.
      • 11. The method of claim 8, wherein the attack simulates a modification of a physical object.
        • 12. The method of claim 11 further comprising: printing an overlay comprising the modification.
          • 13. The method of claim 12, wherein the overlay is printed on a sticker which can be affixed to the physical object.
  • 14. A system for characterizing the robustness of a computer-vision machine learning model comprising: at least one data processor; andmemory storing instructions which, when executed by the at least one data processor, result in operations comprising: receiving an image with a known, first classification by the machine learning model;iteratively modifying the image using at least one perturbation algorithm and inputting the corresponding modified image into the machine learning model until the machine learning model outputs a second classification different from the first classification; andproviding data characterizing the modifications to the image that resulted in the machine learning model outputting the second classification;wherein the at least one perturbation algorithm uses gradient-based optimization to determine how to modify the image.
    • 15. The system of claim 14, wherein the provided data characterizing the modifications explains how the modifying caused the machine learning to change its classification from the first classification to the second classification.
    • 16. The system of claim 14, wherein the at least one perturbation algorithm further uses reinforcement learning to determine how to modify the image, a random selection algorithm or a pattern matching algorithm to determine how to modify the image.
    • 17. The system of claim 14, wherein the iterative modifying of the image simulates an attack on the machine learning model.
      • 18. The system of claim 17, wherein the attack simulates at least one of, a change in an environmental condition of a sensor obtaining the image, a change in a position of a sensor obtaining the image, or a modification of a physical object.
  • 19. A non-transitory computer program product storing instructions which, when executed by at least one computing device, implement operations comprising: receiving an image with a known, first classification by the machine learning model;iteratively modifying the image using at least one perturbation algorithm and inputting the corresponding modified image into the machine learning model until the machine learning model outputs a second classification different from the first classification; andproviding data characterizing the modifications to the image that resulted in the machine learning model outputting the second classification;wherein the at least one perturbation algorithm uses gradient-based optimization to determine how to modify the image.
Menu