Search
Patexia Research
Patent No. US 11151479
Issue Date Oct 19, 2021
Claim this patent
PDF Unavailable

Patent 11151479 - Automated computer-based model development, deployment, and management > Claims

  • 1. A system comprising: one or more processing devices; andone or more memory devices including instructions executable by the one or more processing devices for causing the one or more processing devices to: receive a request from a client device to build a machine-learning model for use in performing a task as part of a first project, the request including one or more parameters associated with the machine-learning model;in response to receiving the request, select a first template that is compatible with the one or more parameters included in the request;based on selecting the first template, convert the first template into first executable code by incorporating the one or more parameters into the first template;based on converting the first template into first executable code, provide the first executable code as input to a first model-building tool for causing the first model-building tool to build a first machine-learning model in accordance with the one or more parameters, wherein the first model-building tool is software;based on building the first machine-learning model, incorporate the first machine-learning model into the first project; andsubsequent to incorporating the first machine-learning model into the first project: select a second template that is compatible with the one or more parameters included in the request;based on selecting the second template, convert the second template into second executable code by incorporating the one or more parameters into the second template;based on converting the second template into the second executable code, provide the second executable code as input to a second model-building tool for causing the second model-building tool to build a second machine-learning model based on the one or more parameters, wherein the second machine-learning model is different from the first machine-learning model, and wherein the second model-building tool is software; andincorporate the second machine-learning model into a second project for use in performing the task as part of the second project.
    • 2. The system of claim 1, wherein the first machine-learning model is a first version of the machine-learning model, and the second machine-learning model is a second version of the machine-learning model.
      • 3. The system of claim 2, wherein the first project is a first version of a project, and the second project is a second version of the project.
        • 4. The system of claim 3, wherein the one or more memory devices further include instructions that are executable by the one or more processing devices for causing the one or more processing devices to, subsequent to incorporating the first machine-learning model into the first project: monitor the system to detect an integration of new software into the system; andin response to detecting the integration of the new software into the system: automatically generate the second machine-learning model in accordance with the one or more parameters using the second model-building tool; andincorporate the second machine-learning model into the second project.
          • 5. The system of claim 4, wherein the new software is the second model-building tool.
    • 6. The system of claim 1, wherein the one or more memory devices further include instructions that are executable by the one or more processing devices for causing the one or more processing devices to, prior to incorporating the first machine-learning model into the first project: execute a plurality of model-building tools to build a plurality of machine-learning models based on the one or more parameters, the plurality of model-building tools being software for building machine-learning models;compare performance characteristics of the plurality of machine-learning models to one another to determine a candidate champion model from among the plurality of machine-learning models; andbased on determining the candidate champion model, initiate an automated champion-model approval process with respect to the candidate champion model to determine whether the candidate champion model is approved for use in the first project, wherein the automated champion-model approval process involves the one or more processing devices determining whether the candidate champion model satisfies a predefined criterion.
      • 7. The system of claim 6, wherein the one or more memory devices further include instructions for implementing the automated champion-model approval process, the instructions being executable by the one or more processing devices for causing the one or more processing devices to: determine that the candidate champion model is not approved for use in the first project;in response to determining that the candidate champion model is not approved for use in the first project, determine a champion model that satisfies the predefined criterion, wherein the champion model is the first machine-learning model; andselect the champion model for use in the first project.
        • 8. The system of claim 7, wherein the one or more memory devices further include instructions that are executable by the one or more processing devices for causing the one or more processing devices to determine the champion model by: selecting a new candidate champion-model from among the plurality of machine-learning models, the new candidate champion-model being different from the candidate champion model, wherein the new candidate champion-model is the first machine-learning model;executing the automated champion-model approval process with respect to the new candidate champion-model to determine whether the new candidate champion-model satisfies the predefined criterion; andin response to determining that the new candidate champion-model satisfies the predefined criterion, selecting the new candidate champion-model as the champion model for use in the first project.
        • 9. The system of claim 7, wherein the one or more memory devices further include instructions that are executable by the one or more processing devices for causing the one or more processing devices to: based on selecting the champion model for use in the first project, publish the champion model to a production environment that is accessible to the client device.
      • 10. The system of claim 6, wherein the one or more memory devices further include instructions that are executable by the one or more processing devices for causing the one or more processing devices to: determine, via the automated champion-model approval process, that the candidate champion model is approved for use in the first project, wherein the candidate champion model is the first machine-learning model; andselect the candidate champion model for use in the first project.
      • 11. The system of claim 6, wherein the performance characteristics include accuracies of the plurality of machine-learning models, and wherein the predefined criterion involves the candidate champion model having an accuracy that exceeds an accuracy threshold.
      • 12. The system of claim 6, wherein the performance characteristics include memory consumption associated with the plurality of machine-learning models, and wherein the predefined criterion involves the candidate champion model consuming an amount of memory that is below a memory usage threshold.
      • 13. The system of claim 6, wherein the performance characteristics include computation time associated with the plurality of machine-learning models, and wherein the predefined criterion involves the candidate champion model having a computation time that is below a computation-time threshold.
      • 14. The system of claim 6, wherein the performance characteristics include processing power associated with the plurality of machine-learning models, and wherein the predefined criterion involves the candidate champion model consuming an amount of processing power that is below a processing-power threshold.
      • 15. The system of claim 6, wherein the predefined criterion involves the candidate champion model being compliant with a legal standard.
      • 16. The system of claim 6, wherein the one or more memory devices further include instructions that are executable by the one or more processing devices for causing the one or more processing devices to: select a plurality of templates from a template repository based on the plurality of templates being compatible with the one or more parameters; andgenerate the plurality of machine-learning models using the plurality of templates.
    • 17. The system of claim 1, wherein the one or more memory devices further include instructions that are executable by the one or more processing devices for causing the one or more processing devices to, subsequent to incorporating the first machine-learning model into the first project: determine a key performance metric with respect to the first machine-learning model;determine a performance score for the first machine-learning model based on the key performance metric;determine whether the performance score satisfies a preset criterion; andexecute one or more operations based on whether the performance score satisfies the preset criterion.
    • 18. The system of claim 1, wherein the first machine-learning model is a first version of the machine-learning model, the first project is a first version of a project, and the one or more memory devices further include instructions that are executable by the one or more processing devices for causing the one or more processing devices to, subsequent to including the first machine-learning model into the first project: detect an event; andin response to detecting the event: automatically generate a new version of the machine-learning model;generate a new version of the project that includes the new version of the machine-learning model; andretire the first version of the project that includes the first machine-learning model.
      • 19. The system of claim 18, wherein the event is a change in a regulation or a law existing outside the system.
    • 20. The system of claim 1, wherein the one or more memory devices further include instructions that are executable by the one or more processing devices for causing the one or more processing devices to select the second template based on the request by: extracting the one or more parameters from the request; andusing the one or more parameters extracted from the request to select the second template.
  • 21. A method comprising: receiving, by one or more processing devices of a system, a request from a client device to build a machine-learning model for use in performing a task as part of a first project, the request including one or more parameters associated with the machine-learning model;in response to receiving the request, selecting, by the one or more processing devices, a first template that is compatible with the one or more parameters included in the request;based on selecting the first template, converting, by the one or more processing devices, the first template into first executable code by incorporating the one or more parameters into the first template;based on converting the first template into first executable code, providing, by the one or more processing devices, the first executable code as input to a first model-building tool for causing the first model-building tool to build a first machine-learning model based on the one or more parameters, wherein the first model-building tool is software;based on building the first machine-learning model, incorporating, by the one or more processing devices, the first machine-learning model into the first project; and subsequent to incorporating the first machine-learning model into the first project:selecting, by the one or more processing devices, a second template that is compatible with the one or more parameters included in the request;based on selecting the second template, converting, by the one or more processing devices, the second template into second executable code by incorporating the one or more parameters into the second template;based on converting the second template into the second executable code, providing, by the one or more processing devices, the second executable code as input to a second model-building tool for causing the second model-building tool to build a second machine-learning model based on the one or more parameters, wherein the second machine-learning model is different from the first machine-learning model, and wherein the second model-building tool is software; andincorporating, by the one or more processing devices, the second machine-learning model into a second project for use in performing the task as part of the second project.
    • 22. The method of claim 21, further comprising, subsequent to incorporating the first machine-learning model into the first project: monitoring the system to detect an integration of new software into the system; andin response to detecting the integration of the new software into the system: automatically generating the second machine-learning model in accordance with the one or more parameters using the second model-building tool; andincorporating the second machine-learning model into the second project.
    • 23. The method of claim 21, further comprising, prior to incorporating the first machine-learning model into the first project: executing, by the one or more processing devices, a plurality of model-building tools to build a plurality of machine-learning models based on the one or more parameters, the plurality of model-building tools being software for building machine-learning models;comparing, by the one or more processing devices, performance characteristics of the plurality of machine-learning models to one another to determine a candidate champion model from among the plurality of machine-learning models; andbased on determining the candidate champion model, initiating, by the one or more processing devices, an automated champion-model approval process with respect to the candidate champion model to determine whether the candidate champion model is approved for use in the first project, wherein the automated champion-model approval process involves automatically: determining, by the one or more processing devices, a characteristic of the candidate champion model; anddetermining, by the one or more processing devices, whether the characteristic of the candidate champion model satisfies a predefined criterion.
      • 24. The method of claim 23, further comprising: determining, by the one or more processing devices and through the automated champion-model approval process, that the candidate champion model is not approved for use in the first project;in response to determining that the candidate champion model is not approved for use in the first project, determining, by the one or more processing devices, a champion model that satisfies the predefined criterion, wherein the champion model is the first machine-learning model; andin response to determining the champion model, selecting, by the one or more processing devices, the champion model for use in the first project.
        • 25. The method of claim 24, further comprising determining the champion model by: in response to determining that the candidate champion model is not approved for use in the first project, selecting, by the one or more processing devices, a new candidate champion-model from among the plurality of machine-learning models, the new candidate champion-model being different from the candidate champion model, wherein the new candidate champion-model is the first machine-learning model;executing, by the one or more processing devices, the automated champion-model approval process with respect to the new candidate champion-model to determine whether the new candidate champion-model satisfies the predefined criterion; andin response to determining that the new candidate champion-model satisfies the predefined criterion, selecting, by the one or more processing devices, the new candidate champion-model as the champion model for use in the first project.
        • 26. The method of claim 24, further comprising: based on selecting the champion model for use in the first project, publishing the champion model to a production environment that is accessible to the client device.
      • 27. The method of claim 23, further comprising: determining, by the one or more processing devices and through the automated champion-model approval process, that the candidate champion model is approved for use in the first project, wherein the candidate champion model is the first machine-learning model; andin response to determining that the candidate champion model is approved, selecting, by the one or more processing devices, the candidate champion model for use in the first project.
      • 28. The method of claim 23, further comprising: selecting a plurality of templates from a template repository based on the plurality of templates being compatible with the one or more parameters; andgenerating the plurality of machine-learning models using the plurality of templates.
    • 29. The method of claim 21, wherein the first machine-learning model is a first version of the machine-learning model, the first project is a first version of a project, and further comprising, subsequent to including the first machine-learning model into the first project: detecting an event; andin response to detecting the event: automatically generating a new version of the machine-learning model;generating a new version of the project that includes the new version of the machine-learning model; andretiring the first version of the project that includes the first machine-learning model.
  • 30. A non-transitory computer-readable medium comprising program code that is executable by one or more processing devices for causing the one or more processing devices to: receive a request from a client device to build a machine-learning model for use in performing a task as part of a first project, the request including one or more parameters specifying (i) a type of the machine-learning model, (ii) a number of hidden layers to include in the machine-learning model, (ii) a number of nodes to include in the machine-learning model, or (iv) a number of connections to include in the machine-learning model;in response to receiving the request, select a first template based on the one or more parameters included in the request;based on selecting the first template, convert the first template into first executable code;based on converting the first template into first executable code, provide the first executable code as input to a first model-building tool for causing the first model-building tool to build a first machine-learning model based on the one or more parameters, wherein the first model-building tool is software;based on building the first machine-learning model, incorporate the first machine-learning model into the first project;subsequent to incorporating the first machine-learning model into the first project, select a second model-building tool based on the one or more parameters, wherein the second model-building tool is software;in response to selecting the second model-building tool, provide second executable code generated using a second template as input to the second model-building tool for causing the second model-building tool to build a second machine-learning model based on the one or more parameters, wherein the second machine-learning model is different from the first machine-learning model; andincorporate the second machine-learning model into a second project for use in performing the task as part of the second project.
Menu