How to: Your guide to Artificial Intelligence on AWS in 2021
AWS offers the broadest and deepest set of machine learning services and supporting cloud infrastructure, putting machine learning in the hands of every developer, data scientist and expert practitioner. To remove the complexity of getting started with machine learning, AWS developed the Sagemaker platform to help customers get started using AI to their advantage, with no experience.With AWS Sagemaker, customers can access the same algorithms used to create Alexa’s voice application, or search recommendations.
Amazon SageMaker is AWS’ fully managed machine learning service. Using SageMaker, teams can have access to quickly build and train machine learning models, deploying them into a production-ready hosted environment. The platform also contains common machine learning algorithms, optimal for efficiency of large datasets in distributed environments. It is built with flexible distributed training options to adjust to varying workflows, and supports bring-your-own-algorithms and frameworks. Deploy a model into a secure and scalable environment by launching it with a few clicks from SageMaker Studio or the SageMaker console.
If you’re new to Sagemaker, Amazon SageMaker Autopilot simplifies the machine learning experience by automating machine learning tasks. It also provides visibility into code generated to automate ML tasks and can be utilized as a resourceful ML learning tool for teams looking to deepen AI knowledge.
Before you can use Amazon SageMaker, you must sign up for an AWS account, create an IAM admin user, and onboard to Amazon SageMaker Studio. From here, you can access Amazon’s free training to help make your first model using SageMaker Studio, or the SageMaker console and the SageMaker API, complete with training algorithms provided by SageMaker.
Depending on your business goals, SageMaker can be used to help you accomplish the following Machine Learning tasks:
1.Use SageMaker to train and deploy your own custom algorithms – Package your custom algorithms with Docker so you can train and/or deploy them in SageMaker. To learn how SageMaker interacts with Docker containers, and for the SageMaker requirements for Docker images, see Using Docker containers with SageMaker .
- Submit Python code to train with deep learning frameworks – In SageMaker, you can use your own training scripts to train models. For information, see Use Machine Learning Frameworks, Python, and R with Amazon SageMaker.
- Pre-process data for ML training - In SageMaker, you preprocess example data in a Jupyter notebook on your notebook instance. You use your notebook to fetch your dataset, explore it, and prepare it for model training. For more information, see Explore, Analyze, and Process Data. For more information about preparing data in AWS Marketplace, see data preparation.
After you train your model, you can deploy it using Amazon SageMaker to help collect data in any of the following ways:
- Set up an endpoint and receive a single prediction at a time using SageMaker hosting services.
- Find predictions for an entire dataset, use SageMaker batch transform.
After deploying a model, you need to evaluate for identity drift, interferences, and deduce your ground truth. From here, you can increase the accuracy of your inferences by updating training data to consider the newly collected ground truth, retraining the model with new data sets. Machine learning is a continuous cycle; as more example data becomes available, you continue refining your models for higher accuracy.
Add a Human View to Data: Amazon Augmented AI (A2I)
Amazon Augmented AI (Amazon A2I) makes it easy to build the workflows that enable human review of ML predictions. Amazon A2I offers all developers the potential of human review, removing the arduous processes associated with building human review systems and/or the trouble of managing 50 human reviewers.
Building human review systems is costly and time-inefficient due to the complex workflows and custom software needed to manage results and tasks, plus managing a large group of human reviewers. Many applications relying on machine learning require humans to review low-confidence predictions for accuracy. For example, extracting information from scanned loan application forms may require human review due to suspicious handwriting, low-quality scans, etc. Companies can use Amazon A2I to simplify this process. A few out of box human-review capabilities include:
Use Amazon A2I with Amazon Textract – Have humans review single page documents to review important form key value pairs, or have Amazon Textract randomly sample and send documents from your dataset for human review.
- Use Amazon A2I with Amazon Rekognition– Have humans review questionable images for explicit/violent/adult content if Amazon Rekognition returns a low confidence score, or have Amazon Rekognition randomly sample and send images from your dataset to humans for review.
- Use Amazon A2I to review real time ML inferences – Use Amazon A2I to review low-confidence inferences made by a model in real time, deployed to an SageMaker hosted endpoint and incrementally train your model using Amazon A2I output data.
- Use Amazon A2I with Amazon Comprehend – Have humans review Amazon Comprehend inferences about text data such as sentiment analysis, text syntax, and entity detection.
Amazon A2I provides built-in human review workflows (see full list here) for common machine learning use cases, such as content moderation and text extraction from documents, which allows predictions from Amazon Rekognition and Amazon Textract to be reviewed easily. You can also create your own workflows for ML models built on SageMaker or any other tools. Using Amazon A2I, you can allow human reviewers to step in when a model is unable to make a high-confidence prediction or to audit its predictions on an ongoing basis.
Using AWS Sagemaker, getting started with artificial intelligence and implementing it in your environment is made possible by building pre-built algorithms. You can continue to refine algorithms with a range of services available with AWS Sagemaker, including Amazon Augmented AI (Amazon A2I) to build efficient human-review into your machine learning processes. With AWS, businesses have access to the advantages of artificial intelligences to enhance processes, services and products.
AWS pre-trained AI Services provide ready-made intelligence for your applications and workflows. For a full list of services head to the full directory at Amazon. Named a leader in Gartner's Cloud AI Developer services' Magic Quadrant, AWS is helping tens of thousands of customers accelerate their machine learning journey.
Stratus10 has helped customers use AWS to leverage ML to achieve business goals in a range of environments. To start leveraging Stratus10’s expertise now in DevOps, AI/ML ,and app modernization, get in touch with someone from our team today.