Skip to content
  • Products
    • Portfolio overview >

      Flexsave™

      Automatically manage cloud compute for optimized costs and agility

      Cloud Analytics

      Make sense of spend and chargeback to align with your business

      google cloud msp

      BigQuery Lens

      Optimize BigQuery costs with actionable recommendations and usage insights

      Spot Scaling

      Maximize AWS Spot savings and minimize disruptions for optimized scaling

      Anomaly Detection

      Autonomously identify cost spikes early, with zero configuration

      Platform overview >

      Organize your billing data for better business decisions

      Integrations >

      Connect your favorite tools to DoiT’s products

  • Services
    • Services overview >

      How we work

      Learn how we’re redefining support with our customer reliability engineering

      Stats

      View our live support and customer satisfaction statistics in real-time

      Cloud solutions

      Proven solutions to cloud complexity

      FinOps

      Learn how DoiT enables critical FinOps capabilities

      Areas of expertise

      Cloud Architecture

      Ensure your cloud architecture is future-ready and built for success

      Cloud Cost Optimization

      Identify opportunities to optimize costs and target spend for added value

      Cloud Migration

      Realize greater efficiency and innovation with successful cloud migration

      Cloud Security

      Center security in your cloud strategy to ensure ongoing efficacy and growth

      Data and Analytics

      Harness the potential of big data and analytics to gain a competitive edge

      Data Management

      Build your data practice with expert guidance tailored to your business goals

      DevOps Jump Start

      Accelerate your AWS workloads & release pipelines while also increasing automation, monitoring & reliability

      Infrastructure

      Maximize the full suite capabilities from your cloud infrastructure

      Kubernetes

      Manage the complexity of Kubernetes to enable innovation and scalability

      Location-Based Services

      Transform geolocational data into real-world, real-time intelligence

      Machine Learning

      Level-up key data with ML capabilities that accelerate innovation

      Multicloud

      Create meaningful business value with a robust multicloud strategy

      Training

      Build skills and capability across teams with certified, expert-led training

  • Partners
    • Alliances

      Proud to be an award‒winning multicloud partner to top‒tier cloud providers

      doit-together

      DoiT Together

      Enabling cloud growth and unlocking revenue through expert partnership

      ISV Go-Global

      Accelerate new customer growth and Marketplace integration on AWS and GCP

  • Resources
    • Resources hub >

      Blog

      Read the latest insights, tips and perspectives from our team of cloud experts

      Case Studies

      See how we’ve helped thousands of public cloud customers achieve their goals

      Cloud Masters Podcast

      Listen to our experts and customers share tangible tips for navigating the cloud.

      Ebooks and Guides

      Discover foundational expertise and future-ready recommendations for the cloud

      Events and Webinars

      Tech talks and interactive expert sessions delivered both virtually and in person

      GCPInstances.info

      Google Cloud Compute Engine instance comparison

      Help center

      Read documentation, product updates, and more

      Newsroom

      See what's new from DoiT in our latest news and announcements

      Trust Center

      How we focus on security, compliance, and privacy

      Videos

      Watch product demos, interviews and more from our cloud experts

  • About
    • About DoiT >

      Careers

      Browse our open positions and learn more about what it takes to be a Do’er

      Leadership

      Meet the team leading DoiT and our customers on a journey of hypergrowth

      Newsroom

      See what's new from DoiT in our latest news and announcements

  • Pricing
  • Contact us
  • Sign In
  • Products
    • Flexsave ™
    • Cloud Analytics
    • Spot Scaling
    • BigQuery Lens
    • Anomaly Detection
    • DoiT Platform
    • Integrations
  • Services
    • How We Work
    • Stats
    • Cloud Solutions
    • FinOps
    • Areas of expertise
      • Cloud Architecture
      • Cloud Cost Optimization
      • Cloud Migration Consulting Services
      • Cloud Security
      • Data and Analytics
      • Data Management
      • DevOps with AWS & DoiT
      • Infrastructure
      • Kubernetes
      • Location Based Services
      • Machine Learning
      • Multicloud
      • Training
  • Partners
    • ISV Go-Global
    • Award-winning public cloud partner
    • DoiT Together
  • Resources
    • Blog
    • Case Studies
    • Cloud Masters Podcast
    • Ebooks and Guides
    • Events and Webinars
    • GCPInstances.info
    • Help center
    • Newsroom
    • Trust Center
    • Videos
  • Pricing
  • About
    • Careers
    • Leadership
    • Newsroom
  • Contact us
  • Sign In
Contact us
Sign in

Blog

AWS Partner Black Belt in Machine Learning

  • Joshua Fox Joshua Fox
  • Date: April 29, 2021

1 grijjhqnffsesnnqi1lhea

Or, how we did a pilot of this advanced certification program

Authored with my colleagues at DoiT International, Gad Benram and Darren Brien

Introduction

Recently, the three of us joined a pilot run of the AWS Partner Black Belt program in Machine Learning and Artificial Intelligence and earned our certifications. In this post, we’ll describe the program, including the learning sessions and the final project.

In the past, we have earned certifications from cloud providers, including the AWS Certified Machine Learning Specialty. The Black Belt is a higher level of certification that represents deep experience rather than potentially crammed knowledge. Instead of a multiple-choice exam, it is evaluated with a final capstone project, roughly at the difficulty level of a university term project.

Participants were ML practitioners working in the AWS cloud. The instructors and evaluators were AWS architects and product managers specializing in the field.

Our specializations at DoiT International differ from cloud infrastructure to ML, and our responsibilities range from advising customers to developing ML systems. Still, the program was suited to each of us, and perhaps to you too.

Lectures and Labs

Below, we’ll describe some of the essential elements of AWS ML that we covered. If you already know all this — great! Passing will be a cinch! But more likely, even if you have years of experience, you have not worked with the full range of AWS offerings across the ML workflow, and this is your chance to learn.

The program starts with online learning sessions, about 10 hours a week for four weeks. These include lectures, structured walkthroughs of technologies, and live labs. The program’s second half is the capstone project, with about that same time commitment.

Give me some data

The first step towards ML projects is to ask: “Where’s your data?”

AWS has several solutions for ingesting, discovering, and transforming data to make it readily available and usable for ML projects.

The Kinesis and Glue service suites form a powerful combination for serverless ingestion and transformation as data arrives in real-time. Another solution for transformation is EMR, a managed Hadoop ecosystem. All these generally put their output in S3, the go-to data storage option in Machine Learning.

The next step in data management is building a massive collection of data that can be analyzed for business insights and patterns useful in ML training. Glue Data Catalog captures your data’s location and schema, while Lake Formation sets up paths for your data together with granular sets of permissions.

Next, you can analyze the data with SQL using the powerful engines of Athena on top of S3. Alternatively, instead of a lake of unstructured data in S3, you can use Redshift as a structured data warehouse.

Let Autopilot fly for you

Sagemaker Autopilot might let you skip a few steps. This tool chooses an algorithm that is likely to work and tries it across many different parameter values. It’s helpful for a quick first run, but you will get more accurate and less costly answers if you explore and prepare the data first, setting it up in a way that your algorithm can get its teeth into.

Explore before

You have to understand what you’re going to do before you do it, and that step is called Exploratory Data Analysis.

Visualization of full datasets can be done in the QuickSight business intelligence tool, but most EDA happens in Sagemaker Notebook, an interactive development environment for Python aimed at ML. You can preprocess, analyze, explore the data, and even run your modeling and evaluation with fast iterations, in preparation for using AWS’s scalable distributed services.

Can we finally do some ML, please?

At last, time to choose and run a machine learning algorithm. Amazon Sagemaker offers a range of algorithms out of the box: DeepAR, XGBoost, and about a dozen more, each with its own use cases. For better tuning, Sagemaker hyperparameter optimization lets you automatically experiment across values in multiple dimensions. If you’ve developed your own algorithms, you can bundle them into an ML container, and integrate them into the full power of the Sagemaker system.

Machine Learning Ops on AWS

The last part of the Black Belt program focused on delivering models into production: ML Ops. Amazon’s approach to this rapidly evolving area is an ML-centric AWS Well-Architected Five Pillars, based on practices that have become standard in software engineering.

The model is a-changing

As new data arrives, the inferences should reflect changes. Sagemaker Pipelines automates this process of training and deployment, creating versioning for APIs and models. The versions can be run alongside each other for end-to-end A/B testing. Model performance monitoring lets you see if the accuracy has slipped enough that retraining is needed.

Security is job zero

To control access on the network level, Sagemaker Notebooks, training jobs, and inference endpoints can each be deployed into your VPC. You also control access to each of these on the user level using IAM.

Keep it going

Sagemaker inferencing endpoints are available for on-demand or batch service. Sagemaker keeps these highly available with autoscaling and automatic failover as needed.

Capstone project

Implementing a solution is the best way to crystalize your learning; it is also the best way to show off your skills. We were excited to find that the Black Belt program included not only live labs, but also a four-week Capstone project. Each participant chose one of the ML projects in areas such as anomaly detection and preventive maintenance. The AWS team provided thoughtful guidance along the way, going the extra mile to make themselves available to us on Slack.

Our projects were graded against a comprehensive rubric, a grading guide listing technologies and solutions to be used across all aspects of delivering an ML system from ingestion to transformation, training, evaluation, and on to ML Ops.

Conclusion

We’re grateful for the support and guidance of the AWS team: Designing the program, delivering the training, and reviewing the capstone projects was no small undertaking.

The AWS AI/ML Black Belt is both an exclusive certification and a different way of learning. We recommend it to anyone who wants to take a step up in ML and demonstrate this to the world.


Thanks for reading! To stay connected, follow us on the DoiT Engineering Blog, DoiT Linkedin Channel, and DoiT Twitter Channel. To explore career opportunities, visit https://careers.doit.com.

Subscribe to updates, news and more.

Subscribe

Subscribe to updates, news and more.

Related blogs

3 Tips to Improve Your AWS Cost Optimization Strategies

Global cloud spend is expected to approach the $600 billion mark in 2023 – 21% growth over the previous

Keep reading 

Easing the Burden of Cloud Commitment Management

Automating your commitment strategy reduces risk and eases your management burden.

Keep reading 

DoiT International Achieves AWS DevOps Competency

A Premier Tier Services Partner of AWS, DoiT earns distinguished DevOps competency to help its customers implement business-critical DevOps

Keep reading 
View all blogs
Let’s do it

From cost optimization to cloud migration, machine learning and CloudOps, we’re here to make the public cloud easy — without the costs.

Ready to get started?

Get in touch

Company

  • About us
  • Blog
  • Careers
  • MS-HT statement
  • Newsroom
  • Privacy policy
  • Terms

Offering

  • Compliance
  • Products
  • Services
  • Solutions
  • Resources

Support

  • Sign in
  • Help center
  • Open ticket
  • Contact us

Never miss an update.

Subscribe to our newsletter

Subscribe to updates, news and more.