Skip to content
  • Products
    • Portfolio overview >

      Flexsave™

      Automatically manage cloud compute for optimized costs and agility

      Cloud Analytics

      Make sense of spend and chargeback to align with your business

      google cloud msp

      BigQuery Lens

      Optimize BigQuery costs with actionable recommendations and usage insights

      Spot Scaling

      Maximize AWS Spot savings and minimize disruptions for optimized scaling

      Anomaly Detection

      Autonomously identify cost spikes early, with zero configuration

      Platform overview >

      Organize your billing data for better business decisions

  • Services
    • Services overview >

      How we work

      Learn how we’re redefining support with our customer reliability engineering

      Stats

      View our live support and customer satisfaction statistics in real-time

      Cloud solutions

      Proven solutions to cloud complexity

      FinOps

      Learn how DoiT enables critical FinOps capabilities

      Areas of expertise

      Cloud Architecture

      Ensure your cloud architecture is future-ready and built for success

      Cloud Cost Optimization

      Identify opportunities to optimize costs and target spend for added value

      Cloud Migration

      Realize greater efficiency and innovation with successful cloud migration

      Cloud Security

      Center security in your cloud strategy to ensure ongoing efficacy and growth

      Data and Analytics

      Harness the potential of big data and analytics to gain a competitive edge

      Data Management

      Build your data practice with expert guidance tailored to your business goals

      DevOps Jump Start

      Accelerate your AWS workloads & release pipelines while also increasing automation, monitoring & reliability

      Infrastructure

      Maximize the full suite capabilities from your cloud infrastructure

      Kubernetes

      Manage the complexity of Kubernetes to enable innovation and scalability

      Location-Based Services

      Transform geolocational data into real-world, real-time intelligence

      Machine Learning

      Level-up key data with ML capabilities that accelerate innovation

      Multicloud

      Create meaningful business value with a robust multicloud strategy

      Training

      Build skills and capability across teams with certified, expert-led training

  • Partners
    • Alliances

      Proud to be an award‒winning multicloud partner to top‒tier cloud providers

      doit-together

      DoiT Together

      Enabling cloud growth and unlocking revenue through expert partnership

      ISV Go-Global

      Accelerate new customer growth and Marketplace integration on AWS and GCP

  • Resources
    • Resources hub >

      Blog

      Read the latest insights, tips and perspectives from our team of cloud experts

      Case Studies

      See how we’ve helped thousands of public cloud customers achieve their goals

      Cloud Masters Podcast

      Listen to our experts and customers share tangible tips for navigating the cloud.

      Ebooks and Guides

      Discover foundational expertise and future-ready recommendations for the cloud

      Events and Webinars

      Tech talks and interactive expert sessions delivered both virtually and in person

      GCPInstances.info

      Google Cloud Compute Engine instance comparison

      Help center

      Read documentation, product updates, and more

      Newsroom

      See what's new from DoiT in our latest news and announcements

      Trust Center

      How we focus on security, compliance, and privacy

      Videos

      Watch product demos, interviews and more from our cloud experts

  • About
    • About DoiT >

      Careers

      Browse our open positions and learn more about what it takes to be a Do’er

      Leadership

      Meet the team leading DoiT and our customers on a journey of hypergrowth

      Newsroom

      See what's new from DoiT in our latest news and announcements

  • Pricing
  • Contact us
  • Sign In
  • Products
    • Flexsave ™
    • Cloud Analytics
    • Spot Scaling
    • BigQuery Lens
    • Anomaly Detection
    • DoiT Platform
  • Services
    • How We Work
    • Stats
    • Cloud Solutions
    • FinOps
    • Areas of expertise
      • Cloud Architecture
      • Cloud Cost Optimization
      • Cloud Migration Consulting Services
      • Cloud Security
      • Data and Analytics
      • Data Management
      • DevOps with AWS & DoiT
      • Infrastructure
      • Kubernetes
      • Location Based Services
      • Machine Learning
      • Multicloud
      • Training
  • Partners
    • ISV Go-Global
    • Award-winning public cloud partner
    • DoiT Together
  • Resources
    • Blog
    • Case Studies
    • Cloud Masters Podcast
    • Ebooks and Guides
    • Events and Webinars
    • GCPInstances.info
    • Help center
    • Newsroom
    • Trust Center
    • Videos
  • Pricing
  • About
    • Careers
    • Leadership
    • Newsroom
  • Contact us
  • Sign In
Contact us
Sign in

Blog

The Advantages of Vizier’s Black Box Approach

  • Joshua Fox Joshua Fox
  • Date: August 17, 2023

Part 2

In Part 1, we described the challenges of slow, costly optimization processes that never come out well enough the first time. Whether it is Machine Learning (ML) training, A/B testing of websites, or choosing the right inputs for a factory, each cycle is expensive, and you want to converge to a good result as rapidly as possible. Vertex AI Vizier on Google Cloud makes this happen by providing suggestions for parameters for each trial.

Just an Advisor

“Vizier” is a title for a king’s advisor, derived from Arabic وزير wazīr through Persian and Turkish. The Vizier advises on, but does not decide, royal policy. And this is how Vizier’s black-box optimization works. It does not carry out the optimization trials for you. In contrast, the hyperparameter tuner services provided by GCP and AWS does everything for you. You give it a range of hyperparameters; for example, the learning rate ranging continuously from 0 to 1; or minimum child weight as an integer parameter from 1 to 3. You create the hyperparameter tuner client, then fire and forget by calling a function like fit(). That function runs for a while, maybe a few hours, and internally executes multiple training iterations, choosing different hyperparameters as it goes. Then the tuner returns the best model it can find.

The Black Box optimization process is out of your control and is invisible to you. (At least as far as the API goes: There are typically monitoring systems in place).

Note: “Black Box” means that your trials are invisible to Vizier. It knows nothing about the ML training or the A/B test. It knows nothing about the gradients, the ups and the downs of the function being optimized inside each trial.

To you, on the other hand, these trials are completely “white-box”: You are in charge.

Advantages

A hyperparameter tuner service seems simpler than Vizier, and indeed Google built its Vertex AI AutoML and its hyperparameter tuners in the AI Platform (the older brand name) on top of a Vizier-based engine. But there are advantages to the Black Box approach in which you interact with Vizier.

Control

You get more control with Black Box Optimization.

The trials are generally in your area of expertise. Your website developers know the ins and outs of your website; your data scientists understand the details of the ML algorithmics. You have thought carefully about how to get the best out of your systems at the lowest cost. With Vizier, you retain full control of the actual ML training: What infrastructure or APIs to use, how many and which GPUs or TPUs, etc.

You can even ignore Vizier’s suggestions and use your own parameters. Of course, usually, you would use the suggestions: After all, that is why you are invoking Vizier. For example, you might read about a new algorithm for your ML, or if the product managers want to try out a new variant that was not in the website A/B testing, you can go ahead and try that. Even when you run the trial with your own parameters, you can still feed the parameters and metrics into Vizier, and it will learn from that, just as it does when you use its suggested parameters

Scalability

Vizier only does the “easy” part of optimizing across multiple trials. It does not do the heavy lifting of ML, which can require lots of specialized hardware. Likewise, it does not do the heavy lifting of the A/B testing of a website or running a factory. This makes it scalable because you handle the actual training or other optimized process, which is the most intense part of the overall workflow, and therefore can apply the best practices most suitable to your exact setup. (In Figure 1 below, your training machines are the “Evaluation workers” under the Vizier REST API.)

Meanwhile, Vizier scalably spawns workers to generate suggestions or suggest when to stop. As its state is stored in a database, rather than inside the context of a single optimization run, it can recover from failure. And because the suggestion workers only do hyperoptimization, not your “heavy lifting”, the Vizier can efficiently scale up these workers.

Vizier’s Black Box

Figure 1. Architecture diagram (Based on the diagram from the research paper.)

Multi-Objective

Most optimization processes, and most hyperparameter tuners, seek to maximize one metric. That is the most common usage of Vertex AI Vizier as well. But you might want to optimize several metrics at the same time. In an A/B test for a website, you might want to both maximize revenue and time on site. Or you might have a two-sided market, and you want the buyers to get good deals, but you also want sellers to maximize profits.

As a simple example to illustrate the concept, you can have a look at this sample Notebook from Google that maximizes two trigonometric functions of the same two input parameters.

Vertex AI Vizier

Figure 2. There is no one optimal point for both functions.

Clearly, there is no one point where both metrics are maximized, so Vertex AI Vizier discovers the Pareto frontier, A line on one side of which you cannot make one metric better, without making the other metric worse: The optimum is not one point, it’s a border.

Next: Flexibility, and some of the first Vizier ML notebooks

Vizier is a straightforward system, and you can (and generally, should) use it with default settings. Yet you can also further tune its capabilities. In the next article, we will show how to do that, and also provide some code samples.

See Part 3 “Beyond the basics: Vizier configurability and code samples”

Subscribe to updates, news and more.

Subscribe

Subscribe to updates, news and more.

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Related blogs

Using predefined IAM roles for enhanced Google Maps Platform governance

Examining using predefined Google Cloud IAM roles dedicated to Google Maps usage to enhance the governance of these activities.

Keep reading 
Ramp Plans Resource Hub Header1

Monitor your cloud commit attainment with DoiT Ramp Plans

DoiT Ramp Plans help you visualize, manage, and track your commit attainment so you can make sure you spend what you committed to, and act proactively.

Keep reading 
DoiT-Google-Kubernetes-Engine-Troubleshooting-Made-Simple-with-Interactive-Playbooks

Google Kubernetes Engine Troubleshooting Made Simple with Interactive Playbooks

In modern application management, Kubernetes is the foundation of container orchestration. It automates software deployment, scaling, and management, revolutionising delivery. However, growing complexity and scale pose challenges in troubleshooting and maintaining dynamic ecosystems.

Keep reading 
View all blogs
Let’s do it

From cost optimization to cloud migration, machine learning and CloudOps, we’re here to make the public cloud easy — without the costs.

Ready to get started?

Get in touch

Company

  • About us
  • Blog
  • Careers
  • MS-HT statement
  • Newsroom
  • Privacy policy
  • Terms

Offering

  • Compliance
  • Products
  • Services
  • Solutions
  • Resources

Support

  • Sign in
  • Help center
  • Open ticket
  • Contact us

Never miss an update.

Subscribe to our newsletter

Subscribe to updates, news and more.