Kunal's Optimization Centre
A Comprehensive Guide to GCP's Auto Scaling and Custom Metrics for Big Architectures

A Comprehensive Guide to GCP's Auto Scaling and Custom Metrics for Big Architectures

Kunal Katiyar's photo
Kunal Katiyar
·

4 min read

Motivation

This week, I was working on optimizing our Kafka consumer services, when I noticed the most load that we used to have was during night when some job runs, and mostly the resources were being unused.

I quickly looked into the lag build up, and it was mostly during the night when we required such heavy resources.

So, I came to the conclusion to scale this resources automatically in the night, while keep the minimum required resources during the day. Here, GCP’s autoscaling feature worked out perfectly for me, but the basis was kafka lag buildup as the metric of scaling. This blog was a real lifesaver in that respect, do check it out!

https://medium.com/google-cloud/kubernetes-hpa-autoscaling-with-kafka-metrics-88a671497f07

This is how my auto-scaling looks now.

As businesses increasingly move to the cloud, managing resources efficiently is critical. Google Cloud Platform (GCP) offers powerful auto scaling capabilities that help ensure applications can handle varying loads without manual intervention. In this blog, we’ll explore how GCP’s auto scaling works, how to customize it, and how leveraging custom metrics can optimize scaling for complex architectures.

What Is Auto Scaling in GCP?

Auto scaling in GCP refers to the automatic adjustment of the number of virtual machine (VM) instances in a managed instance group (MIG) based on traffic or resource demands. This ensures applications maintain performance during peak times and save costs during low-demand periods.

https://cloud.google.com/compute/docs/autoscaler

Key features include:

  1. Dynamic Scalability: Automatically scales up during high demand and scales down during idle periods.

  2. Predictable Performance: Ensures sufficient resources are available to handle traffic spikes.

  3. Cost Efficiency: Optimizes resource usage, reducing waste and cutting operational costs.

  4. Integration with Load Balancers: Ensures traffic is evenly distributed across instances.

How Does Auto Scaling Work in GCP?

GCP’s auto scaling is primarily driven by instance templates and policies. Here’s an overview:

  1. Instance Templates: These define the VM’s configuration, including machine type, image, and disk settings.

  2. Managed Instance Groups: MIGs are collections of identical VM instances managed as a single entity. Auto scaling is applied at the MIG level.

  3. Scaling Policies: Policies dictate when and how the scaling occurs. Common types include:

    • Metric-Based Policies: Triggered by CPU utilization, memory usage, or custom metrics.

    • Schedule-Based Policies: Triggered at specific times.

    • Load Balancer Utilization: Triggered when a load balancer’s capacity exceeds thresholds.

  4. Monitoring and Metrics: Auto scaling relies on Cloud Monitoring for real-time data to make scaling decisions.

Customizing Auto Scaling

Customization allows you to tailor scaling to your application’s unique requirements. Here’s how to achieve it:

  1. Define Scaling Metrics:

    • Use default metrics like CPU or memory usage for standard workloads.

    • Leverage custom metrics for specific use cases, such as queue length or request latency.

  2. Set Target Policies:

    • Define target utilization levels (e.g., 70% CPU usage).

    • Specify minimum and maximum instance counts to control scaling limits.

  3. Create Custom Schedules:

    • Use schedule-based policies for predictable traffic patterns, such as nightly batch processing.

  4. Combine Metrics:

    • Use multiple metrics for more granular scaling (e.g., scaling based on both CPU usage and memory).

  5. Pre-Warming Instances:

    • Set up instance warm-up periods to handle startup latency, ensuring newly added instances can serve traffic immediately.

Leveraging Custom Metrics for Big Architectures

Custom metrics provide flexibility in scaling applications with complex requirements. Here’s how they work:

  1. What Are Custom Metrics? Custom metrics are user-defined data points collected from your application or system. For instance, the length of a message queue or the number of active users. These can be provided by external services, but you will need to register them in GCP to access these metrics (ex. Kafka exporter: https://github.com/danielqsj/kafka_exporter)

  2. How to Implement Custom Metrics:

    • Instrument your application to publish metrics to Cloud Monitoring using the Monitoring API or libraries like OpenTelemetry.

    • Create an auto scaling policy that references the custom metric.

  3. Benefits of Custom Metrics:

    • Enhanced Control: Tailor scaling decisions based on application-specific indicators.

    • Improved Performance: Respond faster to non-traditional load patterns.

    • Scalability for Microservices: Scale individual services independently, optimizing resource usage.

  4. Use Case: Scaling with Message Queue Length In a large architecture with a message queue (e.g., Pub/Sub or RabbitMQ), scaling based on CPU might not reflect the actual load. Instead, use queue length as the trigger:

    • Publish the queue length as a custom metric.

    • Set a scaling policy to add instances when the queue exceeds a threshold and reduce instances when it’s below another.

Conclusion

GCP’s auto scaling is a cornerstone for building resilient and cost-efficient cloud applications. By customizing scaling policies and incorporating custom metrics, businesses can create sophisticated, responsive architectures that meet the demands of modern workloads. Whether you’re scaling a single application or a sprawling microservices ecosystem, GCP’s tools ensure your infrastructure adapts seamlessly to any challenge.

autoscalingGCPoptimizationnodeinstancecost-optimisationmonitoring