Financial Operations

Getting Started with FinOps Hubs Multicloud Cost Reporting with Azure and Google Cloud

Ingrid Johansson

Tags: Cloud Management; FinOps; Multicloud

Getting Started with FinOps Hubs: Multicloud Cost Reporting with Azure and Google Cloud

FinOps Article

The rise of multicloud strategies is reshaping the way organizations manage their IT infrastructure, allowing them to leverage the best features of each cloud provider. Yet, the complexities involved in managing cost across diverse platforms poses significant challenges. This is where Microsoft’s FinOps hubs emerge as a pivotal solution, providing a unified platform for analyzing cloud expenditures, including Azure and Google Cloud.

The Promise of FinOps Hubs

Microsoft’s FinOps hubs provide a robust platform designed to unify billing data across cloud providers, enabling efficient management and optimization of cloud expenses. Built on the scalable foundations of Azure Data Explorer (ADX) and Azure Data Lake Storage (ADLS), these hubs empower organizations to make informed decisions based on comprehensive financial data analysis.

A Hands-On Approach to Multicloud Reporting

One of the key features of Microsoft’s FinOps hubs is their ability to normalize billing data across providers through the FinOps Open Cost and Usage Specification (FOCUS). This specification simplifies the reporting process by enabling side-by-side comparisons of cost and usage data, thus aiding in strategic decisions about workload allocation and resource sizing.

In this guide, we’ll embark on a detailed walk-through to integrate Google Cloud billing data into your FinOps hub using Azure’s capabilities. We’ll connect Microsoft FinOps hubs to Google Cloud, allowing you to export and analyze Google Cloud billing data alongside Azure billing data within your FinOps hub instance.

Prerequisites and Setup

Before diving into the step-by-step process, it’s important to ensure you have the necessary prerequisites:

  • Access to a Google Cloud billing account and a linked Google Cloud project with BigQuery and other required APIs.
  • Proper IAM roles and permissions for working with BigQuery, Cloud Functions, Storage, and billing data.
  • Detailed billing export and pricing export configured in Google Cloud.
  • An existing FinOps hub deployment to connect the data.

Establishing Google Cloud Billing Exports

The journey begins by enabling detailed billing exports and pricing exports in Google Cloud. This involves setting up a BigQuery view aligned with the FOCUS 1.0 schema, which will convert detailed billing exports into FOCUS format for easy normalization.

Let’s delve into the detailed steps to initiate this setup:

  1. Create a Service Category Table: This table maps GCP services to categorized services, simplifying the management of service categories in billing exports.
  2. Configure Metadata Table: This table tracks export timestamps, enabling incremental exports and ensuring data integrity.
  3. Deploy Cloud Functions: Automate the incremental export of billing data from BigQuery to Google Cloud Storage (GCS) using scheduling.

With these Google setups, you are equipped to normalize Google Cloud’s billing data ready for integration.

Transitioning to Azure

Now that you have enabled GCS with FOCUS-formatted exports, the next step is to integrate this data with Azure. Here’s how to set up a Data Factory pipeline to fetch, transform, and load Google billing data into your FinOps hub:

Building the Data Pipeline in Azure

Using Azure Data Factory, you’ll establish a pipeline that encompasses several critical components, such as:

  • Linked Services: Establish connections to both Google Cloud Storage and Azure Blob storage.
  • Datasets: Define both source (GCS) and sink (Azure Blob) datasets to manage CSV to Parquet data transformations.
  • Pipeline Logic: Control the data flow, ensuring only new data from Google Cloud is ingested, leveraging metadata checks for efficiency.

Step-by-step Pipeline Construction

  1. Create a Google Cloud Storage Linked Service: This service facilitates secure connectivity between Azure and GCS.
  2. Set Up Source and Sink Datasets: Configure datasets that handle CSV files from GCS and transform them into Parquet files ready for your FinOps hub.
  3. Implement Incremental Loading Logic: Pull metadata from the exports and ensure only new records are processed.

This setup allows for a streamlined and automated data ingestion process, blending data from different platforms seamlessly.

Conclusion: Towards Unified Multicloud FinOps Reporting

By leveraging the potential of Microsoft’s FinOps hubs and the FOCUS standard, organizations can achieve a unified, actionable insight into their multicloud expenditures. This not only facilitates improved financial decision-making but also drives operational efficiencies across cloud platforms.

With Azure and Google Cloud now speaking the same financial language through the FOCUS specification, you’re poised for a more controlled and insightful multicloud management experience.

Next Steps

  • Schedule your Azure Data Factory pipeline to ensure seamless, ongoing data integration.
  • Visualize your Data with Power BI dashboards, leveraging templates from the Microsoft FinOps Toolkit.
  • Expand the solution to AWS or other cloud providers using similar methodologies.

If you have further questions or need assistance in deep dives of FinOps and multicloud reporting, feel free to reach out and explore more cutting-edge solutions.