This article is next in the Azure Databricks series, where we will learn how to connect data in Azure Databricks to the Power BI Desktop. We will also talk briefly about visualizations in the Databricks service.

In this article, we will learn how we can load data into Azure SQL Database from Azure Databricks using Scala and Python notebooks.

We introduced Azure Databricks, how it is created and some of its important components in my previous article here. We will look at how we can work with Azure Blob Storage in Azure Databricks in this article.

This article discusses what exactly an Azure Data Explorer is and its important features. We will also walk through the Kusto query language and explore a sample data in the Data Explorer in Azure.

This article talks about Azure Data Catalog and how data professionals can use it to locate, understand and consume data sources.

We’ve looked at methods to reduce costs within Azure. We may experience situations where a slight increase in Azure costs will benefit us to help protect our resources and customers when it involves security or other critical updates. When we consider these situations, we must keep up-to-date with the latest patches, updates to development libraries, as well as the possible effects of these updates to our existing code. Likewise, related to resource usage, a resource may be unused or seldom used by a percent of our customers that we keep when we’re ready to switch all our customers while we make the appropriate upgrades to our resources to help with costs. We’ll look at some techniques that we can use to manage the challenge of critical updates while also keeping costs down, or putting costs into a context about what may be more expensive.

To reduce Azure costs on unused and unnecessary resources, we should design with prevention in mind – considering whether we want to commit to reserved use or test with a pay-as-we-go model. We may experience situations where we already have many resources, but are unsure of their use – are these consistently used, sometimes used, never used? Before we can answer whether an unused resource (or what appears to be an unused resource) is unnecessary, we have to determine whether it’s used. In this tip, we’ll look at this challenge.

The Azure Portal offers the free tool Cost Management that we can use for managing Azure costs. As we’ll see, we can use this tool to organize how we manage our spending along with setting limits for thresholds to alert the appropriate members. While this tool can be useful for our organization, it has the potential to cause noise or disruptions, so we still want to review how we use it within our organization for managing the spending of our teams.

Depending on our design and security, we can create functions or use built-in tools to control our Azure costs. In some contexts, we may look at the overall cost of what tools we’re using, which the Azure portal conveniently shows. Applying what we’ve looked at on the organization and development level, we can organize resources on their design (or ad hoc, as we’ll see) along with creating scripts that control our scale for situations where we may want higher or lower scale. We’ll look at both of these scenarios and how they can help us in both organization and development contexts.