Serverless computing: key Azure services

Image: Serverless computing cloud graphic
By Andrew Petty on

Key Azure Serverless Services

As a follow up to my previous blog on the Era of Serverless Computing, here I share some key Azure services commonly utilised in serverless computing. Within this blog I will also describe our involvement in an interesting use case for one of our utility customers. As mentioned in my previous blog, a serverless computing solution means the onus to design, manage and maintain the underlying infrastructure is effectively removed for you. In the case of Azure, this is undertaken by Microsoft.

Serverless computing components in Azure

Within Microsoft Azure, there are a plethora of serverless computing services at our disposal. I will just provide an overview of seven key services. To learn more about the entire Azure serverless platform, see azure.com/Serverless

 

Image: Serverless Computing Azure Functions icon Azure Functions (Serverless Compute)

Azure functions are at the heart of Microsoft’s serverless computing offering. Azure functions are event driven and provide compute that scales to meet demand. They enable developers to connect (bind) to a whole variety of data sources such as Cosmos DB and message services such as Azure Service Bus. This can all be implemented using a variety of languages such as JavaScript, C#, Python and PHP, as well as scripting options like PowerShell, Bash and Batch. Recently released are durable functions where stateful functions can be written. A subject potentially for another blog, but this really takes Azure functions to another level.

 

Image: Serverless Computing Logic Apps iconLogic Apps (Serverless Workflow Orchestration)

Azure Logic Apps allows developers to visually create automated scalable workflows that integrate with apps and data sources, both in the cloud and on-premises. There are over 200 pre-built connectors available from within a connection gallery that can massively assist in development of process workflows. Every logic app is started with a trigger and then the logic app instance runs the actions within a workflow. You may not even need to write code but if it is required, code can be run as an Azure function on demand within a Logic App.

 

Image: Serverless Computing Event Grid Event Grid (Serverless Events)

Azure Event Grid is a message routing service. It provides a single place to manage and listen for events from various sources (subscribers) and send (publish) or multicast information to various destinations. Event Grid is built into the fabric of Azure. This means events can be detected from many sources. Examples of events that can be detected are:

  • data being written into Azure storage
  • new subscriptions being created
  • new record being added to a database.

 

Image: Serverless Computing Event Hub icon Azure Event Hubs (Serverless Ingestion)

Azure Event Hubs is an ingestion service capable of receiving and processing millions of events per second. It is often used as an “on ramp” for Big Data, where the data is typically coming from event publishers such as IoT devices, user data, traffic from applications or web services such as Twitter. Events are captured into Blob storage or an Azure Data Lake and held or buffered for a retention period of up to 7 days. Typically, a retention period of 24 hours is used.

 

Image: Serverless Computing Azure storage icon Azure Storage (Serverless Storage)

Azure Storage provides a fully managed service that is highly available, scalable and redundant. In relation to serverless solutions Azure Storage offers four key services (there are a few others):

  • Blob – Used for unstructured data
  • File – Used for distributed File system
  • Queue – Used for durable queues could use Event Hubs though
  • Table – No SQL and used for key-value storage

 

Image: Serverless Computing Cosmos DB icon Azure Cosmos DB (Database)

Cosmos DB is Microsoft’s globally distributed, massively scalable, multi-model database service. It is being touted by many, as the database for the serverless computing era. Azure Cosmos DB’s database engine natively supports DocumentDB’s SQL dialect, MongoDB API, Gremlin (graph) API, and Azure Table Storage (key value) APIs to name a few. Importantly from a serverless architecture perspective, it has native integration with Azure Functions. Silversands sees Azure Cosmos DB as being the go to database for serverless applications. You can try Cosmos DB for free and get hands on with it at Try Azure Cosmos DB for Free.

Even though Cosmos DB is marketed as globally distributed, multi-node, highly scalable, highly available service, don’t let this put you off using it within your potentially non-global serverless solution.

Image: Serverless computing DB Logo block

Image: Serverless Computing Stream analytics icon Azure Stream Analytics (Event Processing)

Azure Stream Analytics is a service that enables action on and analysis of in motion data. It is used to examine high volumes of data to look for patterns, extract data, trends, etc. These patterns or data can be used to trigger other processes or workflows.

Stream Analytics is often integrated with Event hubs and IoT hubs. If interested, you can see how we have used Azure Stream Analytics in an internal PoC displaying light and temperature levels at the Silversands office.

 

Use Case – Ingestion of Raw Sensor Data (IoT) and Data Visualisation

Silversands has been working with a large utility company which has thousands of Internet of Things (IoT) devices that make up their supervisory control and data acquisition (SCADA) systems. We were asked to look at how the data is processed, transformed into semi structure data and then stored in a suitable repository.

This was a PoC to demonstrate a potential architecture for the ingestion of data and provide ideas on how the data could be interrogated by their line of business application (asset management system) and data scientists. It also provided a vision on how the data could be used to bring about cost savings, improvements to quality of service and potential ways data could be visualised.

As access to the live SCADA system, and therefore the IoT data, was restricted, the initial task was to create a process by which we could emulate the IoT data in a newly defined JSON format. For the PoC it was requested that we write the emulated data to Azure blog storage to emulate a potential production scenario (not directly into Event Hub or IoT Hub as you would typically do). We then developed a series of Azure functions to pick up this data and send it to the Event Hub. The JSON documents were then manipulated into a required format for storage. Table storage was used to look up key value information to append to the JSON files. Finally, the manipulated IoT data in JSON format was written into a desired partition in Azure Cosmos DB (Document DB).

Image: serverless SS IoT and Data Visualisation PoC diagram

 

Once the data was in Cosmos DB it could be utilised for various activities. The Cosmos DB change feed was leveraged to create a trigger which wrote information into the asset management system.

Image: serverless SS IoT and Data Visualisation PoC diagram2

Getting the data into a structure format was another requirement of the PoC. Azure Data Factory was utilised to extract data on a scheduled basis into an Azure SQL (could potentially use Azure Data Warehouse).

Finally, the customer wanted to visualise the data. Power BI was used to visualise the structured and unstructured data.

Image: Serverless SS IoT and Data Visualisation PoC diagram3

As a further extension to this PoC we worked directly with Microsoft to link up R notebooks on Azure Databricks (in preview at time of writing this blog) to demonstrate a potential mechanism by which the data scientists could undertake detailed and very complex analysis of the IoT data.

Image: Serverless Computing Data bricks

How can Silversands help?

Hopefully this blog has provided you with a high-level overview of some of the key serverless Azure services and potentially where some of these services could be utilised within serverless computing solutions. If you need any assistance with Azure development, designing your next serverless application or want to find out more about Azure in general then please complete the enquiry form below. We would love to speak to you and help you at any point along your Azure journey.

We run regular workshops and webinars providing the latest updates and expert advice about Microsoft 365, Cloud and Hybrid IT. We also post regular blogs so please do follow us.

Icon: LinkedInIcon: twitter bird

Contact us

  • This field is for validation purposes and should be left unchanged.

We have the expertise and the experience to provide specialist solutions and drive your business forward

Get in touch