How can I move my ETL logs from a VM to Azure Storage and use KQL for querying?

0
12
Asked By TechieNinja42 On

Hey everyone! We're making the switch from Azure Function Apps to Azure VMs to handle our ETL processes. Right now, we're just saving our logs locally on the VM as .txt files, but I'd really like to store them in an Azure Storage account. Additionally, I want to enable KQL so we can easily query and analyze our logs. Does anyone have tips on how to set this up? Thanks in advance!

2 Answers

Answered By CloudExplorer99 On

You should consider using a Log Analytics Workspace if you want to work with KQL. Ingesting your logs there can be done through Application Insights, which is a great way to handle logging without dealing with local log files. You won't need to set up a separate Storage Account for this process, which simplifies things a bit.

Answered By DevCurious88 On

Switching from serverless solutions to VMs doesn't seem like the best idea to me. Azure Functions might not fit your needs perfectly, but have you thought about using Azure Data Factory for your ETL tasks? It's designed for that purpose and could save you some hassle instead of managing VMs.

Related Questions

LEAVE A REPLY

Please enter your comment!
Please enter your name here

This site uses Akismet to reduce spam. Learn how your comment data is processed.