I'm seeking guidance on how to pull logs from an obscure application into Microsoft Sentinel since there are no syslog or CEF options available, and the customer is adamant about not using a push method. I need to configure everything to pull the logs via API calls instead. I've already set up a Data Collection Rule (DCR), Data Collector Endpoint (DCE), and a custom table based on a sample log format. However, I'm unfamiliar with how to use an Azure Function App to pull the data, despite having a URL and a secret for reference. I've searched for assistance with tools like Copilot and ChatGPT, but haven't found anything that helps. I'm under significant pressure to get this done, and any advice would be tremendously helpful!
5 Answers
You should store your secret in Azure Key Vault and retrieve it using the function. Depending on what the API requires, you might need to set up an authorization flow. The URL could be stored as an application setting, making it easy to access as an environment variable.
You might want to check out the Microsoft documentation for creating connectors. Their GitHub repo has great references for data connectors that employ CCF, which could give you a solid foundation to work from.
I'll give that a look and see if it aligns with my setup. Appreciate the quick help!
Have you considered using a webhook for that application? If the source API provides a few hundred records every few minutes, Logic Apps could be a better fit without the complexity of function apps.
That sounds promising; I'll try the codeless connectors and keep this as a backup! Thanks for the suggestion.
I recommend looking into Logic Apps. Setting up a recurrence trigger for an HTTP request to pull data every 5 minutes could work well. You can filter the data to only get the past few minutes, which would keep everything efficient.
That could definitely help! Right now, I'm trying the codeless connector route, but this is a good backup plan. Thanks!
You're actually on the right path with your setup. What you're missing is something that regularly calls the API and sends the data into Sentinel, which is what the Azure Function helps with. Think of it as a simple script that runs periodically to fetch your data and push it to Sentinel. Don't get too stressed about the function; it's a straightforward setup.
Hey, I appreciate the encouragement. I did try setting up a Function, but it’s leading me to download Visual Studio and other complications I’m not comfortable with. I'll keep that option in my back pocket for now.
To be honest, that feels overwhelming with my tight timeline. I'll consider the other advice first. Thanks for your input!