I'm starting to feel uneasy about how much we're investing in our AI infrastructure, which has become our largest expenditure in engineering. It's now surpassing costs for observability and our data platform, and I didn't anticipate this happening so quickly. The board wants to know what tangible benefits we're getting from this investment, but my answers are pretty vague. I can say that engineers feel more efficient and product development flows better, but I can't provide solid metrics or outcomes to back this up. The challenge is that the positive impacts are spread across different teams and workflows, making it hard to quantify. Is there a reliable model or method to connect our AI spending to measurable outputs?
4 Answers
AI tools can be very useful for rapid tasks but often require constant supervision. Essentially, they could be seen as super-expensive auto-completers needing lots of redirects. I believe measuring the productivity of your engineers is crucial; without data showing the correlation between AI spending and actual productivity, decisions will be tricky.
Bottom line, without quantifiable data, it’s a tough sell when justifying costs.
It's tough to measure productivity since AI impacts various areas differently. If you have any existing metrics, like DORA metrics, you could analyze your performance before and after implementing AI tools. What you might find is that while AI can speed up certain tasks, it might also slow down experienced engineers when they have to manage or redirect AI output. So getting accurate, well-organized data will be essential to understand whether it's worth the investment or just throwing money at an expensive auto-completer.
Great point! Tracking productivity changes over time before and after using the infrastructure could really clarify its impact.
Yeah, I think it could highlight areas where we're gaining efficiency versus where we might be facing setbacks.
Measuring the impact of AI may be complicated, but it’s definitely doable. You could consider time motion studies or various cohort studies to get some metrics over time. AI has a broad impact across workflows, so while it can feel overwhelming, don't be discouraged—there’s potential for real insights if you’re willing to dive into the data.
Great suggestions! It will be challenging, but helpful to break it down into specific studies for better clarity.
For sure, just gotta focus on systematic measurement to get a clearer picture.
Honestly, it feels like a lot of hype surrounding AI right now without much real progress. People talk about flying cars, but I'm still waiting for tangible results. Sure, some areas like computational chemistry are seeing improvements, but for most use cases, I just don’t see any substantial benefits. If the hype doesn’t translate to real success soon, why continue pouring money into it? We need to see some serious ROI, or at least clearer metrics to justify the spending. If not, the board has every right to question it!
So true! I feel like it’s just a cycle of more spending with little payoff. We really should focus on measuring actual outcomes before blindly investing more.
Right? Show me the money is exactly what we need to prioritize!
Exactly! And if you're investing without seeing results, that's just a quick path to problems down the line.