Would you outsource your prompt engineering? [YC W25]

We're a YC company building LLM guardrails and human-in-the-loop systems. Through this, we have access to inputs, outputs, and human-corrected "golden" answers to LLMs. This data lets us make specific recommendations like "you can reduce human escalations by 20% by replacing prompt X with prompt Y."

Here's what I'm wondering: Prompt engineering and evaluation is time-consuming and often gets deprioritized. Why companies wouldn't outsource it? Think of it as hooking to a telemetry provider with few lines of code that also optimizes your prompts based on real usage data.

I'm considering focusing heavily on this "automated prompt optimization" angle - essentially, "we analyze your production data and continuously improve your prompts." Would you be comfortable letting another company handle this? What concerns would you have?

Looking for honest feedback from teams using LLMs in production.