Embracing Imperfection: Turning “Trash” into "Treasure"

Garbage in, garbage out: We’ve all heard it. And yes, data quality matters. But let’s not be too quick to dismiss imperfect data as useless. After all, one person’s trash can be another’s treasure, the real challenge is figuring out how to find that value.

I’ve never encountered perfect data. I’m still holding out hope (never say never), but in reality, our datasets often arrive incomplete, noisy, and full of quirks. When the numbers aren’t pristine, do we just shrug our shoulders and give up? No. Instead, we need to step back and consider analyses that fit the problem as well as the data we’ve got, not the data we wish we had.

Picture a rowdy meeting room: voices raised, papers tossed. The conversation is about boosting production despite tough conditions. Management demanded: “How many months of reduced MTTF is acceptable for extra barrels?” The engineer, cornered, replied: “I can’t quantify it, the data is not good enough.” This standoff is more common than you’d think.

The truth is, we rarely capture data with such end goal in mind, to test the hypothesis. In oil & gas, we just want to make more oil/gas. But before we concede defeat, let’s challenge the assumption. Instead of asking “Can I quantify the risk in exact months?”, start with a simpler but vital question: “Is there really an increased risk at all?” With imperfect data, sometimes it’s more practical to identify “what is not” than to definitively claim “what is”.

My early-career lesson: Elimination is powerful. It’s often more objective and can steer us clear of confirmation bias. Too many smart professionals get trapped trying to confirm what they believe, rather than trying to disprove it.

Recently, I performed a data analysis that helped eliminate an assumption of premature pump failure risk during accelerated production recovery, showing yet again that even with imperfect data, we can find clarity in what isn’t true, even when we can’t perfectly prove what is.

In a world of imperfect data, we can still make progress by asking the right questions, focusing on what we can eliminate, and challenging our own beliefs. It’s not about perfection; it’s about navigating the mess wisely. Find treasure in what first looked like trash.