Origin Story

HOW APEX
CAME TO BE

A 500-mile pipeline. A skeleton crew. Air conditioners nobody thought were critical — until the day everything went black. This is the failure, the lesson, and the idea that became Apex.

I started as an I&E technician for a large, well-structured oil and gas company — one of two technicians covering 500 miles of pipeline across three states. Not long after I started, I became the only I&E technician on the entire pipeline for an extended stretch. I was spread thin, to put it mildly.

The pipeline was about three years old when I arrived. It ran consistently, but we were still learning the system as it made the long transition out of commissioning and into true operation. There were random anomalies and unexpected behaviors all along this fully automated pipeline, and I had no shortage of opportunities to learn what happened, why it happened, how it happened, and how we'd keep it from happening again. I was scheduled for 55 hours a week as an hourly employee and rarely worked less than 60.

This pipeline was automated in ways I'd never seen. The company that built it used automation as a tool to maximize return on investment — almost to a fault. As one example: they ran two pumps off a single VFD, when traditionally each pump gets its own VFD for speed control. It saved them hundreds of thousands in capital. It also introduced a problem nobody on the operations side got to vote on.

How One VFD Ran Two 2,000 HP Pumps

Each pump had two line contactors — one feeding it from the VFD, one feeding it from 60 Hz utility power. With no pumps running, the VFD contactor for Pump 1 closes and the VFD ramps it up to 60 Hz. With the VFD and the utility both at 60 Hz, the VFD syncs its sine wave to the utility. The instant it senses sync, the utility contactor closes — both closed for a brief moment — then the VFD contactor opens.

Now Pump 1 runs straight off utility at 60 Hz, and the VFD reconnects to Pump 2, modulating its speed to hit the flow and pressure needed to meet the daily crude oil nomination. Clever. Lucrative on paper. And here's the unintended cost.

That single VFD just became more than twice as critical as it would have been in a traditional one-VFD-per-pump setup. If the VFD fails, neither pump can run at all. Two or three pumps, zero ways to start them. A traditional setup would have let you run at a curtailed flow rate. This one runs at zero.

And the failure compounds. A station going down means every upstream station slows way down to keep from overpressuring the line. The pressure climbs. The whole system feels it.

The VFDs were, without a doubt, the most critical equipment on the entire pipeline.

So what holds the number two spot? Air conditioners. I'm not crazy — the same technology you have at home was the second most critical piece of equipment on that system.

Like most industrial facilities, our large heat-generating equipment depended completely on HVAC and cooling to run. Without it, depending on the application, you might have hours before things overheat. On others, minutes.

Now layer in how lean a pipeline runs. One person covering an area that takes hours to drive end to end. Most of our operators covered about an hour and a half in each direction.

The Capital Project MindsetWhy the Cooling Was Never Quite Enough

When a capital project is spec'd, the team calculates the cooling load, sizes the units to the need plus a safety factor, and often adds redundant HVAC. Smart on its face. But three forces quietly work against doing it right.

So here's where we ended up. Unmanned facilities. A skeleton crew. The system automated and controlled from a control room in a different part of the country. Operators spent an hour or less at each pump station on their rounds — meaning 23 hours a day on weekdays and a full 24 on weekends, the stations were monitored remotely from Houston and otherwise completely unattended.

We had five HVAC systems cooling each pump station's electrical building. The load needed 3.2 of them — meaning three ran 100% of the time, and a fourth kicked on as needed. On cool days the fourth barely ran. On the hottest days it had to. But hey, we had a hot spare at each location, so we were covered, right?

Wrong. As a technician, I noticed our HVAC failure notifications always came from the control center — on a weekend, at 9pm, at 3am. VFD overheating. The operator drives out and discovers we don't have one failed HVAC. We have two or three.

The Lesson That Stuck

Walking into a cool electrical room doesn't mean it's good. It means there are still enough units alive to keep up.

How many times did someone walk into that cool room, never knowing a unit or two had already failed — never knowing it would become a shutdown emergency the moment the next one went?

So while I was still a technician, I made it a personal rule to prove every unit was working, every time I was onsite. I'd force each HVAC unit on and check how cold the air came out of the vent. I knew I'd be the one getting it running again at 3am. Operators were usually called first — we had ten or so at any given time and only one or two technicians — but they observed and reported. When the tech arrived, some stayed to watch and some headed home.

The routine worked. Then life changed: I was promoted to managing the pipeline operation itself. No longer the technician.

The View From The Other Side

After the promotion, I tried to get everyone on board with the daily HVAC test — the thing that had eliminated the problem for me. I couldn't get people to do it consistently. That was one of my early failures as a people manager, and here's why it failed:

It was a tough sell to ask an operator to walk into a 62-degree room and test air conditioners that all felt fine. And once I was the manager, the old failure calls started creeping back, more and more often.

The Thought That Became Apex

I wish there were a way to know the instant a unit failed — so I could schedule the repair, instead of a pipeline emergency shutting us down from an overheated VFD.

That was it, exactly. Five units, needing three (sometimes four on hot days). An alert and a scheduled repair would erase the worry from my plate entirely. It would make the pipeline more reliable, the operation smoother, and it would lift an unfair burden off a lean ops team. It checked every box I was supposed to deliver as a manager.

Easy fix, I thought. I'll just get quotes to run temperature sensors to the output ducts and monitor the discharge temp of each unit. Run wiring to the compressor contactor to watch compressor status. When the compressor shows "running" but the duct temperature isn't dropping, throw an alarm into SCADA so the control room calls us and says an HVAC failed. Problem solved.

Wrong again — and this is where I ran into the first wall.

Roadblock #1Capex vs. Opex

Labor quotes came back around $5,000 per location, before travel and before the temperature transmitters I'd still have to buy. All in, about $7,000 per site for our four most critical mainline stations. My operating budget covered it easily — I reviewed that budget every single day.

The work was denied. It was classified as a capital expense. At the time I didn't fully understand what made something capital versus operating — I just knew the ops people around me were endlessly frustrated by it.

Why would anyone be upset about not having to spend their own budget? Spend someone else's money, save yours for something else — right? Actually, no. They'd rather use their own. Here's how a lot of ops folks see it:

It's no longer their decision. They have to hand an operational problem they're responsible for to someone in another part of the country who has never been to the facility, doesn't know its challenges, and doesn't get the 2am call. To many operators, the capex/opex line removes their ability to fix the things they're accountable for. Extra red tape. Some half-joke, "you just don't understand big business."

The Skid Steer Question

A manager decides he needs a skid steer. He has two options: request a capital expenditure and wait one to five years, or rent one, pick it up today, and get on with his life. The frustration is real — but making something a capital expense, when it truly is one, often is the right call for the company.

The better way to frame it: does he need to own a skid steer, or does he need the ability to use one? Renting gives him exactly what he actually needs — access — without the responsibility of ownership. That reframe matters more than it looks. Hold onto it.

And the people making capital decisions? It can feel like they don't care about operational efficiency. They do. It just helps enormously to understand how capital budgeting actually works — because once you see it, you can steer your thinking toward solutions that survive it.

A Short DetourHow Capital Decisions Really Get Made

This wasn't meant to be a finance lesson, but it's the hinge the whole story turns on.

Capital decisions start with two numbers. First, a finite pool of money the company sets aside as the capital budget. Second, the cost of capital — often called WACC, the weighted average cost of capital. It's the cost of the company's funds, weighted because a company typically uses both debt and equity.

The cost of equity is the return the company owes its investors, the same way interest is the return it owes the bank. So if equity costs 16% (an investor wants $16 back for every $100 in), the after-tax cost of debt is 8%, and the company is 50% debt and 50% equity, the weighted average cost of capital is 12%.

Put plainly: the money sitting in that budget costs the company 12% just to be there. Now ask yourself — would you rent a machine for $1,000 if you knew you'd only make $750 with it? Of course not, and neither would they. A company won't fund a project that doesn't at least clear its cost of capital.

So we have a fixed budget and a minimum acceptable rate of return. And since companies want to maximize returns, they don't aim for the 12% floor — they aim for 30%, 40%, more if they can get it. The floor is just the hard stop.

Now picture a company with $1,000,000 in its capital budget and 200 managers submitting requests that average $100,000 each. That's $20 million in requests chasing $1 million in funding. Which ones win?

So where do operational and reliability improvements land? In a catch-22. Most reliability outcomes are preventative — their value only shows up if something bad would have happened. Deploy HVAC monitoring and prevent four shutdowns, and it's nearly impossible to put a clean dollar figure on the four shutdowns that never occurred.

A project return is realized. An avoided loss is not — because it never happened.

A company feels far more comfortable saying "we'll add a widget line for $100 and earn $115 back" (15% in their pocket) than "we'll install this monitoring for $100 and eliminate a future cost of $45" (45% in their pocket). It happens constantly: capital budgets approve a 15% return and step right over a 30%-plus saving, purely because of the uncertainty in quantifying it.

That's the real dilemma. Capital budgeting simply isn't built for operational problems. Capex is an investment decision, not an operations decision. Even the best ideas — the ones that keep more money in everyone's pocket — get denied because the return is hard to prove.

That was roadblock number one.

Then It Actually Happened

Not long after, I got the call from control: our origination station was down on VFD overtemp. The local operator was first onsite. He walked into a building that had been down a while and measured an ambient air temperature of 150 degrees. Every screen was black.

"Charlie, it's bad," he told me. "This doesn't look good. Nothing's on fire, but I can't touch the walls. Three failed air conditioners. It got so hot in there we think it tripped the thermostats on the other two and shut them down too."

Several hours and a six-hour drive later, we were still down — testing, inspecting for damage, waiting many hours on an emergency HVAC contractor callout. By the time the units were repaired, the room was cool, and the whole place had been inspected, the screens finally came back. We decided to try a start. Amazingly, the VFD started and kept running.

We came within a hair of being shut down for weeks or months, hunting for a 2,500 HP VFD we had no spare for and that they no longer manufactured. We got lucky.

After downtime like that lights up notifications across the country, my boss called. "Hey — you know that HVAC thing you were trying to do? Get it done."

Roadblock #2The Control Center Can't Take It

Now I had the green light. And I hit the next wall.

A pipeline control center looks like an arcade. There are legal requirements tied to it — for DOT-regulated pipelines, headcount is driven in part by the number of alarms coming into the room. So the control center has a real incentive not to bring non-safety items in. They can add whatever they want, but more alarms can mean more required staff. I hadn't known that.

Which meant I couldn't route the HVAC health alerts into the control room. Fine — I'll set up alerts on the side. So I asked whether the PLC itself could just send alerts when an HVAC failed. Denied.

What it got reduced to: I had a technician install local indication — we turned on a light when a unit faulted. A light, at an unmanned facility, that almost no one ever sees. Better than nothing. At least an operator could tell at a glance that a unit was bad and skip the manual test.

Holding the responsibility to manage a problem without the authority to correct it. Few things are more frustrating.

What I Took AwayThe Lessons That Built A Company

01
Operations knows best what operations needs.The person standing next to the machine at 2am sees the problem most clearly.
02
Capital budgeting is built for investment, not reliability.It rewards realized returns and undervalues avoided losses — even when the avoided loss is bigger.
03
"Why would they design it this way?"Usually the answer is return on investment, not operational efficiency. Once you know that, a lot of strange decisions make sense.
04
Responsibility without authority is the most expensive gap in operations.The fix exists. The budget exists. The line between them is what fails.
So We Built It Differently

I set out to model a business that does the things every wall in that story prevented. One that solves problems effectively and efficiently, doesn't depend on humans remembering to look, and removes the three hurdles I kept hitting — the capex hurdle, the control-room hurdle, and the IT/automation hurdle.

We help the people who can't afford to fail
sleep a little better.

Managers. Technicians. Asset owners. Anyone responsible for something that runs silent until the moment it doesn't. That's who Apex was built for — because I was one of them.

Submit A Machine →
← Back to Apex Tech Systems