A map of feedback loops in pharma, industrials and semiconductors

A model can learn a team's work when the work is graded, and when the grade can be tied to the step that earned it. Teams differ enormously in both. This post places the teams of the three industries we work in on one map.

It follows our model of a single team, the IP portfolio team of a pharma company. Here the same two questions are asked of eighteen teams at once.

The first is the loop time τ. It is the time from an action to a signal that grades it. Loop time sets how often a person, or a model, gets to learn. With T a working year, the number of loops in a year is

n = T / τ

The range is wider than intuition suggests. Take a working year of 220 days of eight hours.

Minutes

About 100,000 loops in a year.

Hours

About 1,800.

Days

220.

Weeks

44.

Months

About ten.

Years

One.

Decades

A handful in a career.

The second is the clarity of the signal. We score it with four yes-or-no questions about the outcome a team is judged on.

Written

Does a system or a third party write the outcome down, so that it exists outside someone's head?

Reasoned

Does it come with a reason that points at a step of the work?

Priced

Is it on a scale that matters to the business, such as money, time or a specification?

Testable

Can an alternative be tried before committing, in a simulation or a rerun?

Each question matters for its own reason. An outcome that is not written cannot be learned from at all. A reason turns a late outcome into credit for one step. A price closes the gap between a proxy and the goal. A test replaces a correlation with an experiment.

Roughly, the learning signal a team produces in a year is the information in one graded loop, times the number of loops.

S ≈ I · T / τ

Clarity raises I. Speed lowers τ. A team can be rich in signal either way.

AllPharmaIndustrialsSemiconductors

PharmaIndustrialsSemiconductorsA factor of three either wayCriterion metNot met

Chip verificationSemiconductors. Clear on 4 of 4. The first signal lands on the same desktop.

First signal after minutes: A simulation run passes or fails within minutes, and the failing check names the signal. Final outcome after months: First silicon works, or the chip needs a respin. Written, reasoned, priced and testable in advance.

Three things stand out.

Clarity and speed go together where a tool grades the work. In the top group a simulator, a checker or a machine returns a verdict within minutes or hours, and the verdict says where the problem is.

Every team has a second, slower loop. The first signal teaches. The final outcome is the one that counts, and it arrives months to decades later.

Slow does not mean vague. The regulatory and patent loops of pharma take years, and their outcomes are written, reasoned and priced by an authority.

The length of a loop is not the hard part. The hard part is the distance between the signal that teaches and the outcome that counts. We measure it in orders of magnitude.

g = log₁₀ ( τ_final / τ_first )

PharmaIndustrialsSemiconductorsEqual ratio of final outcome to first signal

Select a point to read its gap.

A point on the lowest diagonal would be a team whose first signal is its final outcome. No team is there. The widest gap belongs to design, about five orders of magnitude. The narrowest belongs to regulatory affairs, where the first signal arrives late and the outcome follows soon after.

The gap is what a value function has to bridge. Each signal is trained to predict the next one, as in the patent model. A wide gap with many signals along the way is learnable. A wide gap with nothing in between is not.

In semiconductors the tools grade the work. A timing report or a failing check is exact and arrives the same day. The slow loop is silicon. A wafer takes about twelve weeks on average to fabricate, and up to twenty for advanced processes. [1]

Chip verification

First signal, minutes. A simulation run passes or fails within minutes, and the failing check names the signal. Final outcome, months. First silicon works, or the chip needs a respin.

Chip implementation

First signal, hours. A timing report after place and route lists every failing path. Final outcome, months. The chip meets timing, power and area in silicon.

Sign-off

First signal, hours. A physical check comes back clean or with located violations. Final outcome, months. The foundry accepts the tape-out.

Process and yield

First signal, days. Inline measurements on a lot. Final outcome, months. The yield of the lot at the end of the line.

Equipment service

First signal, hours. The tool passes its qualification run after a repair. Final outcome, months. The uptime of the tool over the following weeks.

In industrials physics grades quickly and cheaply, but cost and quality land on other desks, in the plant and in the field. Nothing about a design is published. It all sits in the company's systems and on desktops, so nobody outside can learn from it.

One structural point sets machines apart from patents. A product is protected when any one patent holds. A machine works only when every part does, so reliability multiplies.

R = Π_i r_i

The weakest step sets the outcome, and a second good part does not make up for a bad one.

NC programming

First signal, minutes. The toolpath simulation shows a collision or a clean run. Final outcome, weeks. The first part is measured and the cycle time is on the machine.

Automation

First signal, hours. The control program runs against a simulated machine. Final outcome, months. The plant passes its acceptance test at commissioning.

Design

First signal, minutes. The CAD system flags an interference or a broken rule. Final outcome, years. First articles pass inspection and the part runs in series.

Simulation

First signal, hours. A solver run ends and the result is compared to the load case. Final outcome, months. The physical test agrees with the simulation, or does not.

Engineering change

First signal, weeks. The change board approves or returns the change, with its reason. Final outcome, months. The change takes effect in series and its cost is booked.

Quality engineering

First signal, days. A measurement report confirms or rejects the suspected cause. Final outcome, months. The complaint is closed with a root cause and does not recur.

Costing and quoting

First signal, weeks. The quote is won or lost, usually without a reason. Final outcome, months. The actual cost of the order against the estimate.

Service and warranty

First signal, months. A field report comes back from a customer. Final outcome, years. The warranty cost over the life of the product.

In pharma the loops are long because biology and regulators are slow. The FDA's goal for a standard new drug application is a decision within ten months of the filing date. [2] An opposition at the European Patent Office was decided on average about twenty months after the opposition period ended. [3] In return the outcomes are written by an authority and come with reasons.

Lab research

First signal, days. An assay reads out. Final outcome, years. The programme advances to the next stage or stops.

Quality and manufacturing

First signal, days. A deviation is assessed and classified. Final outcome, years. The batch is released, and the record holds in an inspection.

Clinical operations

First signal, weeks. Data queries and monitoring findings come back from the sites. Final outcome, years. The trial reads out.

Regulatory affairs

First signal, months. The agency sends its questions on a filing. Final outcome, years. The agency approves or refuses.

IP portfolio

First signal, days. A senior attorney edits the draft. Final outcome, decades. The product stays exclusive until the patents expire.

The map says how long a loop is. For a system that learns from work, a third fact matters. Where does the signal land, relative to the person who acted?

On the same desktop

The engineer starts a run and reads its result. One person's record holds the action and the signal, so Ambient sees the whole loop. Five teams: Chip verification, Chip implementation, Sign-off, Simulation and Lab research.

On another desk

The signal lands with another team or in another system of the company, often weeks later. Multiplayer joins the two records. Nine teams: Process and yield, Equipment service, NC programming, Automation, Design, Engineering change, Quality engineering, Costing and quoting and Quality and manufacturing.

Outside the company

An agency, a patent office, a trial site or a customer writes the signal. It has to be joined to the work from outside. Four teams: Service and warranty, Clinical operations, Regulatory affairs and IP portfolio.

Only five of the eighteen first signals land on the desktop of the person who acted. And even there the final outcome lands elsewhere. Silicon comes back from the fab, and a test result comes back from the lab.

So a record of one person's work is where learning starts and not where it ends. Most of the reward in a company is earned on one desk and written on another.

Start where the first loop is short, the outcome is clear, and the final outcome is recorded inside the company. On this map that is chip implementation and verification, NC programming, automation and engineering change.

Pharma is the other strategy. The loops are long, so the work is to learn a value function across delays. The reward for that patience is that the outcomes are unusually well written.

This map is not a measurement. The times are orders of magnitude, taken from public descriptions of each workflow and from our own reading. The scores are our judgment. In a deployment both are measured per team, from the record of the work.