How do you know a signal is still true?

Your targeting runs on signals. A funding round, an open role, a tool that showed up in someone's stack, a project that has not moved in a year.

Every one of them is a claim about the world, made on the day it was collected.

Nothing in a scoring stack turns around and asks whether that claim is still true.

The standard answer to this is decay weighting. Old signals count for less, the curve does the work, and the problem is considered handled.

That answer covers one failure and leaves a second one uncovered. The second one is where your best-looking accounts live.

Why does decay weighting miss this?

Because a whole class of signal gets stronger with age.

An open role is the clearest case. The duration is the score.

A seat advertised for 192 days says something much louder than a seat advertised for nine days, and it gets louder every week nobody fills it.

Decay weighting exists to discount old things. On this class, old is the entire point.

So the number keeps climbing after the seat gets filled.

A posting nobody can fill and a posting nobody took down produce the same reading. Both are old, both keep aging, and from the outside they are one number.

Here is the part worth sitting with. Those failures do not scatter randomly through your ranking.

They climb it. The longer a listing sits there being wrong, the higher it sorts, so the errors concentrate exactly where your confidence is highest.

In one live sourcing list, the top account arrived on a role open 192 days, near the top of the age ranking across 257 tracked postings.

The seat had been filled months earlier and the listing was evergreen recruiting. The account came off the send batch.

The enrichment review is what caught it. Pulling a roster to find the right person to write to, the reviewer saw hires into that exact function and flagged it the same day.

That control is real and it works. It also runs at the speed of a person reading rosters, and it runs after the enrichment has already been paid for.

What does it mean to falsify a signal?

Write down, next to the signal, the one fact that would have to still be true for the signal to still mean what it means.

For a duration signal, that fact is usually "nobody has been hired into this function since the posting went up."

Then give it a threshold, so the check has a line to cross. Then give it a schedule, so it runs again without anyone remembering.

Measured quantity, threshold, trigger. It is the same basic control shape ML teams use for drift monitoring, expressed as three ordinary fields.

GTM already has the word. List drift, data drift, ICP drift, positioning drift.

It gets used descriptively. Ask an operator running signal-based outbound when their last drift check ran and there is usually no check to run.

Nothing in a standard scoring stack builds that check, so the word gets used and the check never gets written.

What the design has to get right

Five decisions that look correct and are not. Each one cost me a rebuild.

Storing state is not the same as validating it. The monitor writes every posting it sees into a state file and tracks ages across runs, which turns a list into a clock.

A clock keeps accurate time whether or not the thing it is timing is still happening.

Measuring a signal and validating one are separate jobs, and the first does not imply the second.

Reporting each crossing once, ever. A reported flag stops the same account showing up every week, which is correct for noise and terrible for validity.

Once a posting was reported it was never looked at again, which is the exact window where the seat gets filled.

Treating "no evidence" as "no hires." The first instinct is to let a company with nothing recorded against it pass. That collapses two very different states.

Somebody looked and found nothing is evidence. Nobody looked is an absence of evidence, and it has to fail closed or the checker inherits the blindness it was built to fix.

Trusting a thorough web search as proof of absence. Two research agents ran clean searches on one company and recorded no hires. A stronger source already held the answer, and there had been a hire in March.

A good search is honest evidence that somebody looked. It is not proof that nothing happened, and where sources disagree the verified one wins.

Reaching for a second signal that cannot be said out loud. SEC Form D filings are free, public, and excellent for finding raises that never got a press release.

Opening a conversation with a filing you read on EDGAR is a move that rarely lands well.

Some signals inform your targeting and never appear in your copy.

Make that visibility a property of the signal itself, a targeting_only flag as against a copy_safe one, so the call lives in the design rather than on send day.

RAW SIGNAL       role open 192 days
     |
FALSIFIERS       age ceiling . hire evidence . still listed
     |
VERDICT          VALID / SUSPECT / INVALIDATED
     |
ROUTING          rank / hold for review / exclude

What does the check actually do?

Three falsifiers ship with it, and each one prints the statement it is testing next to the evidence that moved it.

Every signal gets one of three verdicts.

Valid means every falsifier holds and it may rank. Invalidated means a falsifier broke and it is out.

Suspect means a falsifier is degraded or its check could not run, so it is held for review rather than trusted.

Worst verdict wins. A signal with two clean falsifiers and one broken one is broken.

The thresholds live in a config file rather than in the code, because 180 days is a starting number and not a measured one.

You should be able to argue with it by editing a line.

A ledger records when each signal was last checked and what came back, so the next run only re-checks what is due.

Everything else carries its stored verdict forward, marked so you can tell a fresh answer from a remembered one.

What happens on the first run?

Against a real 257-posting state file with no hire evidence supplied, 3 came back invalidated on age alone and 254 came back suspect.

That is the fail-closed rule working rather than a bug, and it is also the honest shape of the thing.

The checker is only as good as the evidence you feed it.

The strongest falsifier reads a file somebody fills in. There is no roster scraper in it and no enrichment call, which was a scope decision rather than an oversight.

Gathering hire evidence is an enrichment problem, with its own vendors and its own per-record cost.

Checking a signal against evidence you already have is a much smaller problem, and it is the one nobody had written down.

How do you run this on your own signals?

The checker is public at github.com/derrtaderr/signal-drift-detector, and the whole install is three lines.

git clone https://github.com/derrtaderr/signal-drift-detector.git
cd signal-drift-detector
python3 tests.py    # 111 tests, no network, prints OK

Python 3.8 or newer, standard library only, no keys and no network calls.

Run it against the bundled fixture first, which is synthetic and calibrated to a fixed date so the output is identical on every machine.

python3 -m sdd check \
  --state fixtures/state.sample.json \
  --evidence fixtures/evidence.sample.json \
  --as-of 2026-09-10

Then point it at your own data.

python3 -m sdd check \
  --state /path/to/your/state.json \
  --evidence /path/to/hires.json \
  --ledger ./ledger.json

--ranking-only prints the eligible signal ids and nothing else, which is the form you want when a scoring stage consumes it.

--format json gives you the verdicts with their evidence attached.

The evidence file is a small piece of JSON with a company, a date it was checked through, and a list of observed hires.

A company present with an empty list means somebody looked and found nothing. A company absent means nobody looked.

Keeping those two apart is the whole design.

What if you do not have a monitor yet?

The signal has to exist before it can be checked, so here is the instrument it came from, which took a weekend.

Scrape one narrow role family weekly with JobSpy, free, against the boards your buyer actually posts on.

Six queries are enough to start, covering the role titles a company posts when it cannot do the thing you sell.

Keep a keep-pattern and a drop-pattern on the title, because "Solutions Engineer" will flood a GTM query and teach you nothing.

Write every posting you see into a state file with the date you first saw it. This is the part that matters.

A scraper answers what is true today. A monitor answers what has been true for a while, and only the second one describes a problem someone is motivated to pay to solve.

Report new crossings past a threshold rather than the standing list.

Sixty days was the starting line, chosen because two months of an unfilled seat is long enough to hurt and short enough to still be live.

Stack a free second signal on each crossing. Google News RSS finds funding and launches, SEC Form D finds raises that never got announced.

Both cost nothing and take seconds, and you only run them on crossings.

Seed an exclusions file from whatever you already ruled out, so the instrument never resurfaces a name somebody already decided against.

Then put the falsifier check in front of enrichment, because enrichment is the expensive step and a broken signal should never reach it.

What this does not establish

One signal class, one operator, one list. Whether validity decay bites this hard on funding triggers or job-change triggers is not measured here.

The age ceiling is a heuristic. Title-to-function mapping is keyword matching, and a title nobody taught it fails closed rather than guessing.

And the first version consumes hire evidence rather than gathering it, so the honest description is that the strongest check depends on a file a human keeps current.

What it does do is turn a habit into a rule.

The reviewer who caught the 192-day listing was running a check that had never been written down anywhere, so automating that step would have removed it with nothing to replace it.

Worth asking of any signal you rank on. What would have to still be true, and when did anything last check?

FAQ

What is signal drift in a GTM context?

A signal drifts when the fact underneath it stops being true while the signal keeps scoring.

A job posting stays open in the data after the seat is filled, so the trigger keeps firing on a vacancy that no longer exists.

Why does decay weighting not solve signal drift?

Decay weighting discounts a signal as it ages.

On duration-based signals the age is the score, so the number climbs rather than falls and there is nothing for the curve to discount.

What is a falsifier?

The one fact that would have to still be true for a signal to still mean what it means.

Storing it next to the signal is what makes the signal checkable rather than merely scorable.

How often should a signal be re-checked?

Match the interval to how fast the underlying fact can change.

Seven days is the default here because hiring moves on roughly that cadence. A quarterly fact does not need a weekly check.

Do I need the vacancy monitor to use the checker?

No. It reads a state file with a company, a title, a posted date and a last-seen date per signal.

Anything that produces those four fields works.