Behind The Feed Data Challenge · Final pitch

Who made your sandwich?

Unwrap, by Sandwich Index. Measure, rebalance and label the feed.

Brian ChoMath and Design
Ali ShaikhComputer Science and Finance
Scroll, or press ↓ or space

Your feed is a sandwich someone else makes.

7:02
Lock screen · this morning
NewsMarkets open lower after rate call
TVLast night's finale, explained
SportsLate goal settles the derby
HealthNew clinic hours start Monday
NewsStorm warning extended to Friday
Illustration
JellyLaid on thick: the things you already click.
Peanut butterSpread thin: the plainer things you need.
An engine made it, and like any cook who wants you back tomorrow, it serves what keeps you eating.

How lopsided? We opened one week of real Microsoft News logs.

Each dot is 10 stories.
The most-shown 1% of stories took 55% of all views
Every other story shown: 45%
51,000
stories, shown to readers 5.8 million times
55%
of those views went to about 500 stories: the most-shown 1%
6 in 10
stories were never shown to anyone

Who pays for a sandwich that lopsided?

The catch

Some people simply have narrow tastes. That isn't the algorithm's fault.

So a fair answer needs a new yardstick.

Same reader, two feeds
The feed the engine served them
News46%
TV13%
Sports11%
Music8%
Other22%
The feed they'd get if the engine didn't know who they were
Lifestyle26%
Entertainment22%
Finance10%
News10%
Other32%
One simulated reader, over the whole run. Share of stories served, by topic.
The gap between the two is the Sandwich Index.
1 − variety of your feed ÷ variety of your own non-personalized feed
Every reader is measured against themselves, not against everyone else.

The narrowest tastes lose two and a half times more.

The replay
  • 500real readers, replayed through a feed that learns from every click
  • ×4full runs
12%
narrower on average, measured against each reader's own non-personalized feed
Each group's own non-personalized variety
Narrowest tastes
20%narrower
Second quarter
12%narrower
Third quarter
10%narrower
Broadest tastes
8%narrower
2.5×
the damage, landing on the readers with the least variety to spare
And nobody sees it happen.

So we built Unwrap: three parts, all running on one log.

The log a platform already keeps
reader U13740story N55189topic sportsshown 1… one row for every story shown
1 · MEASURE

Audit toolkit

Turns that log into every reader's Sandwich Index.

One number
2 · REBALANCE

Balanced Diet dial

One extra term in the ranking moves the held-back topics up the feed.

One term
3 · LABEL

The Napkin

A label that shows each reader what they have been served.

One label

The napkin shows what was really in the sandwich.

One reader, dial off: the serving is 84% jelly.

Now turn the dial to 0.7.

One more bite, and the variety comes back.

Not only this reader: all 500, in every run.

Every topic leaves a mark
JamA topic the engine pushed.
Peanut butterA topic it held back.
Clean spotA topic that never arrived.
Pushed and held back are judged against the reader's own non-personalized feed.
One simulated reader · narrow tastes · dial off
84%
of this serving is jelly
8 of 17
topics are clean spots
15%
narrower than their own non-personalized feed
The same serving as a sandwich: thickness is share.
1.00.90.80.70.60.5
Balanced Diet dial
1.0
The ranking becomes
100%what you're likely to click
30%
what you've been missing
Same reader · one more bite · dial at 0.7
84%71%
of this serving is jelly
85
clean spots. First smears: Autos, Health, Travel.
15% narrower11% broader
than their own non-personalized feed
Narrowest tastes
0.3%broader
was 20% narrower
Second quarter
1.6%broader
was 12% narrower
Third quarter
3.0%broader
was 10% narrower
Broadest tastes
4.3%broader
was 8% narrower
Every group got back the variety it had lost, or more. Over 99% of clicks kept.

What the reader keeps: a nutrition label for their feed.

Feed Nutrition Facts

One reader · 17 news topics · dial off
Diversity grade C+
vs. non-personalized feed 15% narrower
Jelly : peanut butter 84 : 16
Clean spots 8 of 17
Clicks kept 100%

It grades variety. This reader went from a C-plus to an A.

Its numbers come from the same audit toolkit a regulator would run, so the reader and the regulator see the same thing.

That's Unwrap
1
One logthe platform already keeps
2
One numberto measure
3
One dialto fix
4
One labelto show

That fix is not free.

We tested it against the standard method, MMR. It beat us.

But MMR widens everyone by the same rule.

So we gave MMR our target.

The price of our dial, in simulation
Clicks kept
over 99%
Relevance kept
about 90%
Relevance is how well each story fits the reader. The dial gave up about 10% of it.
The price of our dial, in simulation
Clicks kept
over 99%
Relevance kept
about 90%
Method
Relevance kept
Narrowest readers, vs. their own non-personalized feed
MMRthe standard method for adding variety
96%
16% broader
Our dialat 0.7
89%
back to parity
MMR, matched to our dialsame average variety added
100%
3% narrower, in every run
MMR's method + our targetaimed at each reader's own baseline
99%
1% broader, every group back
A fix doesn't have to be the strongest.
It has to know where to aim.
Means of four simulated runs. The last row exists in simulation only.

What we can't claim yet, and who has tried it.

Limits

All of this is simulation.

  • Clicksare modelled, not observed.
  • Datais one week of news, from 2019.
  • Nexta live pilot. That is the real test.
Outside our team

Six people have tried the Napkin.

Faculty advisor
HCI researcher
Visiting doctoral student
UX designer
Tester
Tester

Most said the metaphor works. Two pushed back:

Could publishers game the labels?
Would readers ignore them?

Our report answers both.

Why now: the law already asks for this.

EU Digital Services Act · the largest platforms must
  • Assesswhat their recommender systems do to people. Arts. 34–35
  • Offera feed that isn't built on profiling. Art. 38
With Unwrap, a platform, a regulator or a reader can find out
What went into the sandwich→The Napkin
Who it's shortchanging→The Sandwich Index
How to give the variety back→The Balanced Diet dial
Why it can ship
Runs on the log platforms already keepOne extra ranking term, no new modelNext: a live pilot

So tomorrow morning, when your feed hands you breakfast,

check the napkin.

QR code for unwrap.higgsfield.app
Try the Napkin
unwrap.higgsfield.app
Unwrap, by Sandwich Index
Brian ChoMath and Design
Ali ShaikhComputer Science and Finance
0:00