Skip to content
NewProduct Q&A: shoppers ask on your product page and get your answers.See how it works
IdukkiIdukki
Strategy

AI in the UGC loop, part 3, moderation: the layer most teams skip

Brand-safe goes past profanity filtering. The gap is the competitor logo in the unboxing clip. Run a three-tier queue with human-review SLAs.

Black Friday morning, a customer unboxing video goes live in your homepage gallery. It passed the profanity filter. In the background sits a competitor's box, logo facing the camera. Moderation is the layer that catches that before your shoppers do.

In this article

Most UGC moderation programmes look like this: a profanity filter set up once and never tuned, a "report this" button on the gallery clicked twice a year, and a Slack channel where someone occasionally posts "did anyone approve this clip?" Call it hope. Nobody would call it a moderation programme.

Real brand safety is the gap between catching the swear word and catching the competitor’s logo in the background of the unboxing video that just went live on your homepage. Catching the swear word is table stakes. Catching the logo is what saves you a difficult conversation with the CMO. Twenty years of enterprise work taught me that the unglamorous layer is usually the one that matters, and this is that layer.

What does "brand-safe" actually mean now?

The list of things a moderation layer has to catch has grown long:

  • Profanity, hate speech and slurs: the classic filter, still necessary, no longer sufficient.
  • Faces of minors, almost every brand’s policy says no, almost no brand actively detects it.
  • Competing brand logos and packaging in the background of a clip.
  • Unsafe product claims, "cleared my acne in three days" on a beauty PDP is a regulatory issue.
  • Copyrighted music, especially in clips repurposed across platforms.
  • Sensitive context where the brand association is simply wrong.
  • Quality signals: vertical-only, low-light, unstable handheld footage that looks bad on a PDP regardless of legality.

A profanity filter catches exactly one item on that list. The rest need vision, language understanding and brand context, which is the AI part. That context starts upstream at tagging: the machine-readable layer described in AI content tagging for UGC is what lets the moderation model reason about what is in the frame. The AI on its own is not enough though. What makes a moderation programme actually work is the queue model wrapped around it.

The three-tier moderation queue

The biggest mistake brands make is treating moderation as binary: approved or rejected. Real moderation runs three tiers, because the cost of a wrong call differs wildly from one category to the next.

Tier 1, hard auto-reject

The unambiguous stuff lives here: clear profanity, faces of minors, high-confidence competitor logos, copyrighted music with a strike. The model rejects it, the asset never reaches a human, and the creator gets a polite templated note with the reason. Zero human time spent.

Tier 2, soft flag, human review

The grey area is where the work is: "probably a minor, low confidence", "possible competitor product, partial occlusion", "claim language that might be regulated". Human judgement decides these, and the AI’s job is to surface the asset with the specific concern timestamped, so the reviewer is not re-watching the whole clip hunting for the problem. A good Tier 2 review takes 30 to 60 seconds.

Tier 3, auto-approve, sample audit

The clean stuff goes live. The tier most brands forget is the audit on top of it: you still sample 5 to 10% of Tier 3 for a weekly human check. The point is model drift. Catch it before it turns into a pattern of escapes.

  • 50–70%

    Tier 3 · auto-approve

    Clean, high confidence

  • 15–30%

    Tier 2 · human review

    Grey area, judgement needed

  • 5–15%

    Tier 1 · auto-reject

    Unambiguous violations

A representative healthy inbound mix for a mature programme, consolidated guidance. These are not Idukki-measured customer averages.

Where an incoming asset goes

Start here

How sure is the model about a violation?

  • Unambiguous violation

    Tier 1: auto-reject

    Clear profanity, faces of minors, high-confidence competitor logos, struck music. The creator gets a templated note.

  • Grey area

    Tier 2: human review

    The asset reaches a reviewer with the concern timestamped. A good review takes 30 to 60 seconds.

  • Clean

    Tier 3: auto-approve

    It goes live, and 5 to 10% of this tier gets a weekly human audit for model drift.

What SLAs make a queue work?

A moderation queue without SLAs is a queue that grows. Set them, post them in the channel, report on them weekly.

TierTarget SLAWhy
Tier 1 rejectUnder 1 minuteCreator gets feedback while the upload is still fresh
Tier 2 reviewUnder 4 working hoursAsset is still timely when it goes live
Tier 3 auditWeekly batchTrend monitoring; per-asset latency does not matter here
Moderation SLAs by tier.

Sub-4-hour human review is the threshold that makes UGC feel live rather than delayed. Slower than that, and you lose the trend value of the creator’s original post.

Which numbers should I track?

Track three numbers. Tier 2 P75 review latency in hours is your health-of-queue metric. Escape rate tells you, of the assets that went live, how many were later flagged as a miss. Reject-overturn rate tells you, of the assets the model auto-rejected, how many a human would have approved. A high overturn rate means the model is too aggressive and you are quietly binning good content.

“From the first interaction with Idukki, it's clear this platform is in a class of its own. It's more than just a UGC content platform on Shopify; it's a game-changer that truly revolutionizes the way businesses can leverage user-generated content.”
MOONFREEZE FOODS PRIVATE LIMITED, verbatim, Shopify App Store review, October 25 2023

Three things to do this quarter

  1. 1Write a one-page moderation policy. Not a 40-page legal doc. One page listing what is auto-reject, what is soft-flag, what is auto-approve. If it does not fit on a page, your reviewers cannot apply it consistently.
  2. 2Set the three SLAs above and post them where the moderation team can see them. Report weekly.
  3. 3Run a Tier 2 review queue, even manually for the first month. The queue model itself is the unlock. AI just makes it scale.

Last in the series: part 4, personalisation, why "newest first" leaves conversion on the table, and the maturity ladder to 1:1 matching. In Idukki today, this stage is the moderation queue and rights requests.

Get the full series. AI in the UGC loop

All four parts plus the pipeline self-audit worksheet, in one file.

Sources + note on numbers

  1. 1Bazaarvoice, content moderation + authenticity research · UGC moderation and fraud-signal benchmarks.
  2. 2TINT, State of User-Generated Content · Moderation practice survey across marketers.
  3. 3Note on numbers · The three-tier mix percentages are representative healthy ranges consolidated from the sources above and Idukki’s product experience. They are not verbatim customer-measured averages.
#ugc#content-moderation#brand-safety#ai-in-ugc-loop

Continue reading

4 pieces in this cluster

These long-form pieces on the Idukki blog link back to this article, go deeper on the cluster.

More from Rohin Aggarwal