designing-analytics-projects

Designing Analytics Projects — the Brief

These are suggestions based on the user's CEU MSBA course Designing Analytics Projects (ECBS5228A), taught by Eduardo Arino de la Rubia (ex-Meta, ex-Domino). Source: https://github.com/earino/designing-analytics-projects.

The course centres on one artifact: the Analytics Project Brief. Everything in this skill exists to help write one well.

Cardinal Rule — Use What You're Given

A brief is only as good as its grounding in the scenario.

If the scenario, scoping notes, stakeholder list, data dictionary, prior analysis, or any other local file mentions a number, name, metric, system, or business rule — use it verbatim. Do not invent.

When information is missing, say so explicitly ("not specified in scenario; recommend confirming with X") rather than fabricating a baseline, headcount, KPI, or stakeholder name. Made-up specificity is the single most common failure mode for AI-drafted briefs — it looks professional and is wrong, which is worse than vague.

If the user gives a scenario file, read it first and extract:

• Company name, business model, current metrics (registered users, MRR, NPS, etc.)
• Named stakeholders, their roles, and their stated motivations / KPIs
• Available data tables and known data-quality caveats
• Stated constraints (time, sample size, privacy, can't survey, etc.)
• The exact ask (what was requested vs what's actually needed)

Quote those facts directly in the brief instead of paraphrasing into something more generic.

When to Use

• "Draft / review / critique an Analytics Project Brief" or "project brief" or "scoping doc."
• A scenario file is present and the deliverable is a written brief.
• The user is preparing to talk to stakeholders, kicking off a new analysis, or sanity-checking scope.
• Any mention of: counter-metrics, guardrails vs tradeoffs, Goodhart's Law, pre-mortem, Power-Interest Grid, stakeholder map, decision criteria, "what breaks if we succeed."

Do not auto-trigger for purely technical tasks (training a model, writing SQL, building a pipeline). The brief is the pre-code artifact.

The 10 Sections (one-line each)

1. Problem & Decision — what decision will this inform; who actually decides; why now; one-sentence hypothesis.
2. Metrics — primary metric defined SQL-precisely (event/table, grain, eligibility, time window) + 2–3 counter-metrics labelled Guardrail (must not worsen) or Tradeoff (may worsen within bounds).
3. Stakeholder Map — Power-Interest Grid (4 quadrants) + named Champions + named Blockers with their motivation (budget / ego / workload / KPI conflict).
4. Methodology — 1–3 methods, each tied to a specific hypothesis and the data required, plus Stop/Go data-validity checks.
5. Scope & Deliverables — In Scope, Out of Scope (the line that prevents creep), concrete deliverables.
6. Success & Decision Criteria — analytical success vs business success, decision forum + action owner, pre-committed decision table ("if we find X, we will do Y; if inconclusive, …"), action thresholds.
7. Timeline — milestones with dates, not vibes.
8. Risks & Assumptions — assumptions, risks with L/M/H likelihood × impact, mitigations.
9. Ethics & Privacy — PII? bias against protected groups? GDPR review? mitigations.
10. Pre-Mortem — "It's 3 months from now and this failed. What happened?" Tell the causal story ("we did X, Y happened, because Z"). This surfaces the risks Section 8 misses.

A blank template lives at ~/repos/ceu/designing-analytics-projects/templates/analytics_project_brief.md and as snippets/brief_template.md in this skill. Use it as the literal scaffold.

Quality Bar — what makes a brief strong

The course's rubric (see syllabus.md) rewards four things, in this order:

1. Metric definition precision. Not "conversion rate" but "users with signup_complete on day 0 → users with ≥1 app_open on calendar day 7, eligible cohort: web signups in last 6 months." If you can't write the SQL, the definition isn't done.
2. Counter-metrics that show adversarial thinking. What breaks if we hit the target? Sugar-diet growth, zombie retention, casual-user alienation, brand-trust erosion. Two to three is the right number — five looks like padding.
3. Stakeholder analysis that names blockers and their motivation. "Head of Growth" is a placeholder. "Head of Growth — bonus tied to signup volume, will resist any onboarding friction" is analysis.
4. A pre-mortem that surfaces non-obvious risks. Not "the data could be bad" — a causal story: "By month 3 the recommendation was shipped, retention didn't move, and the post-mortem found that Learning Paths were correlated with retention because engaged users self-selected into them, not because the feature caused engagement."

High-Leverage Patterns

Counter-metric framing — for each candidate primary metric, ask: what's the laziest way to hit this number, and what would break? Examples from the course cheatsheet:

Pre-mortem prompt that consistently produces useful output:

Imagine it's 3 months from now. We shipped what this brief proposes. The project failed — not in a vague way, but specifically. Tell the causal story in 3 sentences: what we recommended, what happened, and the reason it didn't work that we missed today.

Stakeholder Power × Interest in 30 seconds — for each named person:

Then for each High-Power-High-Interest person, decide: Champion or Blocker? If Blocker, what's the motivation (KPI conflict, budget, ego, workload, prior burn)? Pre-brief privately before any group meeting — no surprises.

Anti-Patterns to Flag in Reviews

• Generic stakeholder list copied from a slide deck (no names, no motivations).
• Primary metric without a SQL-grade definition (no event, no grain, no eligibility, no time window).
• No counter-metrics, or counter-metrics that are just other primary metrics.
• "Explore the data" as the methodology — that's not a project, it's a fishing trip.
• No "Why now?" — without urgency, the brief will not get prioritised.
• Decision criteria written after results are in. Pre-commit, in the brief.
• Pre-mortem as a generic risk list ("data could be incomplete"). It must be a story, not a checklist.
• Inventing numbers ("we estimate ~20% lift") when the scenario gave none. Say "to be confirmed with stakeholder" or use the scenario's stated numbers.
• Treating "Out of Scope" as optional. It's the section that protects you when the ask quietly grows.

Worked Examples in the Repo (read at least two before drafting)

In ~/repos/ceu/designing-analytics-projects/templates/examples/ there is one fully-worked brief per foundational analysis:

Pick the example whose analysis type matches the scenario at hand and mirror its section depth, table formats, and tone.

The user's own assignment brief — ~/repos/ceu/designing-analytics-projects/assignments/learnloop_project_brief.md — is also a strong reference for the depth and style expected on a real submission, drafted from scenarios/scenario_03_learnloop.md.

Foundational Analysis Cheatsheet (which one fits the scenario?)

Match the business question to an analysis type, then mirror that example brief.

Source: cheatsheet.md in the course repo (the one-A4 exam cheat sheet — concentrated wisdom).

Suggested Workflow When Drafting from a Scenario

1. Read the scenario twice. Highlight every number, name, system, and constraint.
2. Pick the foundational analysis (table above). Open the m

Content truncated.