Phân tích dữ liệu với AI

Delegation-Diligence applied to analysis

Nhắc lại từ Bài 14.1:

Với analysis, trọng tâm chuyển từ "tôi prompt tốt không" sang "AI output có đúng không". Tức là: Discernment quan trọng hơn Description cho analytical tasks, nhiều khi.

  ┌──────────────────────────────┐
  │  DELEGATION                  │
  │                              │
  │  Should AI do this task?     │
  │  Which parts?                │
  │  Which tool?                 │
  └──────────────────────────────┘
              ▲
              │ iterate
              ▼
  ┌──────────────────────────────┐
  │  DILIGENCE                   │
  │                              │
  │  Did AI do it right?         │
  │  Can I verify?               │
  │  Do I own final result?      │
  └──────────────────────────────┘

Phương pháp "Known-Answer Testing"

Đây là cách Rio validate AI's analytical capabilities cho công việc cụ thể của anh:

Điểm hay: bạn validate công cụ, không chỉ 1 output. Sau khi validated approach, bạn có thể apply với confidence cho new data.

┌────────────────────────────────────────────────┐
│                                                │
│     KNOWN-ANSWER TESTING METHOD                │
│                                                │
│   1. Chọn analytical task                      │
│      thường làm                                │
│              │                                 │
│              ▼                                 │
│   2. Tìm past data đã phân tích trước         │
│      (biết câu trả lời đúng)                   │
│              │                                 │
│              ▼                                 │
│   3. Work với AI reproduce analysis            │
│      using Description-Discernment             │
│              │                                 │
│              ▼                                 │
│   4. Compare AI result với known answer        │
│              │                                 │
│      ┌───────┴───────┐                         │
│      ▼               ▼                         │
│   ✅ MATCH      ❌ MISMATCH                    │
│   → Trust for    → Identify gap                │
│     future       → Refine / or don't delegate  │
│     similar      this task                     │
│                                                │
└────────────────────────────────────────────────┘

Kịch bản: Rio test AI cho quarterly analysis

Rio mỗi quý phân tích program attendance + employment outcomes cho job training program. Cụ thể:

Analysis này thường lấy 6-8 giờ của Rio mỗi quý (data cleaning + formulas + interpretation).

Step 1: Delegation decision

Rio think through:

Decision: Test AI cho heavy-lifting phân tích. Keep interpretation.

Platform choice: Claude Team plan (no training on user data). Data có PII participants — cần safe tier. Rio sanitize file trước upload (anonymize participant IDs).

Step 2: Test setup

Rio uploads:

Wait — Rio không upload answer key. Điểm của test là: AI work từ raw data, Rio check output against known answer offline.

Step 3: First prompt

AI responds với summary. Rio reviews:

Match:

Mismatch:

Step 4: Iterate — refine description

Rio doesn't reject AI. He refines:

AI catches it this time. Rio notes: "For future quarters, I need to specifically request program-type consideration."

Step 5: Test harder

Rio probes giới hạn:

AI response:

Rio realizes: AI wasn't given that data. He notes: "AI needs explicit enrollment dates để do cohort analysis. Otherwise it'd infer — which I don't want."

Results of testing

Sau test run, Rio has:

Critically: Rio hasn't blind-trusted AI. He now has tested approach he can confidently apply — with clear notes về data to include và context to add himself.

Next quarter: Rio applies this approach with fresh data. Analysis takes ~1.5 giờ instead of 6-8. Diligence continues: check numbers make sense, take accountability for final report, be transparent về AI role.

Savings: 4-6 giờ per quarter. Confidence level: validated, not blind.

• Participation rates
• Monthly attendance changes
• Correlation giữa attendance và job placement
• Breakdown theo cohort (enrollment date) và program type
• Interpretation & decisions: Giữ. Rio muốn continue ra strategic decisions dựa trên data.
• Data cleaning + formulas: Candidates for AI — tedious, error-prone manually.
• Pattern identification: Candidates for AI — potentially faster.
• Contextual nuance: Giữ với Rio — AI không biết lịch sử chương trình, staff changes, ngoại cảnh.
• Last quarter's raw data (sanitized)
• Last quarter's completed analysis (Rio's hand work) — as hidden "answer key"
• ✅ Correlation giữa attendance và job placement — số liệu matching Rio's hand analysis
• ❌ AI missed critical insight về combined housing assistance + job placement program — participants trong program kết hợp có outcomes khác hẳn.
• ✅ Validated AI can reproduce correlation analysis với right description
• 🟡 Identified gap: AI needs explicit program-type framing
• 🟡 Identified gap: AI can't infer enrollment dates — must be in data
• ✅ Documented what to include trong prompts cho future quarters

Điều gì nếu bạn không "data-savvy"?

Bạn có thể đọc kịch bản Rio và nghĩ: "Nhưng tôi không có known answer. Tôi never analyzed this data trước đây. Tôi không confident enough to spot errors."

Câu hỏi hợp lý — và AI vẫn có thể giúp, với approach khác.

AI models đặc biệt giỏi với coding tasks — bao gồm data manipulation, Excel formulas, statistical operations. Khi bạn không sure about data analysis:

1. Treat AI as co-analyst, không phải authority

Ask AI explain process, không chỉ give answers:

You learn along the way. AI becomes tutor, not oracle.

2. Ask for explanations của each step

Khi AI gives result:

If AI can't clearly explain reasoning, that's signal to distrust result.

3. Test understanding với simple cases

Start với simple question bạn can verify manually. Build trust từ đó:

Verify manually. Match? Good. Try slightly more complex. Build up.

4. Use AI để learn, không chỉ execute

Over time, AI tutors you. Bạn become more "data-savvy". Rio-level validation becomes accessible.

Validation builds confidence — không eliminate responsibility

Sau validation, you still:

Validated ≠ unmonitored. Testing gives you evidence for confidence. Ongoing diligence maintains that confidence.

• ✅ Check numbers make sense against what you know about programs
• ✅ Take accountability for final report
• ✅ Be transparent về AI role (Creation + Transparency + Deployment Diligence)

Bảng so sánh: Analysis alone vs. with AI fluent

Key insight: AI doesn't replace your analytical judgment — it amplifies it. Bottleneck shifts từ manual manipulation sang strategic thinking.

Ví dụ theo ngành — Data analysis tasks nonprofit

💰 Development / Fundraising

Pain: "Analyze donor giving patterns across 5 years — retention, upgrades, churn."

Approach:

📊 Program Evaluation

Pain: "Annual program evaluation — outcomes, costs, demographic breakdowns."

Approach:

🏥 Health / Social Services Outcomes

Pain: "Patient outcomes data — service utilization, satisfaction, unmet needs."

Approach:

📈 Grants Reporting

Pain: "Quarterly grant reports — service delivery metrics, compliance tracking."

Approach:

🗳️ Advocacy / Policy

Pain: "Analyze legislative voting records, donor influence, policy outcomes."

Approach:

🤝 Volunteer Management

Pain: "Volunteer retention analysis — who stays, who leaves, why."

Approach:

📣 Communications / Marketing

Pain: "Analyze email/social engagement to optimize outreach."

Approach:

🏢 Operations / Executive

Pain: "Dashboard prep cho board meetings — synthesize metrics from multiple sources."

Approach:

• Sanitize donor file (anonymize names, use donor IDs)
• Test AI với past analysis (quý trước bạn đã làm manually)
• Validated approach: cohort retention, LTV ranges, acquisition channel effectiveness
• Apply với full dataset next analysis
• Kết quả: 2 ngày → 4-6 giờ, với deeper insights
• Known answer: last year's evaluation
• Test: can AI reproduce headline findings?
• If validated: scale to this year's data
• Output: evaluation report draft (human polishes interpretation)
• Kết quả: 2 tuần → 3-4 ngày
• Strip PHI (per HIPAA/privacy law — consult compliance)
• Aggregate sensitive categorical info
• Test correlations AI identifies against clinical intuition
• Flag for human expert review: any causal-sounding claims
• Kết quả: Faster insights, clinical team involvement preserved
• Standardize output format through Projects / custom instructions
• Validate against past approved reports
• AI generates quarterly update, human verifies, submits
• Kết quả: Consistency + time savings
• Public data — less privacy concern
• Test AI analysis against published analyses (known answers)
• Use cho tracking systemic patterns
• Kết quả: Richer advocacy briefings in less time
• Anonymize volunteer data
• Test correlations (training attendance, role type, tenure → retention)
• Identify risk factors cho churn
• Apply insights to volunteer engagement strategy
• Kết quả: Data-driven retention improvements
• Lower stakes data (your own public communications)
• Great starting point cho people new to AI analysis
• Test across known-performing content
• Build voice intuition into data backing
• Kết quả: Evidence-based content strategy
• Upload pre-sanitized data từ program, financial, development
• Template prompt reused quarterly
• AI generates dashboard + commentary
• Kết quả: Board-ready in 2 giờ instead of 2 ngày

Prompt templates cho analysis

1. Pattern identification

2. Comparison / segmentation

I'm sharing [dataset type]. Context: [how collected, what represents].

Please analyze and identify:
- Top 3-5 patterns in [dimension of interest]
- Outliers worth investigating
- Potential correlations (NOT claimed causation)

For each pattern:
- Describe pattern specifically
- Cite which data rows/cases support it
- Note confidence level (strong signal vs. suggestive)
- What additional data would strengthen finding

Important: Don't speculate beyond what data shows. Flag 
any inferences as inferences, not facts.

2. Comparison / segmentation

3. Trend analysis

Segment this data by [variable]:
- Group A: [criterion]
- Group B: [criterion]
- Group C: [criterion]

For each group, provide:
- N (sample size)
- Key metrics distribution
- Differences from overall population

Then compare groups:
- Most statistically meaningful differences
- Caveats (small sample sizes, confounds)
- Questions this raises (not answers)

3. Trend analysis

4. Correlation vs. causation check

Analyze trends in [metric] over [timeframe].

Look at:
- Monthly/quarterly direction
- Acceleration or deceleration
- Seasonal patterns
- Anomaly periods

Output:
- Trend narrative (plain language, 200 words)
- Data supporting each claim
- 2-3 hypotheses for what driving trend
- What would refute each hypothesis
- Recommended follow-up analysis

4. Correlation vs. causation check

5. Excel formula / code help

Data shows [observed pattern].

Before I conclude [causation claim], challenge me:
- What alternative explanations could produce this pattern?
- What confounding variables might I be missing?
- What additional data would help establish causation?
- What's weakest link in the current inference chain?

Be skeptical, not confirmatory.

5. Excel formula / code help

6. Statistical sanity check

I'm trying to [describe goal] in Excel/Sheets.

Current data structure: [describe columns]
Desired output: [describe]

Please:
- Suggest formula approach
- Walk through logic step-by-step
- Show formula with example data
- Explain what each part does
- Flag edge cases that might break it

I want to understand, not just copy-paste.

6. Statistical sanity check

7. Data cleaning

Claim from my analysis: [claim]
Supporting data: [numbers]

Please sanity-check:
- Is the math correct?
- Are sample sizes adequate for confidence?
- Are there obvious confounds I'm missing?
- Is the effect size practically meaningful or trivially significant?
- How would a skeptical reviewer challenge this?

Be direct about weaknesses.

7. Data cleaning

8. Visualization suggestions

I have messy data. Issues I know about:
- [issue 1: e.g., inconsistent date formats]
- [issue 2: e.g., missing values]

Please:
- Show specific examples of inconsistencies from data
- Suggest standardization approach
- Generate cleaning formula/code
- Flag any ambiguous cases requiring human decision
- Preview results of cleaning (before/after samples)

8. Visualization suggestions

9. Survey response synthesis

Data: [describe]
Purpose: [presentation to board / funder / community]
Audience sophistication: [basic / moderate / expert]

Recommend 3-5 visualization types that would tell this story:
- Chart type
- What it highlights
- What it obscures
- Accessibility considerations (color blindness, screen readers)
- Tool to create it (Excel, Datawrapper, other)

Rank by effectiveness for this audience.

9. Survey response synthesis

10. Outcome / impact analysis

Data: [N] survey responses.
Fields: [describe]

Please synthesize:
- Top 5 themes (with frequency counts)
- Representative quote for each theme (anonymized)
- Outlier responses worth investigating
- Gaps (questions respondents didn't answer, possible reasons)
- Recommendations for next survey (what to ask, what to drop)

Keep person-first, asset-framing language throughout.

10. Outcome / impact analysis

Program: [describe]
Outcomes data: [what tracked]
Baseline: [pre-program or comparison group]

Please analyze:
- Change from baseline by metric
- Statistical significance of change (caveat assumptions)
- Who benefited most / least? Why might that be?
- What change happened but might not be attributable to program?
- What additional evidence would strengthen causal claim?

Frame as evidence-based honest assessment, not marketing narrative.

Anti-patterns — Sai lầm trong data analysis

❌ Accept first AI interpretation as truth

Triệu chứng: Upload data, AI interprets, you use.

Tại sao là sai: First interpretation often misses context, biases, confounds.

Cách đúng: Always challenge: "Alternative explanations? Confounds? What would refute this?"

❌ Upload non-sanitized sensitive data

Triệu chứng: "AI promised no training, so it's OK."

Tại sao là sai: Belt-and-suspenders. Still risks around retention, access, incidents.

Cách đúng: Strip PII even on trusted tools. See Bài 14.4.

❌ Fabricate "AI said X" credibility

Triệu chứng: Claim in report: "AI analysis shows that..."

Tại sao là sai: AI isn't authority. It's tool. Claim obscures who did analysis.

Cách đúng: "Analysis shows..." + transparency footnote about AI assistance.

❌ Assume AI numerical accuracy

Triệu chứng: Paste data, ask "what's the average", use AI answer.

Tại sao là sai: LLMs can make arithmetic errors, especially with larger numbers.

Cách đúng: For critical calculations, use tools that actually compute (Excel, Python, specialized AI với code execution). Or verify manually.

❌ Miss contextual factors only you know

Triệu chứng: AI says "Program X outperforms Program Y" — you publish.

Tại sao là sai: AI doesn't know Program X had 3x the staff support that year, or served a pre-selected population.

Cách đúng: Always overlay your contextual knowledge on AI patterns.

❌ Let AI do full analysis + interpretation

Triệu chứng: AI generates entire program evaluation report. You sign.

Tại sao là sai: Missing your judgment — the part that actually matters for org.

Cách đúng: AI analyzes + drafts. You interpret + decide. Division of labor.

❌ Run same prompt tweaked until getting desired answer

Triệu chứng: AI says Program X didn't work. You rephrase until AI says it did.

Tại sao là sai: Confirmation bias. Dishonest to your community.

Cách đúng: Take negative findings seriously. Ask AI why Program X underperformed.

❌ Ignore AI's expressed uncertainty

Triệu chứng: AI says "data too sparse for confident conclusion". You publish conclusion.

Tại sao là sai: You override AI's appropriate humility.

Cách đúng: When AI flags uncertainty, take seriously. Get more data or caveat finding.

Mẹo nâng cao

Mẹo 1: Adversarial prompting

After AI analysis, ask:

Self-adversarial reveals blind spots.

Mẹo 2: Compare 2 AI models

If stakes high:

Different training data / architectures reveal biases of each.

Mẹo 3: Ask AI to estimate confidence

Helps calibrate how much weight to give each finding.

Mẹo 4: Iterate on time-cost vs. nuance

First pass: broad strokes (5 mins). Second pass: refinement on key findings (15 mins). Third pass: sanity checks (10 mins).

Don't spend 2 hours iterating when 30 mins gives 80% value.

Mẹo 5: Archive validated prompts

When you've validated an analytical approach:

Build internal library of "known-good" prompts.

Mẹo 6: Share validation với team

Rio's testing insight applies to team:

• Run same prompt trên Claude AND ChatGPT (or Gemini)
• Compare conclusions
• Where they differ, dig in
• Save the prompt
• Save notes on what AI tends to miss
• Reuse for similar future tasks
• Document what AI handles well / poorly cho your specific data
• Share với colleagues doing similar analysis
• Collective validation > individual trial/error

Áp dụng ngay

Bài tập 1: Messaging analysis (~30 phút) — LOW STAKES

This exercise uses low-stakes data (your own public communications) to practice Description-Discernment loop cho data analysis.

Part I — Gather data:

Collect 10-20 examples of your org's communications — social media posts, email subject lines, newsletter headlines, event announcements. Mix high-performing và lower-performing content.

Part II — Analyze với AI:

Share dataset với AI. Prompt:

Part III — Apply Discernment:

Reflection:

Stretch goal: Ask AI audit how messaging compares to stated mission/values. Find discrepancies. Create messaging guide from analysis.

Bài tập 2: Donor giving patterns (~40 phút) — MEDIUM STAKES

Apply data analysis to fundraising data, building on Bài 14.4 hygiene practices.

Part I — Prepare data:

Use sanitized donor dataset (anonymized từ Bài 14.4 exercise), hoặc prepare new by removing PII. Ensure historical giving across multiple periods.

Part II — Analyze:

Ask AI identify patterns in:

Part III — Apply Discernment:

Reflection:

Bài tập 3: Community needs trend analysis (~30 phút) — HIGHER STAKES (STRETCH)

Advanced: combine multiple data sources for predictive analysis.

Part I — Gather sources:

Collect info you use understand community needs:

Part II — Analyze emerging patterns:

Ask AI identify:

Part III — Rigorous Discernment:

Highest level of critical evaluation:

Reflection:

• Do identified patterns match intuition?
• What context is AI missing (audience, goals)?
• Any surprising patterns?
• What are you trying to learn từ this dataset?
• How does high-performing content align với authentic voice + values?
• Are we reaching right audience?
• Donor retention rates over time
• Recurring vs. one-time donation patterns
• Campaign effectiveness comparisons
• Giving trends by amount ranges
• Do trends match what you know về donor base?
• Is AI focusing only monetary value, missing relationship factors?
• What patterns strengthen donor relationships, not just maximize revenue?
• Costs of implementing efficiency recommendations? (Focusing on majors at expense of smalls → impact on community perception or long-term sustainability?)
• Patterns strengthening relationships beyond giving amounts?
• Your program data + service requests
• External reports / datasets about community
• News / policy developments affecting constituents
• Trends in support types requested
• External factors increasing or changing demand
• Gaps between current services and emerging needs
• AI predictions vs. direct community experience?
• Systemic factors / local context AI missing?
• Values to keep in mind anticipating needs with dignity + respect?
• How approach process responsibly?
• Factors / systemic issues explaining or contextualizing what AI cannot?

Phản xạ bài học

• How did testing AI against known data change your confidence using it for new analysis?
• What gaps/limitations did you identify that shape how you'll delegate analysis in future?
• Which D (Delegation / Description / Discernment / Diligence) felt most stretched in this exercise?

Tóm tắt bài học

🎯 Test AI against data you already understand — build validated confidence, not blind trust.

🎯 Use Discernment identify gaps in AI reasoning — note where AI misses context, what Description you need add.

🎯 Build validated approaches, document what works — each testing round teaches you.

🎯 AI assists even if you're not "data-savvy" — use as co-analyst / tutor, ask for explanations.

🎯 Validation builds confidence, doesn't eliminate responsibility — still accountable for checking results.

🎯 Critically: AI analyzes, you interpret + decide — division of labor preserves your judgment.