Liberation Labs AI Tools Ethics Scorecard

Our Scorecard isn't just a reference. It's prompt engineered to be a tool. Copy this into your LLM and tell it which company or tool you'd like to evaluate. Use agentic research tools and extended thinking models for best results.

Overview

This scorecard evaluates AI tools and platforms against criteria that matter for social justice organizations, grassroots movements, and democratic participation. Each category is scored 1-10 with specific benchmarks.

Core Evaluation Categories

1. Data Rights & Privacy Protection (Weight: 20%)

What we're measuring: How well the tool protects user data from surveillance, corporate extraction, and government monitoring.

Scoring Criteria:

10 points: Open source, local processing, no data retention, strong encryption
8-9 points: Minimal data collection, clear deletion policies, strong security practices
6-7 points: Limited data sharing, some retention, basic security measures
4-5 points: Standard corporate data practices, unclear policies
1-3 points: Extensive data harvesting, surveillance partnerships, poor security

Key Questions:

Can users control their data completely?
Is the tool vulnerable to government surveillance?
Does the company monetize user data?

2. Labor Impact & Worker Justice (Weight: 15%)

What we're measuring: How the tool affects workers, content creators, and marginalized labor.

Scoring Criteria:

10 points: Augments human capacity, pays training data creators, strong labor practices
8-9 points: Designed to enhance rather than replace workers, ethical sourcing
6-7 points: Mixed impact, some worker displacement, unclear labor practices
4-5 points: Primarily automation-focused, limited worker consideration
1-3 points: Explicitly designed to eliminate jobs, exploits unpaid labor

Key Questions:

Does this tool eliminate jobs or enhance human capabilities?
Were data workers and content creators fairly compensated?
What are the company's labor practices?

3. Democratic Governance & Accountability (Weight: 15%)

What we're measuring: Who controls the tool and how decisions are made about its development and deployment.

Scoring Criteria:

10 points: Community-controlled, transparent governance, democratic decision-making
8-9 points: Non-profit with strong accountability, user advisory structures
6-7 points: B-corp or ethical business model, some community input
4-5 points: Traditional corporate structure with some transparency
1-3 points: Opaque corporate control, no accountability mechanisms

Key Questions:

Who makes decisions about how this tool works?
Can affected communities influence its development?
How transparent are the development processes?

4. Accessibility & Economic Justice (Weight: 15%)

What we're measuring: Whether the tool is accessible to organizations with limited resources and diverse communities.

Scoring Criteria:

10 points: Free/open source, multiple languages, full accessibility features
8-9 points: Sliding scale pricing, good accessibility, language support
6-7 points: Some affordability options, basic accessibility
4-5 points: Standard pricing, limited accessibility features
1-3 points: Expensive, poor accessibility, English-only

Key Questions:

Can grassroots organizations afford this tool?
Is it accessible to people with disabilities?
Does it work for non-English speaking communities?

5. Movement Security (Weight: 10%)

What we're measuring: Whether the tool protects activists and organizers from surveillance and retaliation.

Scoring Criteria:

10 points: Strong encryption, anonymous use possible, resistance to state pressure
8-9 points: Good security practices, some anonymity features
6-7 points: Basic security, limited anonymity options
4-5 points: Standard corporate security, cooperation with law enforcement
1-3 points: Poor security, active surveillance partnerships

Key Questions:

Can activists use this tool safely?
Does the company resist government data requests?
Are there built-in security features for sensitive organizing?

6. Environmental Justice (Weight: 10%)

What we're measuring: The tool's environmental impact and corporate climate practices.

Scoring Criteria:

10 points: Renewable energy, minimal compute requirements, climate justice leadership
8-9 points: Some renewable energy, efficiency focus, decent climate practices
6-7 points: Mixed energy sources, average efficiency
4-5 points: Limited environmental consideration
1-3 points: High environmental impact, poor climate practices

Key Questions:

What's the carbon footprint of using this tool?
Does the company prioritize environmental justice?
Are compute resources used efficiently?

7. Bias & Algorithmic Justice (Weight: 10%)

What we're measuring: Whether the tool perpetuates or challenges systemic oppression.

Scoring Criteria:

10 points: Actively anti-oppressive, diverse training data, bias mitigation features
8-9 points: Good bias testing, diverse development team, some mitigation
6-7 points: Basic bias awareness, standard industry practices
4-5 points: Limited bias consideration, homogeneous development
1-3 points: Perpetuates harmful biases, no mitigation efforts

Key Questions:

Does this tool reinforce racial, gender, or class bias?
Who was involved in training and testing the system?
Are there mechanisms to address harmful outputs?

8. Community Benefit vs. Extraction (Weight: 5%)

What we're measuring: Whether the tool genuinely serves communities or primarily extracts value from them.

Scoring Criteria:

10 points: Explicitly designed for community empowerment, gives back value
8-9 points: Clear community benefit, some value sharing
6-7 points: Mixed benefit, traditional business model
4-5 points: Primarily extractive with some community features
1-3 points: Pure extraction, communities used as product

Key Questions:

Does this tool empower communities or extract from them?
How does value flow back to affected communities?
Who ultimately benefits from this tool's deployment?

Scoring Framework

Total Score Calculation: Each category score (1-10) × category weight = weighted score Sum all weighted scores for final rating

Interpretation Guide:

9.0-10.0: Exemplary - Actively advances social justice
7.0-8.9: Good - Generally aligned with progressive values
5.0-6.9: Acceptable - Usable with some concerns
3.0-4.9: Problematic - Significant ethical issues
1.0-2.9: Avoid - Actively harmful to movement goals

Special Considerations:

Tools scoring below 5.0 in Movement Security should be avoided for sensitive organizing work
Tools scoring below 4.0 in any category require explicit risk assessment
Consider cumulative impact when multiple tools are used together

Implementation Notes

Assessment Process:

Research tool using public information, user reports, and company policies
Score each category based on available evidence
Weight scores according to your organization's priorities
Consider tool alternatives and context-specific needs
Reassess periodically as tools and companies evolve

Customization: Organizations can adjust category weights based on their specific work:

Increase Privacy weight for surveillance-heavy environments
Increase Labor Impact weight for workforce-focused organizations
Increase Accessibility weight for community-serving organizations

Transparency: All Liberation Labs tool assessments will include source citations and reasoning for scores to enable community validation and improvement.