The Human-Centred AI Assessment Framework: A Reusable Method
This assessment applies the Human-Centred AI Assessment Framework, a weighted seven-dimension rubric (Human Agency and Oversight, Technical Robustness and Safety, Privacy and Data Governance, Transparency and Explainability, Diversity/Non-discrimination and Fairness, Societal and Environmental Wellbeing, and Accountability) for evaluating whether any AI-integrated product is trustworthy and human-centred. Read the full framework there for the scoring methodology and worked examples.
A note on the scores below: this assessment predates that formalised weighted rubric and scored seven related but not identical dimensions on a 1-10 scale, substituting Inclusive and Accessible Design for Accountability. The scores stand as a historical record of that assessment. Re-scoring Miro against the current seven-dimension, weighted rubric is real evaluation work, not a text edit, and hasn't been done yet.
Applying the Framework: Miro's Innovation Workspaces (October 2024)
Miro Logo
G’day
Earlier this week I caught up on Miro’s latest AI announcement – Innovation Workspaces.
In the ever-evolving landscape of digital collaboration, Miro has just upped the ante. Their latest release integrates AI into almost every part of their platform.
It promises to change how we brainstorm, plan, and create together. It sounds exciting, but demos can look great and fall short in practice.
So I was curious to dig deep and find out how human-centred and trustworthy Miro’s AI is. Or, does it risk overshadowing human creativity and true collaboration?
Let’s dive in and find out.
Miro’s new AI suite
Content Creation and Organisation
Create with AI
What it does: Generates content directly on Miro boards based on text prompts.
Real world scenario: A marketing team types, “Create a content calendar for Q3 product launch.” They receive a pre-populated calendar with suggested content types and posting frequencies.
Auto-structuring
What it does: Converts freeform thoughts into organised workflows, diagrams, or mind maps.
Real world scenario: Your team brainstorms improvements to customer experience can be transformed into a visual journey map. AI swiftly arranges scattered thoughts, pinpointing key interactions and pain points along the customer’s journey.
Intelligent Diagramming
What it does: Assists in creating and optimising various types of diagrams.
Real-world scenario: While mapping out a complex software architecture, the AI suggests additional components and optimises the layout for clarity.
Generate diagrams with simple text prompt
These features simplify the messy process of coming up with and planning ideas. They let teams focus on ideas while AI manages organisation. However, a question remains: Will this ease come at the cost of valuable “eureka moments” found in the organising phase?
AI-Powered Assistance and Insights
AI Sidekicks
What it does: Provides contextual help and guidance while working on the board.
Real-world scenario: During a retrospective, the AI Agile Coach sidekick suggests, “It seems your team completed fewer story points this sprint. Consider discussing any unexpected challenges that arose.”
Smart Suggestions
What it does: Analyses user behaviour to offer relevant recommendations for tools and content.
Real-world scenario: While making a project timeline, the AI suggests adding a risk assessment matrix due to the project’s complexity.
Sidekick provides suggestions
Data Visualisation Assistance
What it does: Helps interpret data and suggest appropriate visualisation methods.
Real-world scenario: A sales team uploads quarterly data. The AI suggests bar charts for comparison and line graphs for trends.
These three features act like a digital consultant. They advise, suggest tools, and help teams with their data. It’s like having a team member who’s always up to date on best practices across domains.
Bridging Physical and Digital Collaboration
Content Recognition
What it does: Digitise handwritten notes and sketches on the board using computer vision.
Real-world scenario: A product team takes a photo of their whiteboard after an ideation session. This creates a digital version with text recognition for easy editing and sharing.
AI-powered Prototyping
What it does: Accelerates the creation and iteration of product designs during ideation. This is by no means final designs, rather a way to stay in the same place and quickly visualise an idea under discussion.
Real-world scenario: A UX designer describes a “minimalist login page with social media integration” and receives several design options to refine.
Prototype ideas in the middle of brainstorming
These features blend physical and digital workspaces. They enable teams to switch easily between them and quickly realise ideas. This mix might transform our approach to workspace design and remote collaboration.
Enhancing Navigation and Accessibility
Collaborative Filtering
What it does: Enhances real-time collaboration by prioritising relevant content for each user.
Real-world scenario: In a large product dev board, a UX designer sees UI elements highlighted. A product manager sees feature prioritisation lists emphasised.
Semantic Search
What it does: Enhances the board search function using natural language processing.
Real-world scenario: A team member searches for “customer pain points” and finds relevant sticky notes from brainstorming sessions, even if they don’t contain those exact words.
Workflow Optimisation
AI Shortcuts
What it does: Offers one-click tools integrated into specific workflow templates.
A real-world scenario: After a research planning session with your product team, one click turns scattered ideas into a detailed research plan. This feature saves time by automating tasks. It lets teams quickly move from collaboration to action.
Human-Centred Trustworthy AI Assessment: The Scores (October 2024)
Now, let’s see how Miro’s AI rated against the seven dimensions, as of October 2024.
Transparency and Explainability: 6/10
Users see the AI’s suggestions but not its logic. This creates a black box effect in the tool’s decision-making.
A score of 6/10 reflects the need for enhanced transparency to foster user trust in AI recommendations.
Fairness and Non-discrimination: 7/10
The AI works well with various content types. But, there’s little info on how Miro ensures fairness among diverse user groups.
More transparency about bias testing and mitigation strategies would be welcome. We must safeguard against amplifying biases in our collaborative work and choices. Integrating AI responsibly ensures fairness and equity across team interactions and outcomes.
Human Agency and Oversight: 8/10
Miro’s AI acts more as an assistant than an autonomous decision-maker. Users always have the final say in accepting or rejecting AI suggestions.
However there should be clearer ways for users to feedback on AI.
Privacy and Security: 8/10
Miro has good data protection, but more transparency on AI training data would help. It asks for consent from Enterprise customers and offers opt-out options for others before using their data to enhance AI features. All customers can withdraw consent or object to data usage for AI improvement at any time.
However, I found it very difficult to find out how to turn it off as a non-enterprise user. As AI systems advance, privacy controls our data usage becomes crucial for maintaining user trust.
Reliability and Safety: 7/10
The AI features seem to work well. But, there’s little info on error rates or edge cases.
More transparency in this area would increase user trust. We need to know AI systems’ limits and failure modes or we can’t make informed decisions about trusting the AI recommendations.
Inclusive and Accessible Design: 9/10
Miro’s commitment to WCAG AA compliance is commendable. The AI features, especially in navigation and search, seem to help diverse users.
Given the diversity in companies, its imperative that inclusivity is enhanced. AI tools should expand, not limit, opportunities for all team members.
Societal and Environmental Wellbeing: 7/10
Miro’s AI tools boost productivity and may lessen physical resource needs.
However, more data on the environmental impact of its AI models is needed. As we rely more on AI tools, it’s crucial to consider their societal and environmental effects.
Ethical Implications and Future Considerations
As we embrace AI in collaborative workspaces, we must grapple with several ethical considerations:
- Creativity and Originality: How can we ensure AI helps, not stifles, human creativity? There’s a fine line between helpful suggestions and over-reliance on AI-generated content.
- Skills Development: As AI takes over tasks, how do we ensure that team members keep their skills and can perform without AI when needed?
- Decision-Making Autonomy: With AI giving suggestions, how do we keep human control in decisions, particularly if there isn’t an appropriate amount of transparency in AI systems? It’s crucial to cultivate critical thinking skills alongside AI literacy.
- Digital Divide: As AI-powered tools become more prevalent, how do we ensure equal access and prevent the creation of new forms of workplace inequality?
These ethical concerns should guide AI in collaborative tools. We must ensure it augments, not replaces, human abilities.
Run This Framework on Your Own AI Tools
The seven weighted dimensions above aren't specific to Miro. Score your own team's AI tools, or a vendor you're evaluating, the same way: Human Agency and Oversight, Technical Robustness and Safety, Privacy and Data Governance, Transparency and Explainability, Diversity/Non-discrimination and Fairness, Societal and Environmental Wellbeing, and Accountability. A tool that scores low on Human Agency or Accountability is telling you something worth acting on before you roll it out further.
If you want a structured version of this for your team's overall AI maturity rather than a single tool, the AI Maturity Diagnostic applies the same evaluative discipline across your whole AI adoption, not just one product.
Conclusion: Embracing AI with Critical Enthusiasm
Miro’s new AI features represent a significant leap forward in digital collaboration tools. They address many pain points in remote and hybrid work environments, from capturing ideas to organising complex projects. The AI acts more as an enhancer of human creativity than a replacement, which aligns well with human-centred AI principles.
There’s a need for better transparency in AI decisions and measures to ensure fairness and avoid bias. As these tools become common in our work, Miro must continue to focus on human-centred design and ethical AI.
For users, these features offer exciting possibilities to enhance productivity and creativity. It’s key to critically assess these tools. Be aware of their impact on our work and team interactions. As we blend AI with human effort, your experiences and feedback are vital.
What do you think is the best mix of AI help and human creativity in teamwork? How can we use AI without losing our unique touch in innovation and collaboration?
Let’s keep talking about AI’s future in our work.
Feedback is super welcome!