Tools & Frameworks¶

Purpose: Curated directory of practical AI tools, frameworks and resources for safe and responsible implementation Audience: Technical teams, governance professionals and implementation leads | Time: 20-30 minutes

A curated list of practical tools, frameworks and resources to help Australian businesses implement AI safely and responsibly.

🎯 AI Risk & Ethics Frameworks¶

• Australian Government AI Ethics Principles – 8 principles guiding ethical AI use. (industry.gov.au)
• Voluntary AI Safety Standard (10 Guardrails) – published 2024, aligns with ISO/IEC 42001 and NIST AI RMF. (industry.gov.au)
  • Note: The Guardrails explicitly align with ISO/IEC 42001:2023 and the NIST AI Risk Management Framework 1.0.
• National framework for the assurance of AI in government (DTA) – how agencies assure AI systems. (dta.gov.au)
• NIST AI Risk Management Framework (AI RMF 1.0) – comprehensive, sector-agnostic guidance. (nist.gov)
• NIST Generative AI Risk Management Profile – profile for GenAI use cases. (nist.gov)
• ISO/IEC 23894 – AI risk management guidance. (iso.org)
• ISO/IEC 42001 – AI management system (AIMS) requirements. (iso.org)
• OECD AI Principles – intergovernmental principles for trustworthy AI. (oecd.ai)
• Singapore Model AI Governance Framework – practical implementation guidance. (pdpc.gov.sg)

📋 Governance & Policy Tools¶

• Privacy Impact Assessments (PIAs) – OAIC guidance on conducting PIAs. (oaic.gov.au)
• NSW Artificial Intelligence Assessment Framework – Structured risk-based assessment framework for AI systems; updated to address generative AI. (digital.nsw.gov.au)
• ASD Essential Eight – baseline mitigation strategies. (cyber.gov.au)
• Notifiable Data Breaches (NDB) Scheme – reporting obligations. (oaic.gov.au)
• Australian Privacy Principles (APPs) – core privacy obligations. (oaic.gov.au)

🔬 Technical Testing & Monitoring¶

• Model Cards – documentation standard for AI models. (arXiv)
• Datasheets for Datasets – dataset transparency and quality control. (arXiv)
• Aequitas – open-source bias/fairness audit toolkit. (github.com)
• Fairlearn – open-source fairness assessment and mitigation. (fairlearn.org)
• NIST AI RMF Playbook (TEVV) – testing, evaluation, verification and validation resources. (airc.nist.gov)

🔒 Privacy & Security¶

• Differential Privacy – OpenDP, OpenMined community resources. (opendp.org, OpenMined)
• Homomorphic Encryption – OpenFHE library and docs. (openfhe.org)
• Confidential Computing – architecture patterns and use cases (Consortium). (confidentialcomputing.io)
• Data anonymisation – ARX, Amnesia, sdcMicro (open-source). (arx.deidentifier.org, amnesia.openaire.eu, github.com/sdcTools/sdcMicro)

💡 Explainability & Transparency¶

• LIME – local interpretable model-agnostic explanations. (github.com)
• SHAP – Shapley value–based feature importance. (github.com)
• DALEX – model exploration and explanations. (github.com)

📊 Continuous Monitoring & Ops¶

• MLflow – experiment tracking and model registry. (mlflow.org)
• Prometheus – metrics collection for model/service health. (prometheus.io)
• Kubeflow – open-source MLOps on Kubernetes. (kubeflow.org)

🛡️ LLM Application Safety & Secure Development¶

• OWASP Top 10 for LLM Applications – common risks and mitigations. (owasp.org)
• OWASP AI Security & Privacy Guide – secure AI development guidance. (owasp.org)
• MITRE ATLAS – adversary tactics/techniques/mitigations for ML systems. (atlas.mitre.org)
• Guidelines for Secure AI System Development – joint guidance (UK NCSC, CISA and partners). (ncsc.gov.uk)

🔍 RAG Evaluation & QA (Open-source)¶

• ragas – evaluation for retrieval-augmented generation. (github.com)

🚀 Serving & Data Infrastructure (Open-source)¶

• KServe – model serving on Kubernetes. (kserve.github.io)
• Feast – feature store. (docs.feast.dev)

✅ Procurement & Vendor Risk (Checklist)¶

Further Reading¶

• Voluntary AI Safety Standard – 10 Guardrails
• Australian AI Ethics Principles
• OAIC – AI and Privacy guidance
• NIST AI Risk Management Framework
• NIST SP 1270 – Identifying and Managing Bias in AI
• OECD AI Principles
• OWASP Top 10 for LLM Applications
• ASD Essential Eight