Recommendations – AI in healthcare
Guided by our panel of experts, we have developed 22 recommendations grouped within eight themes. The themes are summarised here and are not listed in any particular order of importance. The recommendations highlight where some of the work could be carried out and specific considerations that might be of interest for decision makers and policy writers.
Theme 1: Mapping the landscape in Aotearoa New Zealand
There are many aspects of the healthcare landscape that will evolve with the ongoing deployment of AI in healthcare delivery. Examples include back-office efficiency, image analysis, research, and technology development. It is important to maintain an awareness of the needs and opportunities within our national context.
a) Canvas national healthcare settings to ensure that the various needs (i.e., staff, individual patient, whānau, and community) are understood. This could:
i) Highlight local, regional, and national needs to identify and prioritise the appropriate deployment of AI healthcare interventions
ii) Be utilised to inform research and development efforts
b) Ensure ongoing horizon scanning to maintain an awareness of emerging technologies in AI and healthcare and the extent to which needs in clinical settings might be addressed
i) Support the identification of areas for future deployment
ii) Enable New Zealand to lead developments in areas of particular priority to our national healthcare needs
- R1:a could be undertaken jointly by agencies such as:
- Manatū Hauora | The Ministry of Health
- Te Whatu Ora | Health New Zealand
- Te Aka Whai Ora | Māori Health Authority
- The regulatory body established for oversight of the Therapeutic Products Act 2023 (TPA)
i) Highlight enablers and barriers for the deployment of AI in healthcare settings (both public and private)
ii) Identify policy/legislation for review
b) Develop an understanding of various capabilities of AI technologies and develop a robust framework to support appropriate regulation. This could:
i) Distinguish AI technologies according to type and output (for example, operational efficiency improvements compared to self-learning AI and diagnostic support) and establish the extent to which regulations are required for distinct applications
ii) Ensure independent testing requirements for the evaluation of impact and safety
d) Ensuring ongoing monitoring of relevant safety signals, performance, and quality of AI-enabled technologies
e) Continuous horizon scanning to maintain awareness of AI-enabling technologies (e.g., quantum computing, VR, etc) to inform regulatory settings
- R2:b could be led by Manatū Hauora with support from other relevant agencies such as Te Whatu Ora and Te Aka Whai Ora
- For R2:b, where AI applications are already well understood and evaluation mechanisms well established, regulation should promote best practice(s). Where there is not yet a well established best practice for evaluation of particular AI tools, regulation should limit adoption until such a time that evaluation best practice is well established
- R2 should take into consideration principles 5, 10 and 15
- There should be ongoing monitoring of rules and regulations established to support the TPA and the implications for AI in healthcare
c) Evaluate public and private capabilities to determine:
i) Potential opportunities to collaborate across public and private settings
ii) The extent to which capabilities should be enhanced to close potential gaps in healthcare needs specific to New Zealand
iii) The size of the technical workforce to conduct evaluation and authorisation of new AI-enabled technologies
- R3:c (iii) The Therapeutics Products Regulator could require appropriately trained staff to effectively evaluate and regulate relevant technologies
i) Provide clarity around research and development outputs from New Zealand that have the potential to be implemented in our healthcare industry
ii) Provide short-to-medium term clarity around future research needs for New Zealand and our research partners
iii) Provide clarity on tertiary AI courses available across institutions
iv) Support the establishment of aspirational mid-to-long term goals for healthcare delivery in New Zealand and related research and development
b) Undertake regular horizon scanning to establish an understanding of future potential areas for research & development
c) Understand enablers and barriers experienced by technology developers in the AI healthcare sector. This should:
i) Be used to inform the ongoing development of suitable legislative settings
ii) Inform discussion around support tools/services that might help to reduce complexities
e) Evaluate research findings and establish future AI research needs
f) Evaluate computing capabilities and appropriateness for future demands
- R4:a could be undertaken by various agencies/institutions including but not limited to:
- Manatū Hauora
- Te Whatu Ora
- Te Aka Whai Ora
- Hīkina Whakatutuki | Ministry of Business, Innovation & Employment (MBIE)
- Universities
- Research institutions/organisations
- Mapping of national capabilities could highlight areas where Aotearoa New Zealand might have a competitive advantage in AI healthcare. This might look like a database that is regularly updated with details of AI and healthcare related research in New Zealand and could be undertaken by an agency such as MBIE
- Mapping of national capabilities should be undertaken alongside R1 to ensure we are developing expertise that is guided by our healthcare needs
- R4:a,b and d should consider resourcing and leadership capabilities for research and development of AI for healthcare delivery.
- R4:b should be undertaken in conjunction with R5:b
c) Evaluate public and private capabilities to determine:
i) Potential opportunities to collaborate across public and private settings
ii) The extent to which capabilities should be enhanced to close potential gaps in healthcare needs specific to New Zealand
iii) The size of the technical workforce to conduct evaluation and authorisation of new AI-enabled technologies
- R3:c (iii) The Therapeutics Products Regulator could require appropriately trained staff to effectively evaluate and regulate relevant technologies
Theme 2: Maintaining the human element of care
While there are clear opportunities for improvements in efficiency and data processing, the extent to which AI systems might augment our current healthcare service delivery is unclear. Establishing an understanding of the crucial human elements of healthcare delivery will support decision makers to deploy AI technologies in the appropriate supporting areas.
i) Canvas a diverse range of voices within the community
ii) Inform governance bodies and decision makers of the healthcare desires and levels of comfort within their respective communities distinguished by application. For example, patients may be fine with an AI scheduling system but might prefer to know if AI has been used in image diagnosis
iii) Identify the factors that contribute to comfort levels
iv) Identify at what stage of receiving healthcare that patients desire to know that AI has been used
b) Understand experiences of AI technology developers around the development and deployment of AI for healthcare in New Zealand. This work should:
i) Canvas a diverse range of technology applications
ii) Inform governance bodies and decision makers of developers experiences and the extent to which New Zealand is a desirable market to partner with
c) Understand the ongoing interactions between clinicians and AI and healthcare delivery
i) Capture any changing attitudes among the public as trust in AI technology is built
ii) Identify factors that contribute to changing attitudes
iii) Inform decision makers of levels of comfort within communities and likely future needs
- R6 should take into consideration principles 5,12,14,15,17
- Those in governance and decision-making roles should maintain awareness of developments in AI to ensure decisions are informed by the most relevant and up-to-date information
i) Canvas a diverse range of voices within the community
ii) Inform governance bodies and decision makers of the healthcare desires and levels of comfort within their respective communities distinguished by application. For example, patients may be fine with an AI scheduling system but might prefer to know if AI has been used in image diagnosis
iii) Identify the factors that contribute to comfort levels
iv) Identify at what stage of receiving healthcare that patients desire to know that AI has been used
b) Understand experiences of AI technology developers around the development and deployment of AI for healthcare in New Zealand. This work should:
i) Canvas a diverse range of technology applications
ii) Inform governance bodies and decision makers of developers experiences and the extent to which New Zealand is a desirable market to partner with
c) Understand the ongoing interactions between clinicians and AI and healthcare delivery
i) Capture any changing attitudes among the public as trust in AI technology is built
ii) Identify factors that contribute to changing attitudes
iii) Inform decision makers of levels of comfort within communities and likely future needs
- R6 should take into consideration principles 5,12,14,15,17
- Those in governance and decision-making roles should maintain awareness of developments in AI to ensure decisions are informed by the most relevant and up-to-date information
Theme 3: Enabling adoption
Adopting AI into our healthcare system will not happen on its own. The appropriate policy settings, targeted information provisions, and resourcing to enable effective adoption of AI technology that will support improved health outcomes for Aotearoa New Zealand will be key to seeing effective outcomes.
b) Ensure that healthcare workforce are adequately informed to understand newly adopted guiding principles for AI in healthcare settings
c) Identify resources required for implementation of best AI practice across the health system
d) Establish and/or adopt formal evaluation processes for pre- and post-implementation of new AI health technology. Evaluation processes should:
i) Take into consideration best evaluation practice for the technology in question (if best practice has been established)
ii) Take into consideration system resourcing and the extent to which AI technologies are compatible with existing resources (for example if AI tools are more efficient at screening for breast cancers, is the system adequately resourced to cope with increased detection)
iii) Where best practice for evaluation has not been established, the technology should be limited in its application with sufficient mechanisms to prevent use on an experimental basis outside of authorised clinical settings
iv) Evaluation results can be communicated to the public (R10) to help facilitate public trust
e) Ensure regular review (annually or as needed) of principles and practices for application of AI in healthcare settings
f) Establish clear frameworks for liability and responsibility of AI when deployed in the healthcare system. This should:
i) Distinguish by application/output
ii) Distinguish by level of supervision
iii) Distinguish by level of associated risk
iv) Establish clear criteria for insurance coverage
h) Technologies with post implementation evaluations that demonstrate clear efficiency improvements should be adopted more broadly as standard practice
i) Automation should become default practice unless there is compelling reason not to
ii) Evaluation for widespread adoption and standard practice should establish the extent to which successful technologies are implemented across different settings as part of standard practice
i) Support policy makers to stay abreast of international best practice (Food & Drug Administration (FDA), TGA or EU)
- The establishment of guiding principles and practices for the adoption of AI will also be key to establishing confidence and trust in the healthcare system. As such R8 should be factored into the communications strategy
outlined in R10 - R8:e could be undertaken by various agencies including but not limited to:
- Te Whatu Ora
- The TPA regulatory body
- R8:f(iv) could be informed by evaluation outcomes from R8:d
- R8:f will need to be informed by legal framework for enforceable product standards and responsibilities to be established by the TPA regulatory body
a) Complete a gap analysis of research and development capabilities within New Zealand. This could inform the development of funding models that require and/or reward developments for supporting positive healthcare
outcomes in New Zealand (considered in conjunction with the outcomes of R3:a)
b) Consider establishing a suitable funding model to facilitate the deployment of AI healthcare research
c) Measure the proportion of locallyproduced AI developments that are deployed in domestic healthcare settings compared with those that are exclusively seeking international markets. This should:
i) Be used to maintain an understanding of AI capabilities being developed locally
ii) Inform research funding policies that incentivise or require benefit to be delivered to the New Zealand
healthcare system
- R9:a could be carried out by various agencies or institutions including, but not limited to:
- Manatū Hauora
- Te Whatu Ora
- Te Aka Whai Ora
- MBIE
- Universities
- R9:a could be informed partly by R4
- R9:a should be considered in conjunction with outcomes from R3:a
- R9:a-c might necessitate the establishment of a research and development leadership body for AI in healthcare
- R9 could inform R20
Theme 4: Establishing confidence and trust
Establishing a sense of confidence and trust in AI technology is important. Effective engagement with the public, various tiers of the healthcare workforce and those in research and development will help to build confidence. Clear communication of AI limitations, risks and associated evaluation outcomes, coupled with the appropriate frameworks for governance, will support AI deployment and grow confidence and trust in AI-enabled technologies across the healthcare system.
i) Present and future potential for improved healthcare outcomes
ii) Clear communication around benefits and limitations of AI
iii) Associated risks of members of the public using AI as an alternative and/or replacement to consulting with a healthcare professional
iv) Inevitability of errors (including types of errors, rate of errors, and comparison of error rates in settings where AI is not in use)
v) National and international use
cases
b) Ensure that targeted information and training is available to AI in healthcare governance and decision-making bodies at all levels
c) Ensure transparency around evaluation and implementation processes/frameworks to provide confidence in decision-making processes
- R10:a should be consistent with principles 12 and 14
- R10:c should engage with the relevant agencies to ensure activities are compliant with relevant regulations such as the TPA. Communication mechanisms could look like:
- Public forums
- Social media content
- Accessible material in healthcare settings
- Accessible material on healthcare websites
- R10:d could be carried out by a relevant
agency and/or independent research group
b) Understand future resourcing and capability requirements and establish pathways to build relevant skill sets
c) Monitor AI companies that indicate potential capability for AI technology to provide training of healthcare staff and/or health students
d) Consult with training providers (including universities, accreditation bodies etc.) to develop evaluation mechanisms and criteria where adoption of AI tools for training of clinical staff and/or students would be acceptable and appropriate
e) Develop an understanding of future AI training needs for health students and healthcare practitioners
g) Monitor evolving AI and healthcare landscapes to determine further areas for deployment of AI training capabilities
- R11:a should ensure resourcing pathways established are consistent with, and complimentary to, the Manatū Hauora Health Workforce Strategic Framework and the Te Whatu Ora/Te Aka Whai Ora Health workforce plan 2023/24
- R11:d-f should be mindful of supervision requirements outlined in principle 10
b) Ensure access to technical resource for government agencies responsible for ensuring data privacy
c) Determine appropriate frameworks for establishing dynamic informed consent
- R12:b might include agencies/official bodies such as:
- Te Mana Mātāpono Matatapu | Office of the Privacy Commissioner
- The Government Chief Privacy Officer
Theme 5: Tackling inequity
The adoption of AI in healthcare should not just replicate our current health outcomes. AI technology deployed in our healthcare settings should facilitate better outcomes for everyone in Aotearoa New Zealand. This necessitates developing an understanding of where our greatest health needs are and ensuring that we deploy technologies that help to close equity gaps.
i) The tool’s effectiveness across various population group
ii) The burden of disease the tool is designed to address across different population groups
b) Require an equity impact and bias assessment before launching any AI tool in the public healthcare system
c) Develop a framework for ongoing systematic evaluation of AI tools to understand the impact on health inequity (including annual reporting)and bias. This should:
i) Be flexible to assess various typesof AI
ii) Inform decision-making bodies, funding bodies, research institutions and the technology development sector
d) Develop frameworks and/or principles for AI development that highlight the need to address inequity and bias in healthcare delivery from the starting point of the development process
- R13: a(i) might be monitored by Te Whatu Ora and Te Aka Whai ora, and overseen by Manatū Hauora at a system-level
- R13:b and c should be consistent with principle 7
- Ensure that AI tools support the provision of healthcare in a way that is no more biased than human decision makers (consistent with principle 15)
- Quantitative metrics of inequity should be considered when establishing the appropriate communications strategy (R10). Effective communication of metrics could help to generate an informed public discussion
- Ensure that evaluation outcomes from R13 are captured and communicated back to stakeholders through the appropriate channels. This should ensure ongoing transparency and work to maintain confidence and trust
Theme 6: Te ao Māori
Unique to the Aotearoa New Zealand context is Te Tiriti o Waitangi. Relevant iwi, hapū, whānau, and Māori organisations should be included in decisionmaking processes as partners alongside the Crown. Partnership should be evident throughout all stages of project life-cycles spanning conception, planning, governance, design, and implementation.
b) Develop a strategy to build Māori capacity including investment into workforce training, data access, data-sharing with appropriate Māori health providers, etc
d) Evaluate Māori workforce capability against healthcare needs
- Oversight for this could be supported by various agencies and groups including:
- Manatū Hauora
- Te Whatu Ora
- Te Aka Whai Ora
- Relevant Māori authorities
b) Establish engagement forums that enable robust discussions around practical applications of the principles
of Māori data sovereignty. Discussions might include:
i) Empowering relevant iwi, hapū, whānau and Māori organisations to determine metrics of health, wellbeing and hauora for their own communities
ii) Ensuring Māori control over Māori data and considerations of potential outcomes
iii) Establishing appropriate tikanga for collecting, classifying, storing, accessing and using Māori data
iv) Appropriate mechanisms of co-design as partners to Te Tiriti
- Effective partnership with Māori, whānau, hapū, iwi, and Māori organisations presents an opportunity for Aotearoa New Zealand to lead globally in addressing Indigenous AI health-related issues
- R15:b should enable discussions amongst Māori leaders, and between Māori leaders and the appropriate government agencies
b) Develop a strategy to build Māori workforce capacity including investment into workforce training, data access, data-sharing with appropriate Māori health providers, etc
- R16 could be supported by various agencies and institutions including but not limited to:
- MBIE
- Universities
- Relevant Māori authorities
Theme 7: Data and systems
We cannot talk about AI without also talking about data and inference. Implementation of AI technologies within our healthcare system requires inference from large data sets. This highlights issues such as data definition, data collection, data storage, data privacy, data sovereignty and security as well as the safety, reliability, and effectiveness of the inference these data enable.
b) Identify computing requirements to enable on-shore data storage, model hosting, and technology development
c) Expand the healthcare data strategy to consider factors relevant to data collection and data use for AI. This could include:
i) The potential for individuals to opt in or opt out
ii) The mechanisms for consent and the impact of individual consent on people groups (e.g., whānau, communities)
d) Ensure robust data collection mechanisms and understand implications of AI tools being used for populations that are underrepresented in current data sets
e) Explore mechanisms for data linking across data sets outside healthcare, being mindful of data sovereignty
- New Zealand has some unique data sets and ability to link national data sets through the Integrated Data Infrastructure. This presents an opportunity for New Zealand with a competitive advantage for AI in healthcare delivery
i) Consider principles of Māori data sovereignty (see R15)
ii) Include guidelines for testing of AI tools using national data sets
- R18 could be supported by relevant agencies and bodies including but not limited to:
- Manatū Hauora
- Te Whatu Ora
- R18 could be supported through engagement with the soon to be established TPA regulator
Theme 8: Exploring future opportunities
AI introduces various opportunities to improve outcomes in our healthcare system. Creating environments that foster research and innovation can enable us to take advantage of new and exciting opportunities.
- R19 should support needs outlined in R1, R4 and R5
b) Establish international research and development capabilities and develop strategic relationships
c) Specific research strategy should be defined based on (1) need within the healthcare system, (2) capacity and capability within domestic research capabilities (or in existing research partnerships), (3) likely impact of research outcomes (4) likely time to deployment and (5) ease of deployment/implementation
- Centre of Research Excellence should be developed in partnership with the health system to ensure guardianship of health data that can be used for research and development, and to ensure research addresses relevant health system needs
- R20:c could be informed by MBIE’s Te Ara Paerangi | Future Pathways initiatives
- R20 could be supported by agencies like Te Amorangi Mātauranga Matua | Tertiary Education Commission
b) Understand from existing AI companies the various enabling technologies that facilitate enhanced AI development
c) Generate targeted information that provides advice to start-up companies attempting to deploy AI healthcare technology in New Zealand
d) Generate advice for AI companies to navigate the legislative environment
e) Generate advice for AI companies to
navigate commercialisation processes
- Mechanisms for connecting with AI companies might be supported by groups such as the AI Forum of New Zealand
i) Span various stakeholder groups (e.g., occupation, iwi, ethnicity, locality, research, industry, government etc)
ii) Highlight factors that are at the forefront of the public conversation, immediate concerns to be addressed and clear opportunities to capitalise on
b) Establish annual expo (or something similar). An expo should:
i) Allow those from the research and development sector to showcase current and future potential
ii) Be used to inform the healthcare profession of available emerging AI technologies
iii) Enhance public visibility of emerging technologies
c) Establish support roles and/or networks for AI businesses. Support should:
i) Provide advice to businesses about deployment of technology in the New Zealand healthcare environment
ii) Provide mechanisms to support SMEs with regulatory costs
d) Establish links with key players in the global AI ecosystem e.g., Microsoft, Amazon, etc
- R22 could be supported by various agencies and institutions including, but not limited to:
- MBIE
- Manatū Hauora
- Te Whatu Ora
- Te Aka Whai Ora
- Universities
- Te Apārangi | Royal Society of New Zealand
- Relevant Māori authorities
Last edited on: 15th December 2023