I have been thinking about AI a lot lately (I suspect most of us have). In addition to the general presence of AI in our lives, two recent events, neither of which occurred in Medtech, prompted this.
The former is the letter the FDA issued last month to a pharma manufacturer (you can read the full text here, especially the part on Inappropriate Use of Artificial Intelligence). I will give you the highlight: just because AI tells you / does not tell you to do something or have something specific in your documents, it does not mean AI is responsible for the end result. Saying “AI made me do it” does not fly with regulators. The latter event comes from outside industry, but still close by: Elsevier (the major publishing company), joined a class-action lawsuit against Meta for reproducing copyrighted materials when training its AI.
The way I see it, there are several issues worth discussing around the use of AI in our line of work, but I will focus on three in this letter: responsibility, transparency and some variation of copyright / use of proprietary materials. I think there will be a part 2 letter, though. Bear with me, this is not as simple as those selling all-in-one AI solutions may want us to think it is.
Over the past couple of years, I have followed the attitude of fellow medical writers towards using AI tools. Some were quick to jump on the AI wagon particularly for literature reviews and found that, in the early days, the results were largely subpar (and fixing the results proved more time consuming than doing the work themselves in the first place). Their enthusiasm turned into skepticism, which got diluted somewhat over time, as AI tools for literature reviews got better. I have played around with a few as well and find some value in them but only for limited tasks. However, I have also had to do massive remediation work on clinical evaluation documents where unchecked AI outputs raised many questions from Notified Bodies. The problem here is not that some AI tool produced a poor piece of text or analysis, but rather that this was not caught in time… and that several people signed those documents. In the end, the legal responsibility for the device officially lies with the manufacturer, but does that really mean we should be irresponsible?
And while writers and regulatory consultants do talk to each other about which AI tools we use and for what, there is little to no transparency beyond these conversations. Within a clinical evaluation, we have to document and justify all literature and vigilance databases we use and how we use them, plus we have to provide evidence in support of our qualifications for the work we are doing. Clinical Investigation Plans and Reports list all tools used for data analysis, statistical calculations and the like. But in neither of these situations does anyone have to disclose the use of AI tools, which ones and to what extent. I wonder what you make of this. Do you see any value in making disclosure of AI use in documents that go to regulatory authorities mandatory? On this front, serious peer-reviewed scientific journals are years ahead, with clear policies on where AI use is acceptable and what must be disclosed (here is just one of many examples).
As for the problem of proprietary materials, beyond the grey area of copyrighted scientific literature used to train various AI tools, I will ask an intentionally provoking question. Isn’t feeding manufacturer-generated documents (Risk analysis, PSURs, PMCF plans etc.) into an AI tool some form of breach of NDAs (since we cannot say for sure what the AI does with that info, how and if it may end up anywhere other than in the output it provides us with)? And, before you ask, I have had clients explicitly request that no AI tools be used in any of the work I did for them.
I would love to know what you make of all of this. Do we need more transparency with regards to how AI is used in clinical and regulatory writing? Should writers have to disclose the tools they use and be made aware they and not “the machine” are accountable for the output? Should we limit the use of AI to those parts of clinical documents that are based exclusively on publicly available information?