Becky Baggett, Senior Project Manager recently completed enrollment six weeks early on a phase 3 study using ePRO with a tablet.

Over the past 18 months, we’ve been working on a study that utilizes electronic patient reported outcomes (ePRO) on tablet devices for a large phase 3 pain study.  We’ll share a number of considerations and recommendations for implementing ePRO on a smart device for a clinical study based on the challenges we’ve faced and the lessons we’ve learned.  For our study, the devices were kept at the sites and subjects only entered data during study visits.  For clinical trials using ePRO for diary data (where subjects take the devices home and enter data on their own) expect for things to get exponentially messier.

Validation

Depending on your phase of development and the type of data you are collecting, validation may be required. The first question you should ask is “what stage of development are you in?” If you are using ePRO for a pivotal trial or the data is likely to be used in a marketing application submission like an NDA, it is highly likely validation is necessary. On the other hand, if you are collecting data through ePRO for an early phase exploratory study, it may not be required. The second question you should ask is “will ePRO be used to record my primary endpoint?” If the answer is yes, it is likely that validation is necessary. If not, it may not be necessary depending on what data will be collected and how it will be used.

Our study was a pivotal study and we were using an ePRO system to collect our primary endpoint. One of the most important pieces of data we collected was pain as measured on a visual analog scale (VAS). In order to validate the scale, a group of individuals was selected that met the inclusion/exclusion criteria for the trial. The group then recorded results on the tablet and on a paper version for each questionnaire and assessment, and each participant was interviewed. We then compared the results from the tablets with the results from the paper and information collected in the participant interview to show that the questionnaires and assessments on the tablet were valid.

Keep in mind that validation costs are not typically included in bids from ePRO vendors. A third party vendor is often required, as was the case for our study, and the costs can be significant (>$50K).

Study Start-up and Site Issues

We had a number of issues with both the tablets and the ePRO software during study start-up. You may not know until site qualification visits if sites have Wi-Fi that is both readily available and reliable. Additionally, the firewalls at some sites created issues. We had several sites where we had to provide Wi-Fi hot spots in order to make the situation workable. Addressing these issues early in the SQV will get the problem solving conversation started. The hot spots can get expensive, so this is a good point to ask about during feasibility surveys so that you can account for it in your upfront budgeting. Make sure that sites have a place to store and charge the devices. It is important to keep devices plugged in to ensure sufficient battery life when they are needed and to prevent loss of data if the battery runs out prior to sending data to the server.

Training is critical for the study team, site staff, and investigators. For the study team, we recommend purchasing an additional device and allowing study team members to go through a hands-on mock set-up. This will aid them when it comes to site training and training during the investigator meeting. Make sure everyone is comfortable with using the tablet, accessories that come with it like a stylus, and the software. It is important that both site staff and investigators understand the rules when it comes to use of usernames and passwords. We had an issue with a site where it came to our attention that site staff were sharing passwords. For audit trail and compliance purposes, it is important that each team member has and uses their own password.

You will need a paper back-up system in case something goes wrong with the system or device. Keep in mind that your paper system needs to be identical to your ePRO system. In our case, that meant having paper copies of each assessment available in the site operations manual. If other language options are available on the tablet, paper backups need to be available in each of those languages as well. You will also need a method to enter data into the ePRO system. In our case this wasn’t straightforward, so it is worthwhile to work out that process in advance. While all of our sites were amenable to using the tablets and many had used them before, they were quick to go to the paper back-up if something went wrong with the tablet. Be clear during training under what conditions sites should utilize the paper back-ups.

Is ePRO a good choice for your study?

In our case, despite all of the challenges we still believe it was.  The quality of the data is often better, there are economies of scale, and the duration and scope of the study warranted it.  However, if you have a small study or a small number of assessments, the time and cost of implementing ePRO may not be worth it.  It is likely that in those instances you can implement a paper-based system faster and cheaper.