Fusing Specifications and Design for Data Collection Casebooks with Veeva Vault EDC
Writing and reviewing stand-alone study specifications is time consuming and can add weeks or months to the study timeline. Much of the effort and delay results from the diverse stakeholders. A ten-page requirements document that is appropriate for the sponsor may translate into a 100+ page technical specification from the CRO or EDC vendor. And the detailed specification documents needed for clinical programmers are laborious and challenging for sponsor teams to review. This inherently inefficient process represents a prime opportunity for technology disruption to deliver significant time savings. Data Management teams can contribute to shortening timelines by adopting a modern EDC and reengineering the study build process.
Agile Design is a new approach to building clinical data collection casebooks. Advances in modern EDC systems enable you to apply many of the process improvements established with agile software development to clinical data. Agile Design addresses challenges throughout the traditional build process, from writing specifications, to casebook designs and UAT, re-use of standards, and post-production changes. The technology and process improvements within Agile Design make it easy to build complex studies, iterate quickly, and deploy updates, which helps your study team become nimble and more successful.
The first principle of Agile Design is fusing casebook specifications and design into a single step. Historically, these were separate processes, using separate tools, by separate people. With Vault EDC, people who have historically defined specifications from a protocol, such as a data manager, now do so directly within Vault CDMS. As they configure a data collection casebook, they are effectively writing the spec. The spec and the casebook are created in a single step. And that spec can be distributed in multiple formats for the study team to review: as a Microsoft Excel™ document, as annotated PDFs, or as the functioning casebook itself.
Technology advances within Vault EDC enable organizations to configure study casebooks, rules, and edit checks without the overhead and delay associated with authoring and reviewing the study specs in Microsoft Word™ and Excel™. Eliminating the need for standalone specs is part of an Agile Design methodology that reduces study build times and effort by over 50%. This paper describes how creating the specifications and design in one step, within the EDC, produces greater quality and speed.
“The protocol is your spec. Everything that needs to be analyzed is in the protocol. Veeva works from the protocol, just like they would from a spec. We’re just eliminating a big time consuming step in the middle.” – Vikas Gulati, executive director of clinical data management and metrics, Vertex
Creating Casebooks Directly from the Protocol
Clinical Programmers as Study Designers
The technical aptitudes of clinical programmers and database developers span a broad spectrum – from those with technical savvy to those with deep software programming expertise.
In most cases, a technically savvy clinical programmer or database developer with knowledge of clinical trials can configure a complete casebook in Vault EDC, including all the rules and edit checks, even for complex oncology studies running adaptive trials.
However, for accuracy and simplicity, we use the title study designer since little-to-no software programming is needed with Vault EDC. The coding of advanced edit checks and custom functions commonly done for traditional EDC systems is rarely needed.
Historically, data managers needed a separate specifications document to physically hand-off to a programmer. Most EDC systems are sufficiently complex that people need extensive training to build Case Report Forms (CRFs) and software engineering skills to write edit checks as custom functions (code that runs outside of the EDC). The phrase “building a database” was an accurate representation of what occurred. And as long as the EDC requires a programmer to build a database and write custom functions, a separate specifications document will be required.
In order to fuse specifications and design, the EDC must provide an intuitive interface that makes it sufficiently easy to configure casebooks and write edit checks that you no longer need software programmers to do the job. Data managers are expert in the data requirements for clinical trials and are the most qualified individuals to design a CRF. EDC technologies should enable these data managers and study designers with clinical expertise to configure studies themselves. See the accompanying sidebar for a more complete description of the study designer role.
The design environment within Vault EDC, named Studio, was conceived as a place for data managers to specify the data collection requirements of their study. Creating a specification in some other system or format adds an additional step. Conceptually, building a casebook in Vault EDC is like writing a spec, only easier because the constructs for data collection are pre-built. There is a drag-and-drop interface for pulling pre-defined data fields into a case report form and defining the edit checks and other parameters.
Creating Rules and Edit Checks
Veeva has a design principle that spans all products which is to automate simple tasks and make the hard tasks easy. Applying that principle to rules and edit checks means automating the creation of simple univariate checks and providing an intuitive user interface to simplify the creation of complex rules.
Automating Simple Checks
Vault EDC uses the properties you’ve entered when defining an item to create system-managed data validation rules beyond the simple univariate range checks. For example, when configuring the event schedule, you simply specify the expected date ranges and offsets of planned visits. During the study, the system automatically checks event schedule dates without anyone programming the validation rules.
“One of the things I like about Vault EDC compared to other EDCs is the reduction in edit checks. For example, for typical study you’ll need 20 checks just to check for future dates. As a CRO building a hundred studies a year, that’s 2000 edit checks. And we don’t just program, we also validate by writing three scripts, one each for a future date, a current date, and a past date, just to test that it’s working. With Veeva, future date checks are just a setting, you don’t have to write any scripts.” – Harbal Sidhu, senior manager of clinical data systems, ICON
Simplifying Complex Checks and Rules
A rules engine in Vault EDC performs the edit checks and executes any branching logic, dynamic form creation, or other tasks specified in your business rules. An intuitive interface for the rules engine makes it easy for data managers or study designers to create sophisticated edit checks and business rules.
Vault can perform a large variety of tasks based on the rules and entered values, such as:
- Adding or removing events, visits, or forms. For example, adding or removing forms based on the route of administration.
- Enabling or disabling certain items on a form. For example, disabling questions about child-bearing potential for patients where sex = male.
- Deriving the value of a related item. For example, calculating a patient’s BMI or returning a value of true if the tumor size in the second visit is less than the first.
Case Study
Veeva re-built a study for one customer using data-driven rules and dynamics to replace the 32 treatment cycles and 128 visits needed in their traditional EDC with a single repeating treatment cycle, and three visits with dynamic forms in Vault EDC.
By pairing a powerful rules engine with dynamic visits, forms, and fields, you can configure complex study designs with relative ease.
Much like Google’s auto-complete for search terms, Vault provides auto-complete suggestions to help you write rules. Using the context of existing identifiers, variables, and operators Vault intelligently suggests appropriate options for your next step. A rule can include multiple conditions and actions; for example, if a comparison of tumor sizes indicates that a tumor has shrunk, automatically add a new treatment cycle with related visits and forms. The rules engine assists users in writing rules, thereby lowering the technical threshold for data managers to become study designers and/or making your current clinical programmers more productive.
Veeva’s goal is to eliminate the need for custom functions and other forms of external code. Every time a study needs a custom function (called triggers within Vault EDC) the product team adds that functional gap as a feature request for a future release. This approach has greatly reduced the number of custom functions required compared to traditional EDC systems.
Capturing Requirements not in the Protocol
Traditional specification documents provide additional requirements and detail that aren’t mentioned in the protocol. These are often standard data requirements for an organization that are used across studies. The best practice within Agile Design is to capture these requirements within the organization’s standards library or a template study. Vault EDC makes it easy to reuse forms from a prior study, leverage Veeva’s existing form templates based on the CDISC standard, or ingest the configuration files from prior studies in Medidata Rave or Oracle InForm. If configuring their own casebooks, sponsors can spec these requirements directly within Vault EDC, just as they would traditionally do within Excel. When working with a CRO, sponsors can create smaller topic-based specs for rules or edit checks that are specific to the therapeutic area or study. Creating these separately, while the casebook is being configured, shortens the overall development cycle.
“The real difficulty in building a study lies in the edit checks – that’s where you require a programming background. But that’s not needed with Vault EDC. If you know how to use Boolean logic and if/then statements, you can write edit checks in Studio. It is really very intuitive and easy to use.” – Harbal Sidhu, senior manager of clinical data systems, ICON
Higher Quality and Fidelity to the Protocol
Multiple factors combine to ensure that studies are reviewed with higher quality and fidelity to the protocol than those built using standalone specs, these include:
- Live, cross-functional design reviews and discussion
- Reviewing the actual casebook rather than a description of the casebook
- Shared access to the casebook and ongoing reviews throughout the updates
- An equally thorough UAT
Reviewing the Database for Accuracy and Completeness
Live, interactive design review meetings with the appropriate stakeholders are critical when fusing specifications and design. In the traditional process, a Word or Excel-based spec is circulated for study team members to review independently and provide feedback. With a consolidated process, participants review the actual EDC screens within a development environment along with annotated PDFs, a system-generated study specifications spreadsheet, and differential reports which indicate what has changed since the last review. Team members come together in a room and/ or via video conferencing and walk through each screen. Questions and concerns are openly discussed so decisions are made based on the collective inputs and requirements of the different teams. Database changes in Vault EDC are made in real-time. There is no need to push the screens into production for users to see updates. Seeing each other’s ideas and suggestions in real-time directly within the EDC significantly increases collaboration and helps teams consider the implications of different options and subsequently make better decisions.
Auto-generating the Study Design Specification
Specifications continue to play an important role in documenting the study for compliance purposes and sponsor sign-offs. In Veeva’s Agile Design process, these specs are auto-generated by the system. Vault EDC is self-documenting and generates a Study Design Specification (SDS) as a Microsoft Excel™ workbook with separate worksheets for key study design components. Every aspect and attribute of the data collection casebook is documented, including the study schedule, form definitions, codelists, rules, and more. The system-generated spec accurately documents each version of the casebook. When postproduction changes are needed, a separate automated report, called the Study Differences Report, documents all differences between the two versions.
“We have experienced data managers who can read through a protocol and build the forms and edit checks in Vault CDMS directly. When done, they can push a button to generate a complete set of specifications, documenting everything you can imagine, and take that to the sponsor for sign-off.” – Tanya du Plessis, vice president of data strategies, Bioforum
Conclusion
The Agile Design methodology uses technology to reinvent the old database build process. Eliminating a standalone spec saves organizations weeks of time spent authoring and reviewing the document. Many of Veeva’s customers provided a traditional spec for their initial studies, but quickly adapted their process to work straight from the protocol once they saw how easy it is to configure studies in Vault EDC. Traditional specs are replaced by defining your casebook within Vault EDC. Manual tasks are greatly reduced—e.g. sponsors provide CROs with only a mini-spec for items outside of the protocol or standards. While automated tasks are greatly increased—e.g. rules and edit checks are productized within the rules engine, and a comprehensive spec is system-generated for documentation purposes. Fusing specifications and design in a single step is one of the ways that Agile Design with Vault EDC reduces the time and effort to produce new data collection casebooks.
“Vault EDC is more agile. It provides an environment where you can deliver a lot faster. You have more capabilities in your forms than people are accustomed to. We’ve had to stop and rethink how we do builds. The team took a step back and considered it as a clean slate. We asked ourselves, how would we like to do builds? It was a great experience and challenge to step outside the box and take a new approach.” – Tanya du Plessis, vice president of data strategies, Bioforum
Fulfilling the Functions of a Spec in Agile Design
| Common Functions of a Spec | Veeva’s Agile Design Approach |
|---|---|
| Translation – Translating clinical requirements defined within the protocol for non-clinical programmers who build the database. | Vault EDC Studio is easy enough to use that experienced data managers and study designers can build casebooks and rules directly from your protocol and standards. |
| Elaboration Supplementing the protocol with additional requirements and detailing how the data should be collected. |
Much of the supplemental information about data capture is best codified in company and industry standards. Creating and reusing standards lessens the need for traditional standalone specs. Sponsors working with CROs can provide mini-specs for studyspecific requirements or edit checks. |
| Confirmation – Providing a description/checklist that ensures everything needed gets built. |
Cross-functional design reviews during the build process ensure that the database fulfills study requirements. These reviews are equally or more effective than the reviews done for a spec, because it is easier to evaluate an actual casebook than a description of what the casebook should look like. |
| Global low cost labor locations – Distributing the build process across multiple people or to offshore resources. | Given the cost and complexity associated with traditional EDC programming, many organizations move work to locations with lower cost resources. With Studio in Vault EDC, few skilled programmers are needed, which reduces the need for lower-cost labor sources. |
| Testing – Provides a description/check-list for reviewers to test against, to know you captured everything. |
Using a single metadata definition for specification, design and execution, eliminates the need to test whether the run-time EDC meets the specification. The focus and effort spent reviewing standalone specs is redirected to ensure the casebook design is correct in the first place. Testing is focused where it matters most, ensuring the EDC casebook meets all study requirements. Sponsors create and provide test scripts based on the protocol and company standards to ensure study design integrity. |
| Compliance – Documenting what was built for compliance purposes and sponsor/CRO sign-offs. |
The system-generated Study Design Specification (SDS) provides thorough documentation of all aspects of the study design and database configuration. The SDS provides a 100% accurate documentation of what was designed and built. It can be reviewed independently and compared to the protocol for official compliance and contracting purposes. |
| Project Management – Tracking what’s been built, what’s outstanding, what’s been tested, and more. | Separate project management documents such as the Configuration Validation Plan, a Validation Status Tracker, and a Traceability Matrix are used to track which elements and rules are complete and which remain to be built. |
| Library – Fostering reuse. Cut and paste edit checks from the specs of prior studies. |
Vault EDC fosters reuse directly within the system. Easily search for previously defined forms and fields and drag them into your new study with or without their rule definitions. Run study validation checks to identify potential issues from copying objects with dependencies, such as crossform multivariate rules. |