R&D’s Dirty Little Secret

Achieving an efficient discovery process

A majority of today’s research laboratories are awash in technology, productivity systems and knowledge management tools. However, to date, these proffered solutions have not manifested a tangible increase in the rate or quality of R&D discovery. So, what can one do? Is there a method that could change the status quo and dramatically alter competitiveness, innovation and effectiveness?

Over the past six years, I have had the privilege of visiting more than 250 different research operations at top-10 pharmaceutical companies, large and small chemical companies, and a wide variety of biotechnology companies. During these visits, I spoke to many scientists and research managers about how they do their work and was given insight into the kinds of products that would provide practical value for them and for their companies.

As the number of organizations I visited grew, a distinct pattern began to unfold. These organizations were spending voluminous sums of money on new technologies to analyze materials, increase throughput and license technologies that would provide an edge over their competition. Yet, despite the investments, the actual practice of documenting an experiment was the same as it had been for the past 100 years — most scientists were and still are writing their work in paper lab notebooks.

The failure to modernize lab notebook practices and overall record-keeping has led to a host of undesirable practices that ultimately undermine the ability to achieve an efficient discovery process. These practices are sometimes as innocuous as not fully documenting an experiment (compromising repeatability), as malicious as “dry labbing” (documenting and not actually carrying out experiments) or as systematically damaging as repeating the failures of previous scientists. However, there are solutions to these problems that do not involve punishing the highly coveted scientists who drive research organizations.
Undesirable practices
My company recently participated in an audit of approximately 10 years of paper lab notebooks from a top-10 pharmaceutical organization. These notebooks were from a single project collection covering approximately 30 scientists. In the best-kept notebooks, approximately 15 percent of the pages were either illegible, incomplete or had an error that would lead a follow-on scientist to deem the record un-reproducible. In the worst-case notebooks, a stunning 60 percent of the entered experiments were not reproducible. This analysis did not take into account common shortcuts taken in paper write-ups such as “run in the normal manner,” making it unclear which method was applied. While discussing these findings with an executive at the organization, he acknowledged the problem and referred to it as the industry’s “dirty little secret.”

The most troubling practice is when scientists become too busy or distracted and do not carry out planned experiments. Commonly, the background, materials and planned experiment are written into the pages of the paper lab notebook as a placeholder to enter the actual experimental results. Following standard notebook practice, a line at the bottom would say “reaction not pursued.” This terminology is often used to describe experiments that were planned but not executed, as well as to indicate that something had failed and the rest of the experiment was not performed.

In some organizations, scientists are evaluated by the volume of work (number of notebooks) produced in one year, which has led to a practice called “dry labbing.” This term is used by many bench scientists to describe a situation in which paper notebooks are artificially padded with a number of entries of planned — but not executed — experiments.
Practical benefits of ELNs
By linking information from scientists with information from instruments to create a more complete digital lab environment, the maturing category of applications known as electronic laboratory notebooks (ELNs) can be employed as a useful tool for both scientists and organizations that need to realize the full benefit of the knowledge and experience within their teams. However, it is critical to keep in mind that, although ELNs can provide tangible business benefits, they are not management’s tool. They are ultimately the scientists’ tool. For ELNs to achieve full usefulness, they need to provide key benefits over paper. In our experience, we have seen three significant practical benefits:
• Reuse of data: Practical data reuse includes the ability to quickly and completely reference past experiment protocols, instrumentation setup and instrument data through a process referred to as “cloning” data. Hard measurements of data reuse point to savings of time previously spent documenting these items that can be measured in hours per week.
• Improved data quality: includes fewer errors in calculations, structures, reagents and approvals, resulting in fewer hours spent explaining past experiments to colleagues or re-assessing one’s own data.
• The ability to search: includes new capabilities to find and assess data on related compounds, procedures and attached reports.

Also, it is interesting to note that, in organizations where ELNs have been deployed, increased transparency and peer review can quickly halt dry labbing activity. Since, in practice, ELNs focus the attention of the organization on the experimental data itself, as opposed to insignificant but nevertheless easy-to-measure values, such as the number of completed notebooks, they can provide meaningfully accurate mechanisms for R&D organizations to better measure their true health.
The use of ELNs can change the roles of the industry’s brightest minds from lone principal investigators to team thought leaders, and can provide a means for them to extend their influence from a lab, to a site, to an entire organization. However, the root of the change offered by ELNs is often misunderstood. It is not about how we physically do things in the lab (e.g., using a computer keyboard and screen in lieu of pen and paper), it is about how we use our time and leverage information that already exists to make the best use of resources. With high-throughput research capabilities and 100+ person research organizations, the challenge is not to create data; it is to use that data to understand chemical behaviors (tradeoff curves) across conditions and scale.

ELNs today offer a host of direct benefits to scientists: automatic reporting, increasing the reproducibility of experiments and assisting in the creative process of planning experiments. Although these highly coveted features are important when evaluating an ELN, it is also important to note that the organizational benefits achieved with a successful deployment of an ELN are often undervalued or overlooked. The examples discussed in this article are part of a collection of systematic practices that lead to inefficiencies within discovery research. Eliminating any one of these practices would lead to a solid justification for return on investment. Collectively, they add up to a pattern of waste that needs immediate attention from any organization performing research in today’s highly competitive market.

Robin Y. Smith is Vice President, Product Strategy at Symyx Technologies. He can be reached at