The U.S. research and development environment is being shaped in 2008 by a myriad of economic, political, and technical factors.
Analysis conducted by Battelle researchers and the editors of R&D Magazine on the conduct of research and development in the U.S. reveals that the federal government,along with academia, industry, and non-profit organizations, will invest about $367 billion dollars in U.S. R&D in 2008, an increase of about 3.7% over that spent in 2007.
Throughout the past 10 to 15 years, the R&D enterprise in the U.S. has undergone changes which have been leading toward a new equilibrium between those who fund research and those who perform it. In large part, these changes have been driven by a number of different factors, some of which relate to theories of technology management and others which have been forced—directly or indirectly—by events over which we have had no control.
Overall, a review of the patterns of nearly a half-century of R&D funding and performance have demonstrated two interrelated basic characteristics—inertia and stability. These features of the system are largely the result of the planning and decision-making processes that characterized both industry and government in the U.S. The time lag between the budget authorization and actual outlays in the federal government and the planning process in industry, where R&D expenditures are influenced by sales and earnings, contribute to a degree of stability. Consequently, the entire R&D system moves gradually, generally with only small year-to-year changes in the patterns of support and performance.
The gradual evolution of the total system has, however, seen some significant changes in behavior. Perhaps the most disruptive of these—in the sense of having nearly step-function changes that are not likely to revert—can be traced to the following major events or activities:
|
| click to enlarge |
The restructuring of the major corporate R&D approaches in industry
The 1990s witnessed major changes in the manner in which industry acquired its technology assets.Many large corporate laboratories diverted their programs to business unit technical facilities and to subcontractors in the supply chain. The process was similar to that adopted by agencies of the federal government, namely the acquisition of technology through commercial off-the-shelf (COTS) vendors.
The significant growth of the practice of offshore outsourcing of R&D
While the practice dates back into the 1980s, more recent major initiatives have been taken in terms of contracting R&D to offshore performers. The initial trend of establishing technical support facilities to serve foreign marketing and/or manufacturing operations expanded to the contracting of R&D to independent performers and then onto and creating major technology development facilities.
The shift in federal government emphasis that is traceable,in large part, to the events of 9/11 and their aftermath
The terrorist attacks of Sept. 11, 2001, and the subsequent needs for greater defensive technologies marked a major event that had a significant effect on the R&D agenda of the U.S. federal government. Coupled with the need for developing technologies that were designed to counter an asymmetric war, a major investment in both materiel and technology development was made through a variety of federal government agencies. Special emphasis was placed on a broad array of research programs, including techniques to counter terrorist attacks and to develop/deploy means to achieve improved intelligence gathering and analysis. The concentrated effort had an immediate impact upon the distribution of resources among the many alternative research activities under the aegis of the federal government, with the consequent concerns regarding the broad strength of the overall U.S. R&D enterprise.
The growth of the federal deficit
The follow-on wars in Afghanistan and Iraq have consumed enormous resources, creating massive federal deficits, and thereby forcing a re-assignment of priorities among the many different elements of the federal discretionary spending. The principal exceptions the needs of the Dept. of Defense (DOD) and the newly created Dept. of Homeland Security as well as those agencies that could demonstrate direct tie-ins with the Global War on Terror and the broad generic area of homeland security.1 As a result, planned expansions in various research support areas have been curtailed.
|
| click to enlarge |
Characterizing U.S. R&D
The U.S. R&D enterprise is characterized by a matrix that describes the four major sectors that provide the funding (the federal government, private industry, academia, and other not-forprofit entities) and these same four sectors as performers of R&D. Ever since its inception, the National Science Foundation (NSF) has had the responsibility of collecting detailed data on the amounts of funds that have been dedicated to the conduct of research and the manner in which these have been distributed among the sources and the performers. In addition, the NSF has established an impressive library of data and analyses that cover many different aspects of the R&D establishment.
The NSF databases provide much detail on the patterns of the past distribution of funds and serve to address the needs of historians, policy analysts, and planners, to name a few. However, the development of forecasts requires inputs beyond this quantitative accounting; it rests on the amassing of qualitative perspectives on where both the federal government and industry are headed with respect to both the funding and the management of R&D programs. In addition, the changing face of the R&D enterprise in the U.S. is—to a degree much greater in the past decade—influenced by events from abroad. Thus other resources are required to obtain a more nearly complete (and hopefully accurate) assessment of the near-term directions and health of the U.S. R&D system.
To this end, the Battelle/R&D Magazine team calls upon a variety of resources beyond those from NSF. In particular, additional quantitative data are drawn from the analysis of the U.S. Federal R&D Budget, as performed by the American Association for the Advancement of Science (AAAS), the Organization for Economic Cooperation and Development (OECD, for data and perspectives on Foreign Direct Investment and off-shore outsourcing of R&D), and from Schonfeld & Associates (for historic company and industry data related to R&D investment by publicly traded companies). Qualitative insights come through surveys that are taken by the Industrial Research Institute (IRI) and R&D Magazine readership, as well as from a broad cross-section of Battelle staff.
As a result of the collection, assimilation, analysis, and digestion of these resources, Battelle and R&D Magazine focus on what is an expected distribution of R&D resources among the sources and domestic performers of R&D for the forthcoming year.Additional discussion is dedicated to the rationale behind the numbers and commentary on implications for the future.
Using figures on the historic GDP deflator and projecting from the most recent trends, we estimate that the 2008 forecast is inflated by a factor of approximately 2%, thus leading to a small real increase in total R&D.
The source-performer matrix
As seen from the structure of the source-performer (S/P) matrix, the federal government supports R&D as performed in all sectors: federal labs, industrial labs, academia, not-for-profit organizations, and a special set of institutions, the Federally-Funded Research & Development Centers (FFRDCs). The 36 FFRDCs, established by various government agencies, are government-owned, contractoroperated entities designed to carry out special long-term research programs on behalf of their parent agencies. The FFRDCs are operated by different entities, such as industry, academia, and not-for-profit organizations. From time to time, the management and operating contract may be re-bid, with the result that one that is managed by one sector could change to being managed by another.
For many years, the NSF's National Patterns data reported the performance of the FFRDCs in two different ways. For those that were managed by academia, the performance numbers were presented as a separate column in the historical accounting of the S/P matrix.However, the performance numbers for the FFRDCs that were managed by industry or by non-profits were not separately identified, but were combined with the numbers that represented federal support of R&D in those two sectors. From the very beginning of the Battelle forecast, we also combined the academia-managed FFRDCs and the federal support of academic R&D into a single cell, thus giving symmetry to the 4x4 S/P matrix.
Ever since the National Patterns data were available in detail on the Web, the NSF has separated the values of R&D such that there exists the capacity to look at the industry-, academic-, and not-for-profit-managed FFRDCs independently from the federally supported R&D in each of these sectors. For the past few years, their numbers have been recombined so as to maintain the continuity of presentation and form.Upon closer study, this form of combining statistics doesn’t permit sufficient insight into the overall R&D structure, as noted for the following two reasons:
1) Examination of the distribution of the character of work (basic research, applied research, and development) for the FFRDCs and their respective management entity reveals that there are significant discrepancies. One would not necessarily expect that the distribution of work effort would be similar: the managing entity is basically in place to manage and operate, not necessarily to define, direct, and lead. Thus, to assume that the FFRDC and its management group can be treated as similar entities is misleading, regardless of the size of the effort.
2) Over the years, the management contracts of many of the larger FFRDCs have changed, with the manager/operator having changed from being a representative of industry to being a nonprofit, or from an industry to a consortium of non-profits and academic institutions. Thus, there are significant discontinuities in the pattern of expenditures for the management sector. To base analyses and forecasts upon patterns of such discontinuities is likely to produce errors and misinterpretations and tends to confound an understanding of the roles of the various supporting and performing sectors. It must also be emphasized that there is greater continuity of the purpose and function of the FFRDC than there is of the management entity.
Thus, the presentation of the S/P matrix has been altered to the non-symmetric 4x5 S/P matrix, with the five performing sectors identified as federal laboratories, industry, academia, not-for-profit, and FFRDCs. This last category represents the sum of all three types of FFRDCs (industry-, academia- and not-for-profit-managed). Given this new classification, we now have the S/P matrix as shown in the table on page F3.
The academic sector
The academic sector is a complex mixture of funding sources and performance roles.As seen in the S/P matrix, this sector received funds from all sources—the federal government, industry, itself, and non-profit organizations.However, when seen as a source of funds, the academic sector does not support R&D in any sector other than itself.However, that which is classified as academic funds is complicated by the fact that these are defined in different ways and consist of three distinct types of income.
First, there is the component that is most simply viewed as income from donations (such as contributions by alumni to a university’s development fund and interest on investments); income from the sale of books, management of conferences, royalties, and associated miscellaneous funds; and support from government entities other than the federal government. Basic institutional support from state or local governments is immediately classified as “own funds”.2
In addition, there are university-based programs that have grown from federal government initiatives that permitted and encouraged the development of consortia of industries which have common interests, such as in semiconductors, materials development, robotics, and the like. Such consortium-supported basic research activities have generally been considered as sufficiently different from the concept of direct research on specific problems that they are not classified as industry-support/academia-performed projects, but as basic research that is directed toward the advancement of knowledge.As such, the financial support generated by industry is again classified as “own funds” and is included as part of the academic source.
|
| click to enlarge |
The federal budget
Prior to 1980, the federal government was the primary supporter of R&D in the U.S., although its share of total support had been diminishing for many years. Over the past 30 years, federal support has continued to grow in absolute terms, although it is still lagging behind the rapidly growing industrial support.
Perhaps the most detailed examination of the federal commitment to R&D has been undertaken in the continuing series pursued by the AAAS. The R&D Budget and Science Policy Program (www.aaas.org/spp/rd/fy08.htm) has achieved a reputation for completeness and accuracy and provides an excellent thumbnail picture of the very complex process of federal support of R&D.
More 2008 R&D Funding Forecast
Slowing Economy Dampens 2008 R&D Spending The U.S. research and development environment is being shaped in 2008 by a myriad of economic, political, and technical factors. Read the article
Government and Industrial R&D Face Challenges An internal Battelle survey draws upon the views and experiences of staff who are closely involved with the conduct of research that supports a wide range of government and industrial clients. Read the article
FY2008 Federal R&D Funding: Juggling Budgetary and Political Priorities Early budgetary plans stemming from Congressional initiatives failed to materialize as announced and planned due to budgetary constraints. Read the article
|
|
The AAAS analysis provides much detail on the federal budget, mining its way down through various layers and presenting an unparalleled and uncommon dissection that serves as the basis for much analysis and planning. Some of the more salient features of the AAAS analysis are as shown in the detailed sidebar on page F5.
As one goes through the AAAS analysis, it is important to recognize that the total federal R&D budget includes authorizations for both the conduct of R&D and the building/enhancement of R&D facilities (and similar major infrastructure, such as capital equipment). In addition, the authorization figures include appropriations that are to be made in both the corresponding fiscal year and in the out-years for long-term projects. Thus, any time-series comparison of total authorizations and expenditures will show gaps that extend for several years into the past.
Federal R&D priorities
There is no better way to look at the changing federal priorities than to inspect the manner in which the R&D budget has been distributed among the major technology-based thrust areas. This distribution of resources is shown in the recently released 2008 Science and Engineering Indicators, published by the National Science Board and available for download at www.nsf.gov.
The emphasis on space research waned after about 1965, by which time much of the research was being infused into operational activities, and continued to plummet following the initial moon landing of 1969. The later resurgence in research was coincident with early stages of the Strategic Defense Initiative, to be followed by a reclassification within NASA relative to what was termed “research.”
Energy research received a major boost following the oil embargo of the 1970s and the creation of the Dept. of Energy and then underwent decline in the early 1980s, partially driven by the perception that activity in the energy field was then in a position to become commercialized and the benefits of past research could be realized. It is expected, as noted in the earlier discussion on the federal budget detail, that energy research will benefit from increases over the next few years. This is being driven by considerations of both resource development and more efficient use.
Science programs have seen a slow but steady increase over the years, accompanied by a sudden rise (due to a reclassification of programs, especially in the Dept. of Energy). The proposed doubling of several general science-oriented programs, as included within the American Competitiveness Initiative (ACI), was expected to have an impact, but that has been dampened with smaller appropriations than those requested by the Administration.
Health research has seen a similar steady rise over the past four decades, with a major recent boost derived from the commitment to double the National Institutes of Health (NIH) budget over a short period of time. The ACI grew, in part, from an attempt to emulate the NIH research doubling process and direct major new funding toward basic physical sciences as well as expansion of education programs designed to create a broader and deeper teaching of science, technology, engineering, and math (STEM) throughout the education system.As of this writing, the ACI has not been fully committed, with budget deficits being touted as one of the principal reasons.
|
| click to enlarge |
Industrial support of R&D
Industrial support for R&D has grown steadily, albeit with a few periods of significant rise and other periods of basically flat behavior. Because of the dominance of industrial funding as a source for the overall R&D enterprise, it is particularly important to pay special attention to industrial attitudes and practices. To this end, we here look at a few of the major R&D funding sectors, the views of industrial leaders, and some of the observations relative to the growing area of off-shore outsourcing of R&D.
One of the more nearly complete compilations of data on industrial R&D spending is published annually by Schonfeld & Associates (SAI, www.saibooks.com) and draws upon the 10K reports submitted annually to the U.S. Securities & Exchange Commission.Various reports are published by SAI, including those which provide five-year historic sequences of R&D expenditures, sales, and earnings, as well as projections of future growth.
While the Schonfeld reports provide one of the most convenient sources for R&D-related data, a major drawback is the fact that they include only those data for publicly traded companies. Furthermore, as might be expected, the data are presented in terms of both the U.S. Standard Industrial Classification (SIC) and the North American Industry Classification System (NAICS). The NAICS descriptor provides a much more segregated, disassociated, and detailed accounting of industrial lines of business.However, for the purposes of inspecting R&D behavior, the classical SIC permits a broader picture of industrial trends.
The Schonfeld data, truncated to eliminate those SICs that do not engage in classical scientific and technical research, provide a means for inspecting medium-term trends in R&D support by industry.We must note that the basic data from which these estimates are derived may have some discontinuities, since they do not take into account the impact of changes caused by mergers and acquisitions.Where errors in the original data occur, they are most pronounced when such corporate ownership changes involve companies whose original SICs were distinctly different.
Based upon inspection of the original Schonfeld data and extension of expected behavior, the data for the top 15 R&D supporting industries, covering the period 2004-2006 and estimates for 2007 and 2008, are provided below.
|
| click to enlarge |
An industrial view
The annual survey undertaken by the Industrial Research Institute (IRI) provides a means of inspecting the trends of thinking within some of the major industrial supporters of R&D.While the membership of the IRI changes over the years, and the response to the questionnaire is similarly variable, one of the major values of this effort is the fact that the output provides a means for the longitudinal study of issues within the industrial sector.
In earlier analyses of their own data, the IRI tended to look at individual questions and follow the responses as stand-alone points of projection and discussion. In more recent times, the Battelle forecast has drawn upon the IRI results in such a manner as to look at small clusters of data which have some direct relationships and to thereby obtain a more cohesive and comprehensive examination of trends within the overall industry.
Some of the major findings from the IRI survey, as extracted from the past 14 years of tracking, are as follows.
Over the entire period of IRI results that are covered here (1995-2008), there has always been a tendency to invest R&D dollars more toward new business projects than to the support of existing business or directed basic research.3 The tendency to look toward new business projects increases in the 1995-2001 period, partially triggered by the collapse of the so-called “dot com” companies. Relatively strong conservative spending patterns were maintained for about three years, followed by a recovery of the new-business posture.When tied in with other observations, it is reasonable to suggest that much of the new business thrust is associated with an increase in subcontracting with outside R&D providers. Furthermore, it can be postulated that the “new business ventures” could well refer to “new markets” (e.g., foreign markets) rather than new products or processes.
The middle- to late-1990s represented a period in which the industry representatives anticipated slight growth in R&D expenditures and in capital spending for R&D activities. This was followed by a sharp decline in the following four years and is now experiencing significant growth in both categories. It would appear that a significant portion of this growth is going to be directed toward off-shore operations—including both the conduct of R&D and the brick-and-mortar projects associated with new facilities.
On inspection of two clusters of data (1—R&D Professional Personnel Level and Hiring of New Graduates, and 2—Outsourcing to Industry and Contracts with Academia), it appears that a significant portion of the increased expenditure on R&D will be directed toward outsourcing rather than to the hiring of new graduates or the increase in professional personnel level.4
Licensing to and from other companies declined in the early 2003-2005 period, followed by more aggressive planning for the increased in-licensing of technology. This latter practice,when coupled with the other observations above, suggest that a greater degree of technology asset acquisition will come from sources external to the primary company, a manifestation of the supplychain source proposition articulated earlier in this report.
|
| click to enlarge |
Offshore outsourcing
Offshore outsourcing of R&D has grown from two relatively small models in the 1980s to a significant practice that connects researchers and research capacity all over the world. It has become a complex weave of relationships, facilities, practices, opportunities, and threats that are changing the manner in which technical services are provided.
The details of just how much R&D support is provided between, for example,U.S. companies and their captive affiliates is not well known. To be sure, recent trade press and other news accounts have provided a broad range of reports on the investment in both research services and in the brick-and-mortar construction of major facilities.As noted in the most recent 2008 Science & Engineering Indicators, specific industrial data on the practice and the implications are largely protected due to the sensitive nature of internal business decisions.
Judging from data that go back to Dept. of Commerce (DOC) reports of 1995 and 1998, there has been a growing amount of R&D that is funded in foreign affiliates of U.S.-based companies. In addition, there has been a great amount of growth in the contracting of R&D to independent performers, such as universities, major research centers, and to private industry. Finally, while the original DOC reports were confined largely to interactions with facilities in Western Europe and the Pacific Rim, the present-day practice has expanded to developing countries in Asia (China, India, Korea, Singapore, Taiwan, etc.), Eastern Europe, and Latin America.
Every indication is that the move toward offshore outsourcing will continue and grow over the next few years. The earlier discussion relative to the IRI survey gives an example of the manner in which the disparate observations lead to a “connect the dots” conclusion that industry may very well be moving toward an extension of the present practice of offshore outsourcing, at least for the near future.
In spite of what appears to be a growing trend, with the expectation that there will be an adverse impact on the U.S. R&D enterprise, it is necessary to look at another continuing phenomenon: the offshore in-sourcing of R&D, work that is performed in the U.S. and supported by prime clients in other countries. The 2008 Science & Engineering Indicators report contains graphics that show the two-way flow of R&D into and out of the U.S. in one recent year, as summarized in the attached table.
All else being equal—including no major disruption in the U.S. economy and the consequent cutbacks of medium- to long-term R&D (too often seen as a "luxury")—the practice of offshore outsourcing will most likely continue and expand over the next few years. In this regard, it is appropriate to paraphrase the closing statement taken from the 2007 Battelle/R&D Magazine forecast:
The bottom line is that there is every reason to believe that the U.S. R&D enterprise will undergo some changes, moving perhaps rapidly to a position that eventually defines a new equilibrium within both the national and international landscapes. In many respects, one may look forward to a process not dissimilar from bungee-jumping, moving through a state of significant change that may be a bit frightening, but then settling to a different equilibrium position. These next few years should be quite an adventure.
|
| click to enlarge |
The need for better data
Just as we have been able to follow detailed historic trends with the NSF compilations and have been able to gain a perspective on the influence of different factors as they relate to R&D funding patterns in the U.S., so also is it important to capture and compare those features of the S/P matrix in other countries. For several years, the Battelle/R&D Magazine forecasts have noted the changing character of R&D, with one of the more recent major influences having been the most significant growth of offshore practices. Great emphasis has been given to the fact that offshore facilities (either extant or developed through host government and foreign direct investment industrial funding) have had an impact on the state of both U.S. business and the U.S. R&D structure and operations.
For the past several years, the authors of this forecast have stressed the need for a better understanding of the R&D structures—not just the totals—that characterize the R&D patterns, habits, trends, and influencing factors in other countries. Such data are important on two counts. First, they provide a better means of comparing the U.S. enterprise to that which exists in our competing or cooperating international partners. Second, such data provide a better understanding of the types of resources and scientific environment that will be faced by both government and industry when interacting with that scientific community.
A major step toward compiling a clearer picture of the patterns in a number of different countries was drawn from the databases of the OECD and included within the 2007 Battelle/R&D Magazine report on global trends in R&D. In that report, we took a more detailed look at the elements of the S/P matrix.
It is expected that U.S. businesses will continue to make, and to increase, their investment in both captive and independent offshore resources. However, for these investments to be made with greater confidence, it is important to understand the general R&D milieu of the host country. To the best extent possible, efforts should be directed toward characterizing the R&D environment and history of the host: total R&D support or performance is nowhere near as useful (or even pertinent) as a more detailed understanding of the relationships among the various types of sources and performers.
Furthermore, continued emphasis should be directed toward the collection of detailed data on another major aspect of industrial R&D. The transition from in-house to contracted R&D within the industrial sector has changed the structure of performance of R&D, yet has been masked by the manner in which statistics are collected and reported. Although such before the NSF even existed (and hence predates the efforts at tracking science and technology expenditures), the practice has been maintained wherein R&D that is funded and performed by industry does not distinguish between in-house and contracted efforts. Industrial support of R&D that is performed by academia and other non-profit entities is routinely reported, but the (estimated to be far greater) R&D that is contracted to other industry has not been isolated. For the sake of completeness, symmetry, and—far more importantly—a better understanding of the entire R&D system, this activity should be added to the list of resources that is available for the planner, analyst, and evaluator.
|
| click to enlarge |
Closing observations
The behavior of R&D funding and performance in the U.S. has undergone a slow evolution over the past few years and has been influenced by a number of different factors. Federal budget deficits and the acceleration of offshore outsourcing have contributed to a slower growth of domestic performance. The economic outlook, including the personal aspects of jobs, causes great concerns. And to a degree not seen in recent years, the average person-on-the-street is calling for long-term relief from high energy costs, improved—but non-intrusive—security, and resolution of environmental problems. The bungee-jumping analogy noted above and in the 2007 Battelle/R&D Magazine forecast seems to be in full swing, yet perhaps exhibiting a slowing of the plunge.
There is little question that there are basic problems facing the U.S. research environment, not the least of which include consideration of what was once termed “The Three Es—Energy, Environment, the Economy.” For over three decades, these have assumed various levels of concern and we are still wrestling with ways in which to attain balance among them. It is also obvious that the entire concept of national security is also intimately related to questions of energy independence and/or energy reliability.
The successes of former massive scientific and technological programs (such as the Manhattan Project and the moon landings) have tempted many to look at those experiences and ask “if we can put a man on the moon, why can’t we solve the energy problem/prevent and/or clean up pollution/improve high school test scores/(fill in the blank with the problem du jour)?”
The answer is simple. For the most part, the successes of NASA and the Manhattan Project were, for as much as we knew of them, purely a spectator sport: relatively few people were directly involved, and the rest merely paid the bills and marveled in the accomplishment. But the resolution of energy and environmental problems, as well as others, actually becomes a participant sport: we are all involved in the implementation of whatever technology is available to address the issue. The solution might come from the laboratory, but the implementation comes at the gas pump.
Finally, we need to address one of the most basic elements of the entire R&D enterprise, which include those who do the research, those who support the research, and those who, as consumers, actually implement the research. And that is education. Not just the education of those who are to become the scientists and engineers, but also of those who will be in a position to make decisions on the need for research and the benefits that can accrue. The future depends upon there being an appreciation of the role of science and technology in our economy and in our society.
Footnotes
1. We note the distinction between the upper case Homeland Security (the Cabinet department) and lower case homeland security (the function, which may be a component of the purview of many different Cabinet departments).
2. We note that such other governments may also contract with universities for the resolution of specific problems that fall under the purview of state or local government. Such funds are not easily captured on a regular basis but have been the basis for other NSF periodic data collection activities. Similarly, state and local governments support special research projects with industry and not-for-profit centers. These, too, are not explicitly identified in the S/P matrix. The amount of such support is relatively small and would not have a major impact on the present analysis and discussion.
3. “Directed basic research,” as used in the industrial context, is most generally that effort which is directed toward research that is supportive of selected lines of business, rather than the type of basic research that is more commonly associated with academia. For the most part, true basic research is conducted in very few industrial labs, with the possible exception of work in the area of biological and pharmaceutical and related areas. The types of basic research that were once associated with the effort that was expended at Bell Labs, GE, IBM, and many others in the early semiconductor days, is now largely confined to academia.
4. As is the case for all of the sets of responders to the IRI survey, we must remember that the IRI membership covers a broad range of companies and industries. While some industries are going into a highly expansive mode in terms of R&D expenditures and hiring, others may be in some decline in one or both of these areas. Depending upon the company and the representative industry, the responses from one or more participants may significantly skew the overall results.
Published in R & D magazine: Vol. 50, No. 1, February, 2008, p.F3-F15.