Measuring Innovation… Gauging Your Organization’s Success
The good news is that innovation can be measured; the bad news is that it can become a complex exercise with many differences between organizations.
Technological innovation is defined as the "successful exploitation of new ideas." This exploitation of ideas can be interpreted as the design and/or development of new or improved products, processes, or services.
Over the past fifty years, technological innovation has become a diffused phenomenon with wide differences between industry sectors, in the size of the organizations involved, and in the complexity of the R&D efforts related to the innovations. Numerous surveys and studies, however, have shown that innovation within an organization can still be measured and the results related to the organization's performance.
Measuring innovation, thirty years ago, was a relatively simple process involving factors that related mostly to the level of R&D spending, the number of patents generated, and the financial costs of the innovation. Since then, however, innovation measurements have become a more complex concept that combines those initial features with the relationships of human resources, bibliometrics, the types and quality of products or processes generated, combinations of existing and new indicators, productivity enhancements, and a number of intangible investments.
The measures of innovation
In today's highly competitive global marketplace, the immediate impact of innovation has become focused on products. Indeed, in a recent R&D Magazine reader survey on innovation, the single largest measure chosen by the readers—by a significant margin—of an organization's success in innovation was noted as the "ability of a new product to solve a customer's problem." The number of patents obtained by the organization—the historical measure of an organization's innovative success—was listed after other product-based measures of innovation.
A recent report by the National Academy of Sciences, "Measuring R&D Expenditures in the U.S. Economy (2004)," confirms this current dependence on a product-based definition of innovation. According to the report, "innovation measures must cover five activities:
• the introduction to the market of new products;
• the development of new processes to produce or deliver products to the market;
• the funding of new sources of supply of raw materials;
• the development of new markets; and
• changes in the organization of firms."
Measuring the number, value, and actual economic success of newly introduced products by an organization, however, is a somewhat subjective measurement of the organization's innovation capabilities. The degree of novelty within this environment can vary from product to product and from organization to organization. Measurements in this framework are best performed within the organization's own framework—making a comparison to other organization's new products are often not comparable and, at best, subject to debate.
Quality of patents
Measuring the number of patents obtained by an organization is a more quantitative measure of its innovation capabilities than that of a purely product-based evaluation. However, outside of the absolute number of patents obtained, the actual quality and innovative value of an organization's patent portfolio can also be somewhat subjective.
For each of the past 12 years, IBM, Armonk, N.Y., has been granted more U.S. patents than any other company. It has an active portfolio of more than 25,000 patents in the U.S. and 40,000 patents worldwide. Does that make it the most innovative organization in the high-tech community? Not necessarily. It surely can be considered an innovative organization, but what IBM does have is an infrastructure that is conducive to creating patents out of its IP.
IBM has led the U.S. for a long time in the ability to create patents and to earn large sums of money by licensing them. But even that policy may be changing slightly. Recently, IBM made more than 500 of its information technology software and hardware patents available to anyone developing open-source software in an effort to foster continued innovation.
According to research by Jean Lanjouw and Mark Schankerman published by the UK's Royal Economic Society and by the U.S.'s National Bureau of Economic Research (NBER), "R&D productivity is not strongly related to the quality of patents held by an organization. But there is a strong association between the stock market valuation of an organization and the average value of the patents they hold." This is particularly relevant in the pharmaceutical and instrumentation industries and likely in the IBM situation as well.
Other measurements
The number of publications produced by an organization's technology staff, the licenses they obtain, and technology awards they receive are among other measurements noted in the R&D reader survey.
Traditional bibliometrics utilizes quantitative analyses and statistics to describe patterns of publications within a given field. This highly technical endeavor can provide very quantitative measures of the innovation capabilities of an organization by the number of citations and citation-coupling obtained within the organization's published works.
A new growth in the area of bibliometrics has been in the area of Webmetrics, or cybermetrics. Similar bibliometric statistical techniques can be used in Webmetrics to scientifically map areas of the Web that are referenced to an organization's IP, based on the number of times they are hyperlinked to other Web sites.
Licensing is another highly quantitative technique that can be used to gauge the value of an organization's innovation capabilities. Often intrinsically linked to an organization's patent portfolio, licensing continues to provide a substantial revenue stream for many technology-based organizations.
Technology awards received by organizations are yet another indicator of an organization's innovative capabilities. These include the likes of our own R&D 100 technology awards, best-of-show awards at trade shows, other niche magazine technical awards, and even Nobel prizes.
Objectives and obstacles
A study by Dirk Vanderloop at the Univ. of Southern California on "Success Factors and Patterns in Government-supported R&D" found that "despite the wealth of studies over the past forty years that have attempted to specify the factors most responsible for the success of industrial R&D and new product development, there is still no single consensus." Vanderloop states that an invention's actual use in a very important part of the overall innovation process—success lies in the implementation of an innovation. This again notes the product emphasis in a successful innovation process.
Another study of 7,000 corporate R&D managers by Nine-Sigma, an innovation-sourcing firm based in Menlo Park, Calif., found that the most important objective for R&D organizations today is to increase the rate of new product innovations—a dramatic shift from "reducing R&D costs" which NineSigma found was the most important objective in its similar past surveys. More than 90% of the NineSigma survey respondents expect innovation to be a top priority in their organization within the next year.
In yet another study by the OECD in Italy, the objectives of innovation were found to again focus on improving the quality of products, developing new product markets, and extending the range of new products. In the same study, obstacles to achieving these innovation objectives were found to be that the costs and risks were too high and that there was a lack of financing.
Similarly, in R&D Magazine's recent reader survey, the largest reader response for the key component of a successful innovation was in the funding of the innovation. Other key components were in obtaining a trained staff, working in a collaborative environment, having a technology champion, and working in a corporate environment that supports innovation.
The value and cost of innovation
Throughout all the studies and surveys mentioned, it was noted that successful innovations are essential to creating the new products and processes that are required for the continued growth of the organization. The key component for ensuring these innovations is to provide adequate funding. Measuring the innovations is an important criterion for tracking the success of current and past innovations to ensure that a continuing flow of new innovations is maintained.
—Tim Studt
Resources:
Council on Competitiveness, 202-682-4292, www.compete.org
IBM, 800-IBM4YOU, www.ibm.com
National Academy of Sciences, 202-334-2000, www.nationalacademies.org
NineSigma, 216-295-4800, www.ninesigma.com
Organisation for Economic Co-operation and Development (OECD), +33-1-45-24-82-00, www.oecd.org
Univ. of Southern Calif., 213-740-2311, www.usc.edu
Technological innovation is defined as the "successful exploitation of new ideas." This exploitation of ideas can be interpreted as the design and/or development of new or improved products, processes, or services.
 click the image to enlarge "The ability of new products to solve a customer's problems" is the leading indicator of an organization’s success in innovation. |
Measuring innovation, thirty years ago, was a relatively simple process involving factors that related mostly to the level of R&D spending, the number of patents generated, and the financial costs of the innovation. Since then, however, innovation measurements have become a more complex concept that combines those initial features with the relationships of human resources, bibliometrics, the types and quality of products or processes generated, combinations of existing and new indicators, productivity enhancements, and a number of intangible investments.
The measures of innovation
In today's highly competitive global marketplace, the immediate impact of innovation has become focused on products. Indeed, in a recent R&D Magazine reader survey on innovation, the single largest measure chosen by the readers—by a significant margin—of an organization's success in innovation was noted as the "ability of a new product to solve a customer's problem." The number of patents obtained by the organization—the historical measure of an organization's innovative success—was listed after other product-based measures of innovation.
A recent report by the National Academy of Sciences, "Measuring R&D Expenditures in the U.S. Economy (2004)," confirms this current dependence on a product-based definition of innovation. According to the report, "innovation measures must cover five activities:
• the introduction to the market of new products;
• the development of new processes to produce or deliver products to the market;
• the funding of new sources of supply of raw materials;
• the development of new markets; and
• changes in the organization of firms."
Measuring the number, value, and actual economic success of newly introduced products by an organization, however, is a somewhat subjective measurement of the organization's innovation capabilities. The degree of novelty within this environment can vary from product to product and from organization to organization. Measurements in this framework are best performed within the organization's own framework—making a comparison to other organization's new products are often not comparable and, at best, subject to debate.
Quality of patents
 click the image to enlarge Almost three-quarters of those surveyed noted that access to technology is the main driver in being technologically innovative. |
For each of the past 12 years, IBM, Armonk, N.Y., has been granted more U.S. patents than any other company. It has an active portfolio of more than 25,000 patents in the U.S. and 40,000 patents worldwide. Does that make it the most innovative organization in the high-tech community? Not necessarily. It surely can be considered an innovative organization, but what IBM does have is an infrastructure that is conducive to creating patents out of its IP.
IBM has led the U.S. for a long time in the ability to create patents and to earn large sums of money by licensing them. But even that policy may be changing slightly. Recently, IBM made more than 500 of its information technology software and hardware patents available to anyone developing open-source software in an effort to foster continued innovation.
 click the image to enlarge Funding to support innovation efforts continues to top the list of necessary components in a successful innovation program. |
Other measurements
The number of publications produced by an organization's technology staff, the licenses they obtain, and technology awards they receive are among other measurements noted in the R&D reader survey.
Traditional bibliometrics utilizes quantitative analyses and statistics to describe patterns of publications within a given field. This highly technical endeavor can provide very quantitative measures of the innovation capabilities of an organization by the number of citations and citation-coupling obtained within the organization's published works.
A new growth in the area of bibliometrics has been in the area of Webmetrics, or cybermetrics. Similar bibliometric statistical techniques can be used in Webmetrics to scientifically map areas of the Web that are referenced to an organization's IP, based on the number of times they are hyperlinked to other Web sites.
Licensing is another highly quantitative technique that can be used to gauge the value of an organization's innovation capabilities. Often intrinsically linked to an organization's patent portfolio, licensing continues to provide a substantial revenue stream for many technology-based organizations.
Technology awards received by organizations are yet another indicator of an organization's innovative capabilities. These include the likes of our own R&D 100 technology awards, best-of-show awards at trade shows, other niche magazine technical awards, and even Nobel prizes.
Objectives and obstacles
A study by Dirk Vanderloop at the Univ. of Southern California on "Success Factors and Patterns in Government-supported R&D" found that "despite the wealth of studies over the past forty years that have attempted to specify the factors most responsible for the success of industrial R&D and new product development, there is still no single consensus." Vanderloop states that an invention's actual use in a very important part of the overall innovation process—success lies in the implementation of an innovation. This again notes the product emphasis in a successful innovation process.
 click the image to enlarge More than half of the readers surveyed noted that innovation is being used a measure of performance within their respective organizations. |
In yet another study by the OECD in Italy, the objectives of innovation were found to again focus on improving the quality of products, developing new product markets, and extending the range of new products. In the same study, obstacles to achieving these innovation objectives were found to be that the costs and risks were too high and that there was a lack of financing.
Similarly, in R&D Magazine's recent reader survey, the largest reader response for the key component of a successful innovation was in the funding of the innovation. Other key components were in obtaining a trained staff, working in a collaborative environment, having a technology champion, and working in a corporate environment that supports innovation.
The value and cost of innovation
Throughout all the studies and surveys mentioned, it was noted that successful innovations are essential to creating the new products and processes that are required for the continued growth of the organization. The key component for ensuring these innovations is to provide adequate funding. Measuring the innovations is an important criterion for tracking the success of current and past innovations to ensure that a continuing flow of new innovations is maintained.
—Tim Studt
Resources:
Council on Competitiveness, 202-682-4292, www.compete.org
IBM, 800-IBM4YOU, www.ibm.com
National Academy of Sciences, 202-334-2000, www.nationalacademies.org
NineSigma, 216-295-4800, www.ninesigma.com
Organisation for Economic Co-operation and Development (OECD), +33-1-45-24-82-00, www.oecd.org
Univ. of Southern Calif., 213-740-2311, www.usc.edu
Use of this website is subject to its terms of use.
Privacy Policy
