R&D Platform for Investment and Evaluation (“R&D PIE”) – Observatory of Public Sector Innovation

The Government of Korea is beginning to implement a new innovation investment model, 'R&D PIE', which leverages big data analytics and machine learning in order to assess disruptive changes in the technology landscape, and to identify overlaps and potential opportunities across the Korean ministries. Through this, the government has a way of identifying missing links in the innovation initiatives, fostering collaboration among agencies, universities, and companies, and solving social problems.

Innovation Summary

Innovation Overview

Technological innovation is a key to national growth and prosperity. Recent advancements in artificial intelligence and cyber-physical systems are accelerating technological and industrial transformation across multiple sectors. In response to fast-changing technological landscape, many governments are allocating their resources to research and development programs, based on policies that maximize each country’s innovative capacities. In Korea, government funding for research and development has been growing steadily. Yet, the increase in investments has not fully contributed to innovative outputs. The fast-follower innovation strategy that propelled Korea’s success has reached its limit.

It is in this context that the Government of Korea designed a new R&D investment and budgeting model. Ministry of Science and ICT has identified three key problems to existing innovation funding policies:

Research and Development programs are fragmented among 14 different ministries and agencies, and information sharing is limited from the planning stage.
Basic, fundamental research is not connected to later stages of applied and commercial research and development.
There are inadequate considerations for regulatory barriers at the development stages. Furthermore, the feedback cycle between evaluation and funding is often misguided.
In order to address these issues, the Government of Korea piloted a new investment process, the Research and Development Platform for Investment and Evaluation(R&D PIE), in order to put the national RD system on a sustainable innovative path. The PIE system is built for each strategic technology sectors and is based on big data analytics of academic research, patent analysis, economic impact, and market information. During the planning stage, PIE system provides a basis for public-private partnerships and consortium building, as well as inter-agency efforts in combining agency resources for cross-sector innovation. The system enables ‘fast-track’ implementation of multi-stakeholder research agenda formation and streamlines bureaucratic procedures coupled with large government programs. At the investment stages, PIE system interconnects individual research projects with relevant national and sector policies and relevant human resource planning for enhancing feasibility and success of national initiatives. At the evaluation level, the system aims to accelerate feedback process, so the programs and projects could change course in real time, instead of going through bureaucratic processes.

Started as a conceptual framework in April 2017, the PIE model was approved for development by Minister Yongmin Yu in June. More than 300 academic, industry, and technical experts participated to develop the model. In January 2018, we came up with four models — Autonomous Vehicle, Precision Medicine, High-Performance Drones, and Air-Pollution Mitigation. In March 2018, We added an additional four models — Smart Farm, Smart Grid, Intelligent Robot, and Smart City. Currently, we are building models for Artificial Intelligence and Alternative Energy.

The model was recognized as one of the best policies of the Ministry, and, on February 2018, the R&D Investment Innovation Plan, based on the model was approved by the Economic Ministers Meeting chaired by the Deputy Prime Minister for Economics. 2018 fiscal year budget of 250 billion KRW was allocated using the model, and, for the 2019 fiscal year budget, 800 billion KRW is in the process of National Assembly approval.

Innovation Description

What Makes Your Project Innovative?

The R&D PIE model creates impact in economies because:

1) It provides evidence-based policy platform for not only innovation policies but also for the wider public policy arena. Agencies can monitor, analyze and manage technologies, talents, and regulatory issues via the PIE model. (example: the PIE platform interconnects more than 700 pollution mitigation research programs with long-range policies of eight different agencies.)

2) It improves the quality of public service to citizens. Instead of vague R&D goals, projects would be clustered and connected, and the public policies would be designed with higher resolution, for more effective public service delivery. (example: personalized medicine initiatives from different ministries are being coordinated through the PIE model, adding improvements to the national medical service provider system.)

3) Most importantly, it enhances the credibility of government innovation policies and brings innovation stakeholders to the same table.

What is the current status of your innovation?

As of this date of submission in 2018, this model has completed its first round of implementation for the fiscal year 2018. We are in the process of adding more machine learning and deep-learning methodologies to enhance data-integrity, as well as prediction and counter-factual functions of the model. In general, we are trying to reduce the labor-intensive part of running the model.

Innovation Development

Collaborations & Partnerships

Ministry of Science created and launched the program. Several other ministries participated in designing and fine-tuning the model, including Ministry of Industry, Ministry of Transportation, National Policy Agency, Ministry of Health, NIH, Korea Weather Service, Ministry of Culture, etc. Ministry of Finance and Economy, the lead agency responsible for national budgeting, fully supported the effort in realizing the model into implementation.

Users, Stakeholders & Beneficiaries

Investment and Budgeting officers are the first beneficiaries of the model. The model is already enhancing the credibility of public policy.

Ministries are benefiting from realizing potentially no-go projects. The model helps with the planning and implementation of research projects, as well as their feasibility.

Program Managers and researchers benefit from the model, especially from the rich set of data that they can reference, as well as creating collaborations and partnerships.

Innovation Reflections

Results, Outcomes & Impacts

We were able to reconcile a number of national programs that were on hold because of differences among ministries. For an example, a national large-scale autonomous vehicle program was on hold for a year and a half because of the disagreement among ministries. We were able to design an agreeable program that emphasizes each ministry’s strength and their policy discretions.

The model is quickly taking hold across different agencies. We would gradually expand the model into a majority of the R&D programs. The Ministry of Economy and Finance is also contemplating using the model for government budgeting other than R&D programs. We might be able to see the model in national SOC programs or Social Welfare Programs as well.

Challenges and Failures

This approach is still shaping, and, while it has a potential for effectiveness, it requires more attention than the traditional approach for now. The dataset for analytical framework are still not so clean and often requires extensive manual labor to be worthy of analysis. We hope things would get easier with additional years of experience in analytics, budgeting and evaluation. Dedication of the PIE team would be important in order to manage the complex analytical system with a not-so-perfect set of data. In the future, as data-consistency improves, and the deep-learning model gets more precise, we hope the model would contribute to making R&D budgeting much easier for the national policymakers.

Conditions for Success

Since the result of the model was extremely uncertain, support at the Ministerial level was essential. Furthermore, the model was a huge departure from how things have been done in many agencies. hundreds of government officials and program managers contributed to making this model work, by doing parallel work of both the old way and the PIE way.

Replication

The PIE model could be replicated in other countries. Given that it is a data-intensive analytical model, factors that would condition replication would include:

Factors that would condition replication would include:

– Availability of comprehensive R&D program data with sufficient integrity

– Availability of experts who can analyze the program data along with research publications, patent, and market analytics

– Ministries willing to change the course of their policy, and officials who are willing to test different approaches to their policy-making process.

Lessons Learned

One of the most important lessons from using the model was the power of data analytics. Stakeholders were able to settle their differences as a result of the evidence-based policy-making process. Often different agencies speak different technical languages and bring in different perspectives. For autonomous vehicle model, Ministry of Industry perceived autonomous vehicle as a unit of physical product, whereas the Ministry of Transportation’s perception was a system of roads and sensors. We were able to narrow the gap.