JultikaUniversity of Oulu repository

Open Research Data is gaining popularity nowadays, and various research units and individuals are interested to join this trend. However, due to variety of Open Research Data technologies, they have found it is difficult to select proper ones for their specific requirements. Thus, a method for evaluating of Open Research Data related technologies is developed in this study for researchers to select proper ones. Firstly, the theoretical knowledge of research data sharing and reusing barriers is resulted from a structured literature review. As a result, from the 19 primary studies, 96 instances of existing barriers are identified and classified to seven categories, where four of them are research data sharing barriers and rest of them are reusing barriers. This knowledge is regarded as an important resource for understanding researchers’ requirements on Open Research Data technologies, and utilized to develop the technology evaluation method. Additionally, the Open Research Data Technology Evaluation Method (ORDTEM) is developed basing on the Goal/Question/Metric (GQM) approach and resulted research data sharing and reusing barriers. To develop this method, the GQM approach is adopted as the main skeleton to transform these barriers to measurable criterion. Consequently, the ORDTEM, which is consisting of six GQM evaluation questions and 14 metrics, is developed for researchers to evaluate Open Research Data technologies. Furthermore, to validate the GQM-based ORDTEM, a focus groups study is conducted in a workshop. In the workshop, nine researchers who has the need to participate Open Research Data related activities are recruited to form a focus group to discuss the resulted ORDTEM. And by analysing the content of the discussion, 16 critical opinions are addressed which resulted eight improvements including one refinement on an existing metric and seven new metrics to ORDTEM. Lastly, a testing process of applying ORDTEM to evaluate four selected Open Research Data technologies is implemented also for validating whether it can be used in solving real-world evaluation tasks. And more than the validation, this experiment also results the materials about usage of ORDTEM, which is useful for future adopters. However, more than developing the solution to eliminate the difficulty of selecting technologies for participating Open Research Data movements, this study also provides two additional contributions. For one thing, resulted research data sharing and reusing barriers also direct the future effort to prompt Open Research Data and Open Science. Moreover, the experience of utilizing the GQM approach to transform existing requirements to evaluation criterion is possible to be studied for developing other requirement-specific evaluation.