At AIT Austrian Institute of Technology a work group of ten young researchers met to discuss their previous experiences in Science 2.0 as well as potential benefits of and current or experienced barriers to Science 2.0. During the second meeting the attendees discussed the questionnaire of the Public Consultation on Science 2.0 by the European Commission with an invited expert form Austria, Dr. Michael Strassnig from Vienna Science and Technology Fund (WWTF). In general the attendees are convinced of the advantages of Science 2.0 approaches in terms of improving scientific results and making science more efficient. However most of the participants are currently not practicing Science 2.0 approaches at all because those cannot easily be aligned with the requirements and constraints of an academic career imposed by the system currently in place – at least this seems to be the case in most disciplines as well as research institutions. The majority of the emerging Science 2.0 practices are not part of the curriculum of a scientist (yet). This should be changed in order to boost those practices (i.e. give researchers more space for these activities). Key challenges identified by the work group are the missing (or currently rather scattered and not well established) infrastructure for scientific collaboration on-line, assessing and defining good scientific practice in terms of Science 2.0 outputs, and moving from the current focus on (mainly successful) scientific results to the whole research process (i.e. to think of science as an on-going process). It is not clear if policy interventions on European level can substantially support changes towards implementation and acceptance of Science 2.0 practices. On the one hand it is fundamental to create a framework supporting the policies, e.g. through funding of specific activities. The attendees are of the opinion, that requirements for EU-funded projects towards Science 2.0 will have the greatest and most direct impact on the habits of researchers working in EU-funded projects. On the other hand uncertainty, concerns or limited awareness about Science 2.0 could lead to reactionary reactions. The adoption of new practices in the research environment needs a cultural change in the current research system. The European Commission could work towards building public awareness of Science 2.0 in order to support acceptance and uptake of Science 2.0 practices as well.
First Meeting on 2014-09-02
Attendees: Markus Hofstätter, Roman Karl, Georg Nebehay, Andreas Opitz, Alexander Schindler, Bernhard Schrenk, Christoph Weiß, Oliver Zendel, Michela Vignoli, Axelle Gasne.
The attendees work in the following Research Areas/Business Units in AIT Austrian Institute of Technology: Next Generation Content Management Systems, Optical Quantum Technologies, Safe and Autonomous Systems, Video and Security Technology.
Introduction to Science 2.0 by M. Vignoli
Open discussion on Science 2.0
1. Previous experiences with Science 2.0/Open Science
a. The majority of the attendees never encountered this topic during their research work (i.e. did never hear of it or were never asked to publish Open Access or to blog).
b. Attendees from the groups Next Generation Content Management Systems, Safe and Autonomous Systems, and Video and Security Technology are experienced with Open Source practices (i.e. sharing code and research results with the community, open licences).
c. One of the attendees from Optical Quantum Technologies was involved in a project proposal for a H2020 Open Data Pilot. They decided to opt out from publishing the data openly.
d. A few attendees have own websites and blogs. One of the attendees is additionally active on Twitter and posts demo-videos.
i. The frequency of blogging and updating the personal website depends on the availability of time.
ii. One of the attendees regularly blogs about his research and re-uses the posts for his papers.
iii. On the personal website of one of the attendees there are much traffic and on-going discussions.
e. One of the attendees reported about his experiences from working at the Technical University of Vienna: data citation, benchmarking, and ensuring reproducibility are common practice there. At AIT this seems not being practiced to the same extent (yet).
f. One of the attendees has some experience with Figshare and published some data there.
g. One of the attendees fought for a project deliverable to become public and is currently working on making the data related to this deliverable public as well.
a. Increasing practicability and efficiency of research through e.g. Open Data and shared algorithms. Some attendees underlined that collaborative work always let them going further and deeper in their research.
b. Increasing visibility of own work.
c. Quicker detection of errors (principle of multiple-assessor verification).
d. Changes to scientific system: e.g.
i. Alternative publication system and Web 2.0 technologies: possibility to have dynamic and evolving publications.
ii. Alternative publication possibilities (e.g. Open Access): unglue communication of scientific progress from high priced journals leading to disproportional profit by publishers.
iii. Avoid double work: e.g. negative results should be considered for publication because this could spare other researchers to do the same unfruitful research again (i.e. they could analyse the methodology conducting to these results and/or start from another point, which in sum leads to enhanced scientific progress). One of the attendees reported that this already proved to be very straightforward in the context of Deep Learning (Part of machine learning methods, see http://en.wikipedia.org/wiki/Deep_learning).
iv. Impact on the peer-review process: Science 2.0 enables researchers to publish on innovative topics that may be otherwise blocked by peer-reviews (The issue is known, e.g. in the context of funding. See http://arstechnica.com/science/2014/09/is-there-a-creativity-deficit-in-science/). On the long term, Science 2.0 can foster open peer-reviewed practices (e.g. making the peer-review process more transparent).
a. Time constraints: with exception of one attendee, all attendees having a personal blog or website do not frequently blog, mainly due to time constraints.
i. One of the attendees reported that writing project proposals and laboratory work have high priority, thus leaving no time for writing blog posts.
b. Requirements of an academic career and related constraints in the current system, e.g. publishing papers:
i. Often no freedom to choose where to publish (i.e. pressure to publish in journals with high Impact Factor or through Conferences with high impact).
ii. Acceptance of publications: if the content of a paper has been (partially) published in e.g. blog posts beforehand the paper may not pass the peer-review because the content was already published (i.e. no original publication).
c. Relevant measures do not include new Science 2.0 practices:
i. Funding agencies measure success of a project mainly through the number of publications.
ii. Assessment and evaluation of scientist and his/her work: number of publications and impact factor are the critical measures.
d. Lacking incentives for Science 2.0 activities:
i. In the current system, which is influenced by capitalistic ideology, there are no incentives for sharing. This mentality should be changed for the sake of a more open scientific practice.
e. Competition and profit:
i. Reluctance to publish content, which could give competitors an edge (e.g. idea for new projects).
ii. Negative results are not being published to avoid bad reputation or profit drop, e.g. in Pharmacology.
iii. Science is often highly connected to production environments and industries, which tend to be more closed environments.
iv. Opening up research is particularly difficult when patenting is involved.
4. Conclusion and challenges
a. Science 2.0 brings advantages for scientific system: In general the attendees are convinced of the advantages of Science 2.0 approaches in terms of improving scientific results and making science more efficient. However most of the participants are currently not practicing Science 2.0 approaches at all because they cannot easily be aligned with the requirements and constraints of an academic career imposed by the system currently in place.
b. Implementation of Science 2.0: According to the attending researchers the majority of the emerging Science 2.0 practices are not part of the curriculum of a scientist (yet). This should be changed in order to boost those practices (i.e. give researchers more space for these activities).
c. Missing infrastructure: Create appropriate social tool(s) and infrastructure for scientific collaboration online.
i. The question is if online tools will be sufficient to guarantee a good collaboration of distributed scientific teams. Can collaboration restricted to the internet really work out? Networks of Excellence, like EURO-FOS, could be a good framework to boost collaboration and coordination of international expert teams.
d. Assessing and defining good scientific practice: With more scientific content being produced (e.g. blog posts) it will be more difficult to assess what good science is.
i. What will be future measures replacing the Impact Factor (i.e. which alternative metrics will prove to be adequate for measuring impact)? Is it conceivable to have measures similar to the Google website ranking?
ii. Quality assessment will be a crucial point, especially in terms of emerging and less controlled/moderated Web 2.0 contributions. If everything is open, how will qualitative research be defined (and identified)? How can its quality be guaranteed?
e. Moving from Results to On-Going Science: The current focus on (mainly successful) scientific results should be shifted to the whole research process.
i. We should not only be oriented towards achieving positive results, but also towards gaining new insights. If resources should be spent on new and innovative ideas, which are at risk of generating uncommon or exceptional results, how can this be justified within the current system?
Second meeting 2014-09-15
Attendees from AIT: Georg Nebehay, Andreas Opitz, Alexander Schindler, Bernhard Schrenk, Christoph Weiß, Michela Vignoli, Axelle Gasne; from Vienna Science and Technology Fund WWTF: Michael Strassnig.
B. Recognition of the issue
Do you recognise the trends described in the consultation paper as 'Science 2.0'?
- In general yes, we recognise the trends described in the consultation paper as ‘Science 2.0’. However, as the term includes many different aspects it is also somewhat blurred and can evoke insecurities.
What are the key drivers of 'Science 2.0'?
- Certainly the availability of digital technologies and their increased capacities, as well as the increase of the global scientific population are important drivers for Science 2.0. The new technological and social context has the potential to generate new approaches and solutions within the research system.
- Especially in research fields, where the traditional system does not properly work anymore, e.g. in life sciences, researchers already looked for new ways of collaboration and for disseminating their outputs (including publications). This is more and more the case in other disciplines as well – but not in all of them likewise.
- In general, the number of publications increases, and the criticism of the current peer-review system is growing as the traditional process proved not to be infallible (E.g. two Nature papers had to be retracted due to errors and plagiarism, see http://www.nature.com/news/stap-retracted-1.15488). In the context of project assessment by funding bodies, the current peer-review system tends more and more to fail when international review teams are involved (this is not the case in countries where projects are mostly nationally reviewed, e.g. in Switzerland).
- Public demand for faster solutions to Societal Challenges and for better and more effective science (replicability of research results, avoidance of duplication of research etc.), as well as growing public scrutiny with regard to research integrity and accountability of science and research are certainly driving Science 2.0 approaches as well. However, the influence by the public on scientific processes differs from research area to research area. Public demand may be a more relevant driver in research areas like life sciences. In general the research system is rather self-adjusting as yet.
- Some disciplines adopted Citizen Science approaches a while ago, e.g. in medical research. However participation was more restricted to collect data and/or metadata. More projects, where citizens can act as scientists are emerging as well. We do not have an opinion on how much these approaches are driving Science 2.0. We think that citizens should be able not only to contribute with data, but also to learn something from these activities (i.e. Citizen Science projects as means of education).
- Finally, we think that public funding supporting ’Science 2.0’ is a more important driver than scientific publishers engaging in ‘Science 2.0’. For instance, funding bodies/programmes requesting Open Access publications (e.g. ERC, H2020, and the Austrian Science Fund) have a major impact on the adoption of Open Access. Policies alone are not a mean for a successful implementation of Science 2.0 practices. It is fundamental to create a framework supporting the policies, e.g. through funding of specific activities. Funding bodies have a key position to support the adoption of Science 2.0 approaches fundamentally.
- Several scientific publishers contribute to the Science 2.0 movement by offering great possibilities to publish Open Access. However, especially big publishers like Elsevier or Springer offer several Open Access solutions, which require very high Article Processing Charges (APCs). Scientific publishers engaging in ‘Science 2.0’ can be a driver for Science 2.0, but in some cases they act rather profit oriented.
D. Implications of 'Science 2.0' for society, the economy, and the research system
We did not discuss the implications of ‘Science 2.0’ for society, the economy, and the research system.
On what specific issues within 'Science 2.0' to you see a need for policy intervention?
- We did not discuss which of the listed options are more important. However, from the discussion above the conclusion can be drawn that the European Commission should mainly work towards building public awareness of Science 2.0. Issuing recommendations for the member states on how to support the Science 2.0 movement, as well as supporting Open Access to publications and to research data in H2020 should be done further. Otherwise, it is hard to say what kind of policy intervention can be done on European level to promote and support Science 2.0 in the member states. The requirements for EU-funded projects towards Science 2.0 will certainly have the greatest impact on the habits of researchers working in EU-funded projects.
- The fact that Science 2.0 includes new aspects which do not correspond to what we know to be good scientific practice and habits (e.g. introducing blogging etc. as a new mean for publishing or communicating scientific results) could provoke reactionary reactions.
- From the discussion above it can be concluded that from our point of view following aspects have least need for policy intervention: Citizen Science, Open Source/Open Code (as this seems already to be rather well integrated in the research culture of Information Technologies). Rather important aspects are: Research metrics, Assessment of quality of research, Research infrastructure.
- We did not discuss the aspects of Text and data mining, Data-intensive science, Alternative reputation systems. However it should be added that especially data mining and defining alternative reputation systems are rather important aspects as well.
With regard to the first three priorities you indicated above could you please specify what kind of policy intervention would be desirable?
We do not have recommendations for policy interventions.
- We agreed that uptake of Science 2.0 practices varies from discipline to discipline. There are disciplines which have already taken up (or started to take up) Science 2.0 practices, e.g. life sciences. But basically all sciences have the potential to engage with Science 2.0.
- The disciplines with slow impact so far either do not have a strong connection to technology or web applications, and thus their researchers may have stronger barriers against those technologies (but this is not necessary the case, see e.g. digital humanities); or there is not a pressing need to adopt other technologies for improving communication and collaboration of researchers or data analysis/manipulation within these fields.
- In our opinion there are no specific disciplines without real potential to engage ‘Science 2.0’.
E. Implications of 'Science 2.0' for researchers
Acknowledgement of ‘Science 2.0’-based activities
- We think that ‘Science2.0’-based activities (including data curation) should somehow be taken into account for career progression of researchers in order to help implementing those practices into the research culture.
- We did not discuss the answer ‘Science 2.0’-based activities should not have any impact on the recruitment practices of research performing organisations.
What are the most effective channels for awareness-raising of ‘Science2.0?
We did not discuss this question.
F. Opportunities for and barriers to 'Science 2.0'
What are the opportunities for 'Science 2.0'?
Potential opportunities at the level of the individual scientist
- As discussed in the last meeting, Science 2.0 practices (e.g. blogging, Open Access publications) can widen dissemination and sharing of research outputs, offer greater publication opportunities, and give the opportunity to involve in extended, international networks of researchers. Science 2.0 improves knowledge-sharing in general (e.g. through re-posting or linking), and the social impact of a publication can be tracked more easily by the researcher, e.g. via social networks.
- In certain cases the increased visibility of individual researchers through Science 2.0 practices may enhance career perspectives (E.g. Dr. Victoria Williamson was invited to talk at TEDMEDlive only a few years after receiving her PhD because she had gained a lot of impact through her Web 2.0 activities). However these cases are rather exceptional, as a vast majority of the bloggers do not have a very high impact. Already known researchers with certain fame in a specific area are often more successful in terms of social impact. This may lead to a stronger hierarchy among the Web 2.0 application users, which is driven by the social impact of the users’ contributions. Furthermore, depending on the research field, Science 2.0 activities like blogging are not always accepted and may not be helpful for a researcher’s career.
- Science 2.0 practices could help revising the peer review system towards a more open and transparent peer review process. However this is also attached to the acceptance of more open practices within the research communities.
- E.g. blog posts can help engaging with a wider public and with society at large, as posts are a good medium for communicating research results on a high-level. Science 2.0 can also help engaging e.g. neophytes.
- We did not discuss the answers Involvement in more multidisciplinary research and Research on problems that could not be addressed otherwise.
at the institutional level
- The listed answers, e.g. Driving economic growth, Promoting better science, Better value for money through avoiding duplication, Better value for money through accelerating the research process, are all valid opportunities. However, at this point it is very hard to tell which ones are more realistic projections or will be more important in future. We remarked that one may have high expectations of Science 2.0. However, despite the many good opportunities Science 2.0 can bring, we should pay attention to appropriately address potential risks as well.
- There is no direct causal relationship between Science 2.0 and economic growth. It is very plausible that Science 2.0 will lead to economic opportunities for future business models. How much research institutions will profit from these is hard to predict.
- Science 2.0 practices have the potential to improve science in terms of enhancing transparency, efficiency, and accelerating the research process. However it is to question if this can be labelled with “better science”. Science 2.0 can help overcoming some weaknesses of the current research system, but it is not the answer to all problems.
- Accelerating the research process can be a benefit, but it can also be a disadvantage in terms of sustainability.
- Science 2.0 can help avoiding duplication of work through availability of e.g. research data, documentation of negative results. In this context it is to question who will define what duplication of work is, e.g. when it is about re-doing experiments to prove the validity of methods and/or results. This is an important question, especially in terms of funding/justifying a specific research project.
What are the barriers to 'Science 2.0'?
Potential opportunities at the level of the individual scientist
- A major barrier is the lack of acknowledgement / credit-giving for ‘Science 2.0’ activities (e.g. curated data, science blogs, etc.). The currently missing appreciation of Science 2.0 efforts hinders a successful introduction of such approaches. The adoption of new practices in the research environment needs a cultural change in the current research system, from a rather closed (and e.g. in the case of peer-review anonym) approach to a more open and transparent one. This process can be supported by introducing incentives for Science 2.0 activities (e.g. by research funders, or on institutional level).
- As we discussed in the last meeting, lack of integration in the existing infrastructures (which somewhat goes with a lack of financial support) are major barriers. This results in lack of time for and/or acceptance of Science 2.0 practices. However, this differs from research area to research area. E.g. in IT, where Open Source practices are common, blogging etc. is more integrated into the researchers’ routine. E.g. in Photonics Science 2.0 practices are not common (except for occasional blog posts for EU projects, which are required by the research funder).
- Lack of new research skills necessary in the context of ‘Science 2.0’, e.g. data management skills, is certainly another crucial barrier. In this context researchers may not know which tool to use to reach the right audience and/or stakeholders (e.g. where to blog, which social media) or to manage their data efficiently.
- Concerns about quality assurance of new and non-traditional research outputs can be a valid barrier. The aspect of quality measures is discussed below (see G. Development of research metrics and quality assurance).
- Legal constraints (e.g. copyright law) are certainly another barrier. Often it is not the constraints themselves, but the fear of infringing legal constraints which prevents researchers from e.g. putting published papers on a green Open Access repository.
- Uncertainty / doubts about the potential benefits of ‘Science 2.0’ for research, Concerns about ethical and privacy issues, as well as Limited awareness about the potential benefits of ‘Science 2.0’ for researchers can be major barriers for researchers, who are not aware of and/or informed about Science 2.0 aspects. This can be mitigated by information and education measures.
- We did not discuss the answers Lack of incentives for early-stage researchers specifically to participate in new science and research practices, and Uncertainty / doubts about the potential benefits of ‘Science 2.0’ for the economy and society.
at the institutional level
- All the listed barriers (i.e. Limited awareness of ‘Science 2.0’ and its potential benefits; Concerns about quality assurance of new and non-traditional research outputs; Concerns about ethical and privacy issues; Uncertainty / doubts about the potential benefits of ‘Science 2.0’ for research and for the economy and society) can be valid barriers preventing institutions to take steps towards Science 2.0 initiatives (e.g. agreeing on a statement, adopting a corresponding policy).
- An institution’s readiness/willingness to change/modernise its policies/standards is another barrier. However impact and success of Science 2.0 on institutional level are strongly dependent on its integration with the “enterprise culture” in place.
- We think that missing recommendations on how to tackle Science 2.0 aspects (e.g. Open Access, using social media in science) are a major barrier. We are aware of the fact that it is a hard decision to take 1. whether Science 2.0 should be promoted at institutional level at all, or 2. which action should be taken in order to do so, e.g. recommending specific tools (as this may go with investment decisions and risks attached to proprietary tools). For researchers it would be very helpful to have some sort of guidance or recommendation on which tools to use for Science 2.0 purposes.
G. Development of research metrics and quality assurance
- We agree that the determination of research metrics cannot be left to private actors, such as Mendeley or Research Gate.
- General remark about metrics: We think that it is really important to consider qualitative and quantitative metrics in balanced measure. In Austria conventional quantitative metrics tend to prevail in the assessment of projects, institutions, and individual researchers (i.e. number of publications, Impact Factor) as they are an easy mean to assess scientific output. However, metrics like Impact Factor are insufficient quality measures. An alternative model which considers alternative metrics is the Research Excellence Framework (GB). Here the social impact of research output is the central measure. Altmetrics can help adding more factors to measure impact. Still, those metrics are not the best mean to measure quality of scientific outputs – at least not on their own.
- As the Impact Factor is a rather straightforward way to measure the success of an institution, it is often referred to in terms of the institution’s output quality and ranking. In order to change that it is necessary to change the ways how we measure scientific achievements and reputation.
- We think that in the transition period, in which we currently are, Altmetics should supplement conventional metrics. On the long term it is important to assess if Altmetrics should replace conventional metrics, or if there are other metrics which help better determining the quality of research output. For this aim it is certainly recommendable to do research in order to advance quality assurance procedures.
- We did not discuss Altmetrics and if the European Commission should fund research to advance those metrics.
H: Role of research funding organisations, Member States, and the European Union
Public authorities could facilitate the uptake of ‘Science 2.0’ by
- Developing policies on data sharing for research purposes and on facilitating public access to scientific publications are certainly good initiatives. However, in our opinion public authorities should focus on increasing awareness of Science 2.0 within the public and the research communities in order to increase their acceptance. E.g. supporting the education of researchers in Science 2.0 methods would be a very useful initiative. [Addition: the main message of Emilio M. Bruna's recent post The Opportunity Cost of my #Openscience Was 36 Hours + $690 is that teaching/communicating concrete methods and practices of Science 2.0, i.e. how to do it, is crucial.]
- Increasingly taking into account ‘Science 2.0’-related activities by setting benchmarks is a good strategy to help supporting the necessary cultural change in the current research system. But as already mentioned it is crucial to implement framework conditions enabling the uptake of ‘Science 2.0’ activities. Thus we think that public authorities should focus on that.
- A remark regarding the planning of the implementation strategy: it is important to think of unintended effects of Science 2.0 policies. We think that Science 2.0 approaches have a strong potential to improve the scientific system. However, as already mentioned, they will not be a solution to all problems. It is important to observe what the actual impact of Science 2.0 approaches on the research system is, and especially to differentiate between research areas. Science 2.0 approaches may not be equally useful for all disciplines, and may be even overcharging the researchers.
- Reviewing evaluation criteria of research proposals as well as procedures of quality assessment of research are crucial steps. This aspect is discussed below (see G. Development of research metrics and quality assurance).
- ‘Science 2.0’-based research output should increasingly be acknowledged, but it is important to adapt the review metrics to this medium (e.g. a blog post is in very most of the cases not the same as a paper).
- A bottom-up driven process is happening anyway, but it is not very well organised and heterogeneous. As already mentioned, having no policies or guidelines at all is not a good option. However the question is how helpful initiatives of public authorities to encourage the uptake of new science practices are, i.e. on which level those should take place. Initiatives on a national level would probably have a higher impact than initiatives on a European level.
- The implementation of Science 2.0 should be supported by soft incentives. Too much pressure towards adopting new methods and workflows may provoke even more opposition to this.
- The European Commission is already promoting Open Access and to some extent Open Data under Horizon 2020. This could be widened out to other ‘Science 2.0’ aspects.
- We did not discuss the answers The European Commission should dedicate specific actions under the European Research Area to ‘Science 2.0’, and which ones would be desirable to be taken into account.
I: Terminology of the phenomenon 'Science 2.0'
- The term Science 2.0 is not frequently used in this context, as far as one attendee can tell from her involvement in the Open-movement in Austria. The more commonly used term is Open Science (E.g. see the OKFN work group on Open Science in the UK: http://science.okfn.org/, in AT: http://okfn.at/arbeitsgruppen/open-science-austria/). However, we do not have a specific preference for one or the other term.