Important Information for Reviewers of FWF Proposals

Should you receive an invitation from FWF to review a (stand-alone) project proposal, please read this first. Note, this information is based on several discussions with other Austrian researchers in Computer Science and my personal experience with FWF. If you have comments/feedback, feel free to send them to me via email.

The Situation in Austria

In Austria, it is particularly difficult to acquire funding for fundamental research from FWF, Austria’s central funding organization for basic research. The main reason is simply lack of money. And it is even more difficult for applicants in the areas of Computer Science, among them Software Engineering. Please note, in Austria we compete with ALL disciplines which in my opinion is unfair as indicated by the acceptance rates published by FWF!

Looking at stand-alone project proposals, 37.9% of the submitted proposals in Mathematics, 37.6% in Linguistics and Literature, 36.1% in Physics and Astronomy, and 32.6% in Chemistry got accepted in 2020 with an overall acceptance rate of 26%. But, only 20.4% of the submitted proposals in Computer Science and, I strongly assume, even less in engineering disciplines, such as Software Engineering, got accepted. For more details see the statistics published by FWF in their Dashboard.

Honestly, I do not think that Austrian researchers in Software Engineering write worse proposals than the researchers in other disciplines. Instead, I believe the main problem is that our Software Engineering community is TOO critical compared to other communities. Other communities seem to know that rating an FWF proposal as “Very Good” (the proposal being among the top 15% world-wide) and giving some recommendations to improve the proposal actually means “REJECT”. And it seems way too easy for our community to identify and state few such recommendations in the reviews with the consequence that lot of our proposals get rejected.

So, reviewers of Austrian FWF proposals, how can YOU help us?

1. Forget FWF’s rating scale for funding recommendation

Note, in the FWF’s stand-alone program, between 25% and 30% of the proposals are accepted by FWF. However, if you rate a proposal only with “Very Good” it most likely will be rejected because too many other proposals are rated as “Excellent”.

Therefore, forget the FWF guideline for rating a proposal. Instead use the following guideline:

  • if you think the proposal is in the top 25%-30% and should be funded, rate it as “Excellent”. Furthermore, provide several strong arguments why the proposed research must be funded and refrain from reporting minor issues that only will result in rejecting an otherwise strong proposal.
  • if you think the proposal is good but not good enough to receive the funding, point out the issues and rate it as “Very Good”, “Good” or “Average”. As a consequence, FWF will reject this proposal and ask the authors to revise and resubmit it.
  • if you find several strong points against the proposal, you might rate it as “Poor”. The proposal might still get a chance for a major revision and resubmission. But, it can also happen that resubmission of these proposals will be blocked for 12 months.

2. Minor/Major Revision = “REJECT”

As mentioned before, if your review reveals minor issues (even just a few), this means that the final decision will be “REJECT” with a recommendation to revise and resubmit the proposal. Please refrain from pointing out such minor issues for a proposal that you would like to get funded.

Moreover note, while your minor issues and recommendations will help to improve the proposal, the resubmission means that the review process will start from scratch. Unlike for journal submissions, FWF’s review process will most likely NOT assign all the original reviewers to the resubmitted proposal - at least one reviewer will be replaced and often an additional reviewer will be added. This increases the odds that the proposal will be assigned to new reviewers that receive the response letter but miss the context and information of the minor issues and recommendations from the previous reviews. Furthermore, having more reviewers reviewing a proposal increases the odds that one reviewer reports some minor issues that help to reject the proposal.

3. Only rating a proposal as “Excellent” != “ACCEPT”

Note, as a reviewer of an FWF proposal you have to fight for the proposal to get accepted. Only rating it as “Excellent” might not be sufficient to get a proposal accepted. You need to provide strong and convincing arguments in your reviews that the proposal must be funded. The FWF Board will this.

Please keep these three recommendations in mind when you review a grant proposal for FWF. You will greatly help our Computer Science community in Austria to receive funding for project proposals that really deserve it.