Important information for reviewers of FWF stand-alone proposals

Should you receive an invitation from FWF to review a stand-alone project proposal, please read this first. Note, this information is based on several discussions with other Austrian researchers in Computer Science and my personal experience with FWF. If you have comments/feedback, feel free to send them to me via email.

The situation in Austria

In Austria, it is difficult to acquire funding for fundamental research from FWF, Austria’s central funding organization for basic research. The main reason is simply lack of money. And it is even more difficult for applicants in the areas of Computer Science, among them Software Engineering. Please note, in Austria we compete with ALL disciplines.

Looking at the acceptance rates of FWF stand-alone project proposals during the past two years, you can spot quite some differences between the various disciplines. For instance, in Mathematics 37.9% (2020) and 34.9% (2021) of the submitted proposals got accepted. In Physics and Astronomy the acceptance rates are 36.1% (2020) and 34.9% (2021). In Computer Science only 20.4% (2020) and 26.7% (2021) got accepted. While the acceptance rate in Computer Science improved from 2020 to 2021, it is still significantly lower than in Mathematics and Physics and Astronomy. For more details see the statistics published by FWF in their Dashboard.

Honestly, I do not think that Austrian researchers in Software Engineering write worse proposals than the researchers in other disciplines. Instead, I believe the main problem is that our Software Engineering community is TOO critical compared to other communities. Other communities seem to know that rating an FWF proposal as “Very Good” (the proposal being among the top 15% world-wide according to the FWF reviewing guideline) and giving some recommendations to improve the proposal actually means “REJECT”. And it seems way too easy for our community to identify and state few such recommendations in the reviews with the consequence that too many of our proposals get rejected.

So, reviewers of Austrian FWF proposals, how can YOU help us?

1. Forget FWF’s rating scale for funding recommendation

Note, in the FWF’s stand-alone program, between 25% and 30% of the proposals are accepted by FWF. However, if you rate a proposal only with “Very Good” it most likely will be rejected because too many other proposals from other disciplines are rated as “Excellent”.

Therefore, forget the FWF reviewing guideline for rating a proposal. Instead you might use the following guideline:

  • if you think the proposal is in the top 25%-30% and should be funded, rate it as “Excellent”. Furthermore, provide strong arguments why the proposed research must be funded and refrain from reporting minor issues that only will result in rejecting an otherwise strong proposal.
  • if you think the proposal is good but not good enough to receive the funding, point out the issues and rate it as “Very Good”, “Good” or “Average”. As a consequence, FWF will reject this proposal and ask the authors to revise and resubmit it.
  • if you find several strong points against the proposal, you might rate it as “Poor”. The proposal might still get a chance for a major revision and resubmission. But, it can also happen that resubmission of the proposal will be blocked for 12 months (which is OK because the authors should invest enough time to thoroughly address the major issues).

2. Rating a proposal as “Minor/Major Revision” means “REJECT”

As mentioned before, if your review reveals minor issues (even just a few), this means that the final decision will be “REJECT” with a recommendation to revise and resubmit the proposal. Please refrain from pointing out such minor issues for a proposal that you would like to get funded.

Moreover note, while your minor issues and recommendations will help to improve the proposal, the resubmission means that the review process will start from scratch. Unlike for journal submissions, FWF will most likely NOT assign all the original reviewers to the resubmitted proposal - at least one reviewer will be replaced and often an additional reviewer will be added. This increases the odds that the proposal will be assigned to new reviewers that receive the response letter but miss the context and information of the minor issues and recommendations from the previous reviews. Furthermore, having more reviewers reviewing a proposal increases the odds that one reviewer reports some minor issues that only help to reject the proposal again.

3. Only rating a proposal as “Excellent” does not mean “ACCEPT”

Note, as a reviewer of an FWF proposal that you want to get accepted you have to fight for the proposal. Only rating it as “Excellent” might not be sufficient to get a proposal accepted. You need to provide strong and convincing arguments in your reviews why the proposal must be funded.

Please keep these three recommendations in mind when you review a grant proposal for FWF. You will greatly help our Computer Science community in Austria to receive funding for project proposals that really deserve it.