I’ve been on the Australian Research Council (ARC) College of Experts now for a little over two and a half years. It has been a time-consuming, yet insightful experience. Without attempting to breach all the confidentiality agreements I signed when I joined up, I would like to explain a few of the internal machinations that go on behind the scenes once a grant application is submitted.
Given that academics spend A LOT of (i.e., way too much) time writing research grants, I think it’s essential to understand not only how to maximise your probability of success (see this post for some generic tips), but also how your grant is treated once you submit it. I’ve heard (and been responsible for myself) many unhappy gripes from colleagues about the ARC over the years, which appear to have increased over the last five year in particular.
There are certainly some very good reasons to be upset about the research-grant environment in Australia. While I will restrict this post to issues concerning the ARC because that’s what I know best, I gather that many of the same issues plague other national agencies, such as the National Health and Medical Research Council (NHMRC). But to dispel the suspicion that the ARC is just out to make our lives hell, I’m going to provide a list of my experiences on what I think they do exceptionally well. I’m definitely not taking sides here, because after the list of pros, I’ll provide a detailed list of cons and some ways I think the ARC can move forward.
Impartiality
The ARC is very, very good at avoiding bias in the assessment process. Even if some potential bias does manage to creep in, the ARC is also extremely efficient at identifying and removing it. First, all assigned ‘carriages’ (College Experts) assigned to grants cannot work at the same institution as the applicants, they cannot have published with any of the applicants, nor can they have any other association with them. All potential conflicts of interest are declared and dealt with immediately up front.
Second, carriages cannot assign assessors with any of the aforementioned conflicts of interest given restrictions in the online applications that we use to identify and assign suitable assessors.
Third, during the actual deliberations, anyone who has any perceived conflict of interest must ‘leave the room’ (done in Zoom these days), nor can those people even see the grants under discussion for which they’ve been deemed conflicted.
Democracy
I have to admit that I’ve been involved in few processes that were more democratic than advisory panel meetings for deciding the fate of ARC grant applications. Any grant under discussion is not only pored over by the ‘detailed assessors’ (those are the comments to which you have to write a rejoinder), it is discussed in gory detail by the carriages. We not only read all of the detailed assessors’ reports and your rejoinder (after already having read the proposal itself many times), we also compare our scores among carriage members, discuss any scoring disparities, argue for or against various elements, and generally come to a consensus. For those grants under discussion, we also vote as an entire panel, with only majority ‘yes’ grants getting through.
Word of advice here — treat your rejoinder very seriously, and be succinct, polite, erudite, and topical. A good rejoinder can make or break any application.
Equity
It might seem to some applicants that we ignore things like career breaks, unfair assessments, and in-kind contributions, but I can assure you that we don’t. If someone has a genuinely good reason for a career break (e.g., children, caring for others, industry employment, COVID, moving between labs, …), we most definitely take that into account when assessing the quality of the investigator(s).
We are also very good at picking out nasty assessors who are unreasonably harsh, or who have provided very disparate scores from the other assessors. We often disregard low scores from such assessors. We also note when the comments provided to the applicant do not align with their scores, and adjust accordingly.
Disciplinary agnosticism
Some people might think that if there isn’t anyone on the panel who fits in the applicants’ specialist field exactly, that they somehow have a disadvantage. Untrue. We have such a diversity of panelists that there is always enough broad-scale expertise to give every field a fair go. I don’t know how many grants way outside my field that I have voted to support because of the careful explanation and championing by panel members acting as carriages for them.
So, those are some of the great things about the ARC grant-assessment process. Now here’s a list of many things that I think can be improved.
The piggy bank
The amount of research money available for a high-income nation like Australia is pitiful compared to most other comparable countries, and things have been getting worse. That means only a small fraction of applicants is ever successful. For the prosperity and resilience of the nation, we need to put much more emphasis on funding research in this country.
Interference
Political interference in the assessment of grants has been a big issue lately. I won’t comment on that much in this post because you can read all about it here and here. Suffice it to say that any acting minister should be banned from interfering with grants unless (and only unless) they breach ethics, laws, or national security. Just because some meathead minister doesn’t value the topic under investigation after it has been ratified by peers and College Experts, it should not give her or him any right to void the grant.
Workload
There is no question that there is a huge amount of wasted effort on both sides of the grant-application fence: writing AND assessing. With most applications never getting over the line, it is a complete mystery to me why the ARC doesn’t have an Expression of Interest process for all grant schemes. I used to sit on a Royal Society of New Zealand Marsden Fund Panel where we first ranked Expressions of Interest submitted by the applicant (a single page description + short CV), and then invited the top-ranked applicants to submit a full proposal. This ‘proof of concept’ phase is so efficient because it saves most people all the trouble of writing the damn thing, as well as all those hundreds or thousands of assessments, re-assessments, discussions, etc. necessary for the decision to fund or not.
There are no schemes more in need of the Expression of Interest than the fellowships (e.g., Discovery Early Career Researcher Awards, Future Fellowships, etc.). These tend to have the lowest success rates, and as a panelist, it is clear from the outset who has a chance and who doesn’t in terms of investigator quality and capacity.
Some could argue that adding an Expression of Interest phase would promote many more people submitting them because they require so little effort. Bollocks, I say. It is far more difficult to write a succinct, engaging, clear, and concise one-pager than a waffling 10-page grant. If people just winged it, it would actually make my life as an assessor all the easier because it would be even clearer where they should be ranked.
Feedback
Everyone complains about the opaque feedback they receive from the ARC when their grants don’t get funded. I couldn’t agree more. Where do I stand relative to the other applications/applicants? Where did I fall down? Where can I improve most?
There are many reasons why the feedback is so unhelpful, of which the massive effort required to provide it (see previous quibble) and the disparity in scoring versus comments that we receive from detailed assessors are the main.
That said, if we had a better method to standardise our scoring, as well as a standardised set of feedback comments that we could easily slip into the relevant sections, I think it would improve the entire process for everyone’s benefit.
It’s important to acknowledge that many of these components are provided regularly to the senior administration at the ARC by the College of Experts members. Great cogs sometimes turn slowly, but the messages are getting through, and change can happen. The more we insist on such changes as a community, while simultaneously acknowledging the things the ARC does well, the closer we’ll get to a great and fair granting system.