In relation to scholarships how do you "score" essays and letters of recommendation?
We receive over approximately 500 scholarship applications each year and each application has three letters of recommendations and an essay question that relates to each individual scholarship (we have approx 40 scholarships total). Currently we have a scholarship committee consisting of board members and volunteers that gives a score of 1-5 for the letters of recommendation and a score of 1-5 for the essays. Each volunteer reviews each application. We are trying to figure out a way to either divide the review process and/or automate the scoring of the letters of recommendation/essays. Does anyone have any input on how to best achieve this?
As a side note, some of our applicants can apply for multiple scholarships so if we divide essays by applicants two different people might review different applicants for the scholarship versus if we divide by scholarship it might lead us to have two individuals have to read the same applicants information. Sorry if this sounds confusing.
Comments
-
Here is how I handle essays and recommendations:
2 -
We also do not do letters of recommendation because they're all always raving reviews of the applicant. We implemented a "recommendation rubric" using third party email feature and that works well.
5 -
One way to cut down on the work is to lessen the number of recommendations you require. We found that whether it was one, two, or three, all the recommendations were glowing and didn't really help us see much of a distinction between applicants to determine who would be a better fit. Their personal essay was more informative. So we dropped the requirement to one recommendation. Like the responder above, we also use the third party form feature with quantitative and qualitative questions so that each applicant is being scored apples-to-apples.
One lesson learned this year: If we only require one recommendation, some recommenders don't respond to the emails (junk mail, busy, etc.) so that student may be without any recommendations. This will impact scoring. So I think next year we'll ask for 2 emails in the 3rd party form option, but only score for one. That way if one reference doesn't submit something, there's another to fall back on.
Am I reading correctly that you have each volunteer read all 500 applications? We divide ours up by scholarship, with each application being read by 2 people. Then we average the scores. X and Y reads all 7 applicants for the VVV Scholarship, then X and F read 20 of the GGG Scholarship. We try to make it so that no one is reading more than 25 or so applications.
2 -
Our committee reviews 1 essay and 1 short answer statement - similar to Carissa, our committee does not rate Letters of Recommendation anymore and instead get ratings form a questionnaire. The questions reflect leadership, community service, and work ethic - three qualities referenced to in most of our scholarship eligibilities. These statements are then scored based on their selection of 1-5 and factored into full scoring. (Attached)
I hope this helps, I'm struggling to streamline our process as well because it is a lot of work on volunteers!
2 -
Hi!
Currently, we have separate groups of reviewers for each scholarship. This lessens the load on reviewers and allows them to focus on one set of criteria. As some of our scholarships grow in popularity, we have considered splitting the applications among reviewers in a similar manner as we do for grants. Here is our general process for splitting grant reviews:
- We split reviews if one committee/specific opportunity received 30+ applications.
- We randomly assign applications so each application receives at least 3 reviews and each committee member reviews fewer than 30 applications (sometimes this has involved bringing in more volunteer reviewers). Each reviewer should have a unique set of applications to review.
- We utilize a rubric, targeted questions, and a reviewer guide to ensure consistent scoring.
I hope this helps! Please let me know if I can clarify anything!
2 -
Thank you all for your responses. This was all very helpful information!
1
