Getting Applicants

Because in the end, you need hackers for a hackathon.

Creating the application

Following Berkeley, Stanford, etc., we decided to admit hackers based on their resume and their answers to a couple of essay questions, and we aped most of the questions from these hackathons' applications. That part was fairly easy. We did spend a lot of time thinking about our admission criteria though because there was some tension in the interests of the different stakeholders here.

Our corporate sponsors mostly wanted to see employable college seniors, while it was in the best interest of Minerva to send lots of (freshmen) Minervans, and we thought the hackathon would be most enjoyable if we could get a diverse group of participants. Meanwhile, the venue could only hold so many people.We had to agree to admit all GAer applications in exchange for using the venue.

After extensive deliberation, we decided that our "values" were really inseparable from our goals and guiding principles. In the end, what we were trying to do was to bring people together to build cool projects as an end in and of itself---as opposed to some other hackathons, who choose a specific cause such as diversity or climate change. This is because we all have a firm faith in the positive power of technology, and we believe that it is in the interest of the world to forward technology in the abstract. [1]

This led us to choose the admissions criteria that (1) allowed the hackathon to exist, i.e., admitting GAers, and (2) made for an environment in which we believed the best projects would be born. This second point, for us, meant slicing the applicants into subpopulations by skill-domain and experience-level and selecting the most qualified and motivated from each, with the goal of creating a diverse but passionate community of hackers. It also led us to reject some admissions criteria, such as automatically admitting Minervans, and some possible process options, such as giving every application an exactly equal chance of acceptance.

Our application questions

• General info

  • Name

  • Gender

  • Team entry

• Background

  • School

    • What is the highest level of education you have achieved?

    • Are you a Minerva student?

  • Work

    • Which best describes your area of expertise? (Coding, Design, Business)

    • Are you professionally employed in this area?

    • LinkedIn, Github, Resume

• Long answer

  • Describe a project you have worked on. What are you proud of? (500 w?)

Running the application process

Although we never launched applications, a first thought here was to try to automate the process somehow, given the daunting task of evaluating the applications by hand. Unfortunately, we failed to acquire the application data from last year, making any kind of machine learning impossible. Our next best idea was for a rule-based alternative---stratifying the applications by domain knowledge and experience and then accepting those with the highest skill and motivation.

The goal here was not to have the computer make every decision but to eliminate to obvious decisions (hopefully about half) from the pool of those to be assessed. So, we planned to designed a series of metrics, likely including essay length, number of grammar mistakes (using a grammar api), number of empty application fields, etc. As a heuristic, we decided to give each metric equal weight. So, to get an application's total score, we would estimate the z-score of the application on each metric and then simply add these together.

We expected to admit about half of applicants, and we were quite sure we would admit at least 25%. So, we would generate a confidence interval around the "total score" or each applicant, and, if we could be 95% sure that an applicant was in the top or bottom 25% of total score, we would auto accept or reject.

Technical part

The math to do this is actually not that complicated. First, we need to adjust the features. To do this, we first make them independent with principal component analysis, keeping every result vector. (To prevent unpredictable variations in admissions criteria, we would do this once after about 30 applications and keep those features for thes rest of the analysis.) Once these are independent, the size of the confidence interval for total score. Then we normalize these with min-max scaling on each feature.

After that, since they are independent, we can calculate the size of a 95% confidence interval for the sum of the features by taking the Euclidean mean the sizes of the 95% confidence intervals for each feature. To determine whether to accept, simply subtract this average from the sum of the applicant's adjusted scores and check whether this difference is greater than the 75% quantile of sums of adjusted scores for all applicants. To determine whether to decline, add this average to the sum and see if this is less than the 25% quantile of sums of adjusted scores. [2]

Statistically, this isn't perfect because it requires doing the PCA once and keeping those vectors, which, since they are only based on a sample, will likely not be perfectly independent in the total population. However, basing these on 30 applications, we will probably be close enough for decent results.

Getting the participants

We really had no idea what to expect with this. Some team members thought that posting the link on Eventbrite would be enough to overwhelm us with applications. Some thought we would be hanging up posters USF the day before when no one applied. The one thing we really learned was the GA was very eager for us to accept their students, which suggests that we would have been fine in the end. But, we really don't have anything useful here to day.

Here are some ideas we had for promoting the hackathon if it came to that:

• Eventbrite

• Meetup

• Posters at USF, Berkeley, etc.

• Social media marketing (Fiverr?)

• Word of mouth

One more thing

This is an interesting resource that explains how TreeHacks, one of the Stanford hackathons, approaches this problem. https://medium.com/@hackwithtrees/how-we-choose-our-hackers-treehacks-admissions-2019-d40dd98eeb80. If you don't like how we approached this, or you want more perspective, we recommend checking it out.

HC footnotes:

[1] #purpose: We looked deep to identify our guiding values---advancing technology---and then we translated these into substantive decisions about how the hackathon application process should proceed.

[2] #confidenceintervals: We apply confidence intervals to a complex inference problem. We are careful with the assumptions behind our statistical methods, taking steps to make sure features are independent and commensurable.