nunosempere.github.io/ESPR-Evaluation/Questionnaire-stable.md

Index:

Intro:
- Prisoner's dilemma
- Demographics.

Common endgoals:
- Life satisfaction 
- Social

Epistemic rationality. Less sensitive. 
- Calibration
- Thinking probabilistically
(- Changing your mind)
(- Noticing cognitive biases.)

Instrumental strategies
- Communication. NVC.
- Goals / clarity about goals.
- Internal design, introspections, emotions
	emotional stability.
- Habits and self modification
- Trying out new things
- Communication
- Projects
- SPORT!

Interest in. Less sensitive.
- Effective Altruism
- Existential risk
	- AI safety

## Introductory questions

What is the purpose of this study? The purpose of this study is to estimate the impact of ESPR. For this purpose, the answers to a baseline questionnaire (this one) from both participants and applicants which could have been participants will be compared with the answers from both groups to an followup questionnaire (after the camp has taken place). 
Who is sponsoring this study? The Effective Altruism Funds, a program under the Center for Effective Altruism.
Who is doing the study? Nuño Sempere. For any questions, comments or complaints, I can be reached at nuno@espr-camp.org
What kinds of questions will be asked? Questions about factors in which ESPR could have an effect, as well as questions about cofounders, like wealth or gender. 
How long will it take? [estimate]
Are my answers confidential? Yes. Your answers will never be used in any way that would identify you. They will be combined with answers from other respondents to make a statistical report. Your email will only be used to send you the followup survey, and will not be made public.
What do I need to know before taking this survey? You need to know that 
Do I get any compensation? Yes, and the exact payoff depends on a prisonner's dilemma! (see the first question).

Do you consent to having your survey responses be used for research purposes?

Do you consent to having your answers added to a public dataset?

Do you promise to give accurate answers, to the best of your ability?

## Prisoner's dilemma with real payoffs.
Reasoning for the question: Studies show that beginning with an stimulating question increases participation and data quality. We also want to give a small incentive to people who answer this questionnaire, and we think that this is a cool way to do it.

Are you familiar with the prisoner's dilemma game?

-> Yes.
-> No.
Read this: https://en.wikipedia.org/wiki/Prisoner%27s_dilemma

You will be randomly matched to another prospective ESPR participant in a prisoner's dilemma with the following real payoffs:
- Cooperate/Cooperate: $10 / $10 (if both players cooperate, both receive $10.
- Cooperate/Defect: $0/$15 (if one player defects, he receives $15, and the other gets nothing)
- Defect/Cooperate: $15/$0 
- Defect/Defect. $5/$5 (if both players defect, both get $5)
Do you choose to defect or to cooperate?

What is your email?
Your email will be needed to send you the payoff, and for the followup survey.


(an unconditional gift of $5 was twice as effective as a conditional offer of $50 in getting people to fill out surveys
	Oh, come on.
)

## Demographics
- Reasoning for this question: We want to distinguish between the effects of ESPR and the effects of growing up. Though we don't expect it to, we would also like to know if gender makes any difference in the impact of our program.

What is your date of birth? (Month and Year)
What is your gender?
- Male.
- Female.
- Other.
- Don't want to answer.

Growing up, would you characterize your family as working class, lower middle class, middle class, upper middle class, or well off?
The reasoning for this inexact question is:
- We don't feel comfortable asking detailed income information.
- But we still want to get a modicum of information on the topic.

[Alternative:
We understand that in some cases, the words income and family might not be well defined. If that is the case, use your best judgement to resolve any issues that might come up. That being said, (before/after) taxes, what is your family's combined income?
- Problems with the question: Definition of income, definition of family.
- It also takes too much time (potentially >10 mins) to answer. The applicant will probably not know the answer and have to ask his parents, who might not want to answer.
]

## Probabilities and Calibration
Consider a scale from 0 to 10, where 8,9 and 10 indicate "to great extent"; 0,1, and 2 "to a very small extent"; and 3 through 7 are in between. To what extent are you comfortable using probabilities and probabilistic reasoning?

Without taking into account this questionnaire or anything related to ESPR, when was the last time you can remember reasoning in terms of probabilities?

When was the last time when thinking in terms of probabilities changed what you were going to do?

Consider a scale from 0 to 10, where 8,9 and 10 indicate "to great extent"; 0,1, and 2 "to a very small extent"; and 3 through 7 are in between. To what extent do you feel that probabilities and probabilistic reasoning are useful tools in your repertoire?

### Calibration
This part asks you to answer some questions, and then pull some probabilities out of your ass.

1. What date is it today?
2. Without checking a source, estimate your subjective probability that the answer you just gave is correct

3. Which is heavier, a virus or a prion?

4. Without checking a source, estimate your subjective probability that the answer you just gave is correct.

5. I'm thinking of a number between one and ten, what is it?

6. Without checking a source, estimate your subjective probability that the answer you just gave is correct.

7. What year was the fast food chain "Dairy Queen" founded? (Within five years)

8. Without checking a source, estimate your subjective probability that the answer you just gave is correct.

9. Alexander Hamilton appears on how many distinct denominations of US Currency?

10. Without checking a source, estimate your subjective probability that the answer you just gave is correct.

11. What's the diameter of a standard soccerball, in cm within 2?

12. Without checking a source, estimate your subjective probability that the answer you just gave is correct.S

13. What probability do you assign to Donald Trump winning a second presidential election? 

14. What is your subjective probability that there is a god, defined as a supernatural intelligent entity who created the universe?

15. What probability do you subjectively assign to any of humankind's revealed religions being more or less correct?

16. At the beginning of this survey, you took part in a prisoner's dilemma. You will be matched with a randomly selected prospective participant. What probability do you assign to that other participant defecting?

17. Other participants are also trying to answer this survey. What probability do you assign to yourself having a better Brier score than 50% of participants (not including this question)?

18. What probability do you assign to Donald Trump, the current president of the USA, winning the 2022 presidential elections?


## Life satisfaction.

Consider a scale from 0 to 10, where 8,9 and 10 indicate great satisfaction; 0,1, and 2 very little satisfaction; and 3 through 7 are in between. How satisfied are you with your life as a whole?

Consider a scale from 0 to 10, where 8,9 and 10 indicate "to great extent"; 0,1, and 2 "to a very small extent"; and 3 through 7 are in between. To what extent do you feel like your life is stuck?

In same scale as before, how satisfied are you with the following domains?
- romantic relationships
- work / school / career
- friendships / non-romantic social life

Subjective Happiness Scale.
http://sonjalyubomirsky.com/subjective-happiness-scale-shs/

1.  In general, I consider myself:

not a very happy person     
1            2            3            4            5            6          7    
a very happy person

2.  Compared with most of my peers, I consider myself:

less happy
1            2            3            4            5            6            7    more happy

3.  Some people are generally very happy.  They enjoy life regardless of what is going on, getting the most out of everything.  To what extent does this characterization describe you?

not at all    
1            2            3            4            5            6            7    a great deal

4.  Some people are generally not very happy.  Although they are not depressed, they never seem as happy as they might be.  To what extent does this characterization describe you?

not at  all   
1            2            3            4            5            6            7    a great deal

## Social.

Consider a scale from 0 to 10, where 8,9 and 10 indicate great satisfaction; 0,1, and 2 very little satisfaction; and 3 through 7 are in between. Excluding romantic relationships, how satisfied are you with your social life?

In that same scale: how satisfied are you with your romantic life?
(These are people 15-18. Probably not winning at romance. Maybe exclude this question).

Some common ways to generate an estimate of the number of people you socially interacted with in the past week are:
- Episodic enumeration: Count the amount of people you can recall, and report that number. 
- Availability heuristic: As a first order estimate, equate frequency with ease of recalling: i.e., report a higher number if you can remember many examples in a short amount of time. 
- Automatic or moral estimation: Resort to some innate or normative sense of frequency, for example depending on whether you think of yourself as a very social person.

Studies show that the most accurate strategy is episodic enummeration. Do you promise to use this method throughout the survey?  -> Yes

In the past week, how many people can you recall socially interacting with?
How many people in which you'd be willing to confide about something personal can you come up with?
How many people can you name who would let you crash at their place?

## Emotions
When you feel emotions, do they mostly help or hinder you in pursuing your goals?

- General self-efficacy scale (Chen, 2001) 

For the following questions, do you agree or disagree?
1. I will be able to achieve most of the goals that I have set for myself.
2. When facing difficult tasks, I am certain that I will accomplish them.
3. In general, I think that I can obtain outcomes that are important to me.
4. I believe I can succeed at most any endeavor to which I set my mind.
5. I will be able to successfully overcome many challenges.
6. I am confident that I can perform effectively on many different tasks.
7. Compared to other people, I can do most tasks very well.
8. Even when things are tough, I can perform quite well.

- Subjective happiness Scale (Lyubomirsky, 1997)

1. In general I consider myself
    not a very happy person :   1   2   3   4   5   6   7   a very happy person
2. Compared to most of my peers, I consider myself
    less happy:   1   2   3   4   5   6   7   more happy
3. Some people are generally very happy. They enjoy life regardless of what is going on, getting the most out of everything. To what extent does this characterization describe you?
    not at all:   1   2   3   4   5   6   7   a great deal
4. Some people are generally not very happy. Although they are not depressed, they never seem as happy as they might be. To what extent does this characterization describe you?
    not at all :   1   2   3   4   5   6   7   a great deal.

Emotional Maturity Scale (Schutte, 1998). 33 items. 

## Techniques
Can you think of any changes that you’ve made in the past month to your daily routines or habits in order to make things go better? These can be tiny changes (e.g., adjusted the curtains on my bedroom window so that less light comes in while I’m sleeping) or large ones. Spend about 60 seconds recalling as many examples of these kinds of changes as you can and listing them here. (If you want to skip this question, leave it blank. If you spend the 60 seconds and no specific examples come to mind, write "none.")
- is 60s enough time?

How many changes did you write under the above question?

Even capable people make stupid mistakes, f.ex. forgetting a computer charger at home before an international flight. Take a moment to think about the last one you made. What was it?

Did you take any measures to avoid making that mistake or class of mistakes in the future?

(If so), which measures did you take?

This last week, how many hours did you do sports, or other physically streneous activities?

Write your answer to the following questions in the format "day/month/year" to the best of your abilities. Note that this can mean that the day you give is approximate. 

Take 60 seconds to recall the last time you did something you thought had a very low chance of succeeding. What was it?
- Option for "I don't 

When was it? (Give your best estimate in the form day/month/year.)
- Free form answers in the pre-test, from which we can maybe get a couple of categories.
- Option of: I don't rember.

Take 60 seconds to recall the last time you tried out something new; something which you hadn't done before. What was it?

When was it? (Give your best estimate in the form day/month/year.)
- Free form answers in the pre-test, from which we can maybe get a couple of categories.
- Option of: I don't rember.

When was the last time you *read or heard about* another plausible-seeming technique or approach for being more rational / more productive / happier / having better social relationships / having more accurate beliefs / etc.?

When was the last time you *tried out*  another plausible-seeming technique or approach for being more rational / more productive / happier / having better social relationships / having more accurate beliefs / etc.?

When was the last time you found out about another plausible-seeming technique or approach which *successfully helps you at* being more rational / more productive / happier / having better social relationships / having more accurate beliefs / etc.?

How many hours did you spend productive work yesterday (or on your most recent workday)?
(maybe ask Dan for the exact questions?)

In a scale from 0% efficiency, where you are not getting anything done, to 100% efficiency, where you are working as productively as you're capable on, what was the efficiency of the time you spent working yesterday (or on your most recent workday)?

For how many of those hours did you feel physically motivated to do the task at hand, i.e., the thing that you were doing was the thing that you felt like doing at that moment?

What was the last proyect you undertook? Projects can be things like writting a program, seducing a romantic interest, writting a book/blog, researching something in depth, etc. If you have many going on at the same time, pick one. Do not change it afterwards.

Do you feel that this project is broadly representative of the general way you go about undertaking projects?
- No, I'm usually much less efective/capable/etc.
- No, I'm usually somewhat less efective/capable/etc.
- Yes, this is roughly typical for me.
- No, I'm usually somewhat more efective/capable/etc.
- No, I'm usually much more efective/capable/etc.

In general, when you undertake a project, do you plan out what specific tasks you will need to do to accomplish it?
- 1-6 scale from “Almost never” to “Almost always”.

For the project you mentioned above, did you plan out what specific tasks you will need to do to accomplish it?
- Yes
- No

In general, when you undertake a project, do you try to think in advance about what obstacles I might face, and how I can get past them
- 1-6 scale from “Almost never” to “Almost always”.

For the project you mentioned above, did you seek out information about other people who have attempted similar projects to learn about what they did
- Yes
- No

In general, when you undertake a project, do you end up getting it done?
- 1-6 scale from “Almost never” to “Almost always”.

For the project you mentioned above, did you end up getting it done?
- Yes.
- No.
- Too soon to tell, but I suspect I will.
- Too soon to tell, but I suspect I won't.
- Too soon to tell, and I couldn't say whether I will or won't.

## Effective Altruism
Had you heard the term "Effective Altruism" before?
- Yes.
- No.
(Don't give a "maybe" option.)

(If no: skip everything)

(If yes:)
Without checking a source, how would you define the concept?

In a scale from 0 to 10, where 8,9 and 10 indicate a very positive opinion; 0,1, and 2 a very negative opinion; and 3 through 7 are in between: What's your overall opinion of Effective Altruism?

Have you previously heard of any of the following organizations? (select all that apply)
- GiveWell
- Center for Effective Altruism
- 80,000h
- EvidenceAction
- Against Malaria Foundation
- add previous GW top charities

(if at least one:)
Have you made any donations to any of those organizations?
- Yes
- No
- Not personally, but I'm nonetheless causally responsable for a donation being made.

(If yes, or c):)
What amount?

(if at least 6 in opinion of EA:)
In the future, do you expect to donate to organizations which fall under the umbrella of Effective Altruism, such as the ones mentioned previously?
- Yes.
- No.

(If yes: Approximately, how much do you expect your total life donations to be?)

From 0% to 100%, how likely do you judge it that you will work for an EA organization?

## Existential risk.
Have you heard the term "existential risk" before?

(If no: skip everything).

(If yes) Without searching the internet, looking at Wikipedia, etc., how would you describe the concept?

Consider a scale from 0 to 10, where 8,9 and 10 are the highest importance; 0,1, and 2 are low importance; and 3 through 7 are in between. What number would you give to the importance of existential risk?

In that same 10 point intensity scale, how interested are you in it?

(If higher than 5) Have you read any papers, articles or books related to the topic?

(If yes) Can you mention some of them?

(If higher than 5) In percentage points, how likely do you judge it that your future career will in some way be related to existential risk?

( Does not apply to CFAR:
## Other instrumental goal:
- Are you aiming to go to university?
- Rank by admission. Do espr people go to more prestigious universities?
)