The Million Dollar Survey
If I were going to raise $1 million for charity, this is how I would do it.
When I published a short survey on Riskology.co back in October, I asked readers to answer questions to help me learn about the psychological effects of generosity. I had no idea what was going to happen.
Likewise, readers who took the survey had no idea they were actually the subjects of an experiment I was running in an attempt to separate them from their money.
But it was all for a good cause, so my conscience remains clear.
$2,300 and more than 5,000 data points later, I’ve learned some fascinating lessons about asking people for money that I most certainly could not have predicted before running the experiment.
If you copy this experiment for yourself, you’ll likely do much better.
The initial focus of the research and lessons you’re about to read were to shed some light on the fundraising process for small, non-profit organizations who are incredibly passionate about what they do, but notoriously horrible at getting anyone else to care or give them money to keep the doors open.
But the results I uncovered are—I think—far-reaching and can be applied to a number of situations where money is exchanged for, essentially, nothing of tangible value.
- Did your car run out of gas on the side of the road, and you need $20 to get home? Read on.
- Want to test your skills panhandling in the public square? You’ll probably learn something here.
- Taking donations for your Amazonian butterfly enthusiast website? Pay attention.
- Launching a product and trying to figure out what sales copy will work the best?
You get the picture. Let us begin. But first…
How 18-Year-Old Tyler Lost $40 and Did NOT Get Laid
I was sitting in my dorm, minding my own business—just like I did every night back when I had no friends—when there was a knock on the door.
“Who’s there?”
“It’s Brianne and Carol. There’s a party up the street tonight, do you want to come?”
[Names changed to protect the guilty because I don’t remember them.]
I had no idea who these girls were, but when you’re 18, in a brand new city, and don’t have any friends, you do not ignore two girls knocking on the door.
So, I opened. That was my first mistake.
“Do I know you?”
“Nope, but we know you. You’re the cute guy we see on campus all the time.”
“Really?” I asked… suspiciously.
“Yep. And there’s a party over in Goose Hollow tonight if you want to come with us. Are you doing anything? We’re bored; can we come in?”
Tyler’s internal monologue: “Are you kidding me? Of course I want to go? And can you come in? Damn right you can! Let’s do this! I think I saw this in a movie one time!”
Tyler’s external dialogue: “Umm…okay.”
The girls came in and sat down on my bed. Brianne must have noticed the Shaun Murray poster on my wall because she immediately started asking me questions about wakeboarding.
“Oh, do you wakeboard? When was the last time you went out? Can you do a flip?”
“Oh my God, these girls are even into wakeboarding,” I think to myself. I am set. What I don’t notice is that the questions get more and more specific as the conversation progresses.
“Do you, like, know all the guys on the pro-circuit? Do you keep up with everything in the industry?”
I keep nodding and wiping drool away from my mouth as they barrage me with wakeboarding questions that allow me sound like a badass as I answer. “Yes, I know all those guys, and I love to keep up with everything going on. It’s so exciting.”
“Very cool! Do you read Wakeboarder Magazine?”
“Yep, the latest copy is on my bookshelf right there,” I pointed.
“Wow, that’s perfect! Carol and I actually sell subscriptions. Do you want to renew your subscription with us? I even have the form right her in my purse.
28-Year-Old Tyler Interjection: Holy shit! It’s a sales call! These girls are experts. The old Trick Him Into Thinking He’s Having an 3-way, Then Sell Him a Magazine Routine—classic!
“Um, I don’t know. I still have a few months left on my current one, and I might be moving soon, so I’d need to change my address and…”
“Oh, that’s all easy. This will just renew as soon as your current subscription expires, and you can change your address any time you like. Here’s the form. Fill it out.” The words spill out of her mouth so fast I can’t even find space to argue as she pushes the form into my lap.
I was backpedaling hard, but they’d hooked me. I’d just told them I was exactly the right person to buy a Wakeboarder Magazine subscription. How was I supposed to now tell them something different?
I gave in and filled out the form.
“Let me get my checkbook,” I resigned.
“Oh—we can’t take checks. Cash only. We only get our commission if we take cash.”
“Right. That makes perfect sense.”
28-Year-Old Tyler Interjection: That makes no f***ing sense!
The minute these two seductresses had my cash, they were gone. They got up, gave me the address to the party, told me they’d meet me there, and left.
I didn’t even realize I’d been completely bamboozled until later when I learned the address they gave me didn’t exist, and the middle-aged gentleman who answered the door at the closest house informed me that no, there was no party and I should probably get off his porch.
And I didn’t realize I’d been swindled until a few months later when my Wakeboarder Magazine subscription didn’t renew.
The Lesson in My Tragic Story
Ten years later, the first thing I still think when a cute girl approaches me is, “What are you trying to steal from me?”
But in ten years, I’ve also learned a thing or two about how to sell things and ask for money. And I can say, without a doubt, these girls were experts that I wanted to learn from.
If they weren’t good-for-nothing thieves, I’d call them geniuses. The way they worked and the questions they asked all but assured I was going to hand over some money.
- They used an unassuming opener to get their foot in the door (literally) without objections.
- They quickly picked up on something that was important to me.
- They asked a series of questions that lead me down their path, getting me to set up a line of logical consistency. “YES, I love wakeboarding. YES, I know all the big shots. YES, I keep up with the industry.”
- Once I reached the point of no return, they pounced, and I had no choice but to either break my logical consistency (very difficult), or shell out $40.
This same strategy is used all the time—as it should be—by expert fundraisers for important causes around the world.
For instance, if you’ve ever been walking down the street and had someone with a clipboard try to shake your hand and ask, “Do you have a moment for the environment/starving kids in Africa/homeless kittens/etc.,” then you know how awful it feels to say, “No, I don’t have a moment for that.”
But how do you, a fledgling, web-based non-profit/other doer-of-good harness these lessons?
- Can they be employed online where I don’t get to have a face-to-face conversation?
- Is there a way that works better than others to use them?
- Will I be able to pull this off even if I suck at selling things?
The answers are yes, yes, and yes. See what I did there?
The answer just may be in creating an online survey. Here are the results from my own.
The Temporary, Unadvertised Survey That Generated $2,304.10 for Charity
On October 23, 2012, I discreetly published a survey on Riskology.co and sent an email to my list of around 7,000 subscribers asking them if they’d take a short survey “about generosity” to help me collect data about the habits of generous people.
What participants didn’t know was that I’d actually published three surveys using different combinations of questions and installed a script on the site that would randomly show only one of the three variations of each survey taker.
What this allowed me to do was split respondents into three distinct groups—without their knowledge (to preserve the accuracy of results)—in order to study the responses to each survey and make better educated guesses about:
- Which types of questions would encourage the most people to donate and should be used often.
- Which combination of questions would elicit the most donations.
- Which types of questions, in different scenarios, would discourage donations and should be avoided.
By the time I closed the survey, $2,304.10 had been raised, but with the promise of much more. The following is what I learned.
Survey Design: How I Put This Test Together so that You Can Better Understand The Results
When thinking about what types of questions to ask, I decided to focus on two distinct categories:
- Personality. These were questions that forced the survey taker to consider how they think about themselves. The goal was to present questions that lead the respondent to consistently answer that they were a “generous person.” This is the same logic trick that the magazine-selling girls played on me.
- Knowledge. These were questions that tested the respondent’s knowledge about a particular subject. The goal was to present information about a problem in the world that the respondent may not know of, triggering shock and the desire to help.
Important note: I would have loved to test a number of question types in different arrangements, but decided to test only two variations because I was not confident I could collect enough responses to make the results worth sharing. The more variations there are to a test, the more data you must collect to draw any kind of significant conclusion.
For all the surveys, I asked respondents to commit to making a one-time donation of $10 to One Girl, a non-profit organization that works to keep girls in Sierra Leone in school, an excellent organization run by a friend of mine.
This question was asked at the end of the survey after all other questions were answered. It appeared on a new page before submitting the survey so as not to alert respondents to that the survey would be asking them for a donation. A true sneak attack!
Why One Girl?
A valid question. And the only answer I can give is, “Why not?”
Arguments could be made for days about the best partner to work with for this study. In the end, though, it doesn’t really matter; I just needed somewhere to send the money. Riskology.co is not a non-profit org, so I simply picked a charity that I was familiar with and, in my opinion, does excellent work.
For a brief moment, I considered setting up the “Riskology.co Foundation” to allow me to collect the money and better control the experiment, but quickly scrapped that as a “very bad idea.”
When you use this information to solicit donations, you’ll direct contributions to yourself and, as a result, likely experience better results than I did because the people taking the survey will already be pre-conditioned to support the work they’re about to donate to.
The Survey Questions
For Survey #1, I focused on personality questions. These are the questions—in the exact form and order—that respondents saw them in when completing the survey:
| Would you describe yourself as a generous person? |
| The last time you did something for someone less fortunate, did it make you feel good or bad? |
| How do you tend to feel when you help someone in need? |
| Are people who give without expectation of return better than people who do not? |
| Is it good to help people less fortunate than you? |
| How would it make you feel to know that you made a positive difference in a child’s life? |
| TRUE OR FALSE: I have enough money to have a happy life. |
| TRUE OR FALSE: I have enough money to give some to worthy causes. |
| TRUE OR FALSE: When I donate money to charity, I feel good about myself. |
| Do you feel better when you help someone in need than when you buy something for yourself? |
For Survey #2, I focused on knowledge-based questions that raised awareness specific to the work that One Girl does in Sierra Leone. These are the questions—in the exact form and order—that respondents saw them when completing the survey:
| Did you know more than 60 million girls around the world don’t attend school? |
| Did you know 62% of uneducated girls in Sierra Leone get married before age 18? |
| Did you know a girl in Sierra Leone has only a 1 in 6 chance of attending high school? |
| Did you know an uneducated girl in Sierra Leone has a 50% chance of being sexually abused before turning 18? |
| Did you know 76% of women in Sierra Leone earn less than $2 per day? |
| Did you know that an educated girl in Africa is 3 times less likely to contract HIV/AIDs than an uneducated one? |
| Did you know that an educated girl in Africa will marry later than an uneducated girl? |
| Did you know that an educated girl in Africa will have fewer, healthier children than an uneducated girl? |
| Did you know that for each year of school that a girl in Africa completes, she will increase her income by up to 25%? |
| Did you know that an educated girl in Africa will re-invest 90% of her earnings back into her family? |
Survey #3 was a hybrid of the questions asked in the previous two. Five questions were chosen from each at random and combined to create this survey.
As you’ll see later on, the results of this survey were very interesting.
These are the questions—in the exact form and order—that respondents saw them when completing the survey:
| Would you describe yourself as a generous person? |
| The last time you did something for someone less fortunate, did it make you feel good or bad? |
| How do you tend to feel when you help someone in need? |
| Are people who give without expectation of return better than people who do not? |
| TRUE OR FALSE: When I donate money to charity, I feel good about myself. |
| Did you know that an educated girl in Africa is 3 times less likely to contract HIV/AIDs than an uneducated one? |
| Did you know that an educated girl in Africa will marry later than an uneducated girl? |
| Did you know that for each year of school that a girl in Africa completes, she will increase her income by up to 25%? |
| Did you know that an educated girl in Africa will re-invest 90% of her earnings back into her family? |
| Did you know a girl in Sierra Leone has only a 1 in 6 chance of attending high school? |
An astute survey designer, after reading these questions, would correctly point out that my surveys are filled with “bias language” and a lack of appropriate response options that would invalidate the results of the survey.
Remember that the goal of these surveys are not to correctly gauge answers to specific questions but to actively encourage people to make a donation to charity. The questions were specifically designed this way!
Onto the results.
How did people answer the questions?
From here on out, we’ll refer to Survey #1 as the “Personality Survey.” Survey #2 will be the “Knowledge Survey.” And Survey #3 will be the “Hybrid Survey.”
In all, 464 people responded to all the surveys.
Results of the Personality Survey
There were 186 respondents to the Personality Survey, and their responses look like this:
| Question | Yes/True/Good | No/False/Bad | N/A |
|---|---|---|---|
| Would you describe yourself as a generous person? | 86% | 14% | |
| The last time you did something for someone less fortunate, did it make you feel good or bad? | 88% | 1% | 11% |
| Are people who give without expectation of return better than people who do not? | 55% | 45% | |
| Is it good to help people less fortunate than you? | 98% | 2% | |
| How would it make you feel to know that you made a positive difference in a child’s life? | 4% | ||
| TRUE OR FALSE: I have enough money to have a happy life. | 74% | 25% | |
| TRUE OR FALSE: I have enough money to give some to worthy causes. | 80% | 20% | |
| TRUE OR FALSE: When I donate money to charity, I feel good about myself. | 80% | 19% | |
| Do you feel better when you help someone in need than when you buy something for yourself? | 67% | 32% |
| Question | Happy | Sad | Angry | Excited | N/A |
|---|---|---|---|---|---|
| How do you tend to feel when you help someone in need? | 72% | 5% | 1% | 17% | 6% |
| How would it make you feel to know that you made a positive difference in a child’s life? | 73% | 0% | 0% | 23% | 4% |
Based on the results of this survey, most respondents feel a certain way about themselves:
- They are generous people.
- They have the ability to give.
- They associate giving with good feelings in themselves.
Results of the Knowledge Survey
There were 223 respondents to the Knowledge Survey, and their responses looked like this:
| Question | Yes | No |
|---|---|---|
| Did you know more than 60 million girls around the world don’t attend school? | 26% | 73% |
| Did you know 62% of uneducated girls in Sierra Leone get married before age 18? | 15% | 85% |
| Did you know a girl in Sierra Leone has only a 1 in 6 chance of attending high school? | 13% | 86% |
| Did you know an uneducated girl in Sierra Leone has a 50% chance of being sexually abused before turning 18? | 21% | 78% |
| Did you know 76% of women in Sierra Leone earn less than $2 per day? | 16% | 83% |
| Did you know that an educated girl in Africa is 3 times less likely to contract HIV/AIDs than an uneducated one? | 39% | 60% |
| Did you know that an educated girl in Africa will marry later than an uneducated girl? | 49% | 49% |
| Did you know that an educated girl in Africa will have fewer, healthier children than an uneducated girl? | 56% | 43% |
| Did you know that for each year of school that a girl in Africa completes, she will increase her income by up to 25%? | 18% | 81% |
| Did you know that an educated girl in Africa will re-invest 90% of her earnings back into her family? | 26% | 74% |
Based on the results of this survey, it’s safe to say that a majority of the respondents were not aware of many of the conditions facing young girls in Africa and around the world.
Results of the Hybrid Survey
There were 55 respondents to the Hybrid Survey—far fewer than the other two. This is because I didn’t think to add this variation to the study until it was over halfway finished. The results are still interesting, though, and they look like this:
| Question | Yes/True/Good | No/False/Bad | N/A |
|---|---|---|---|
| Would you describe yourself as a generous person? | 84% | 15% | |
| The last time you did something for someone less fortunate, did it make you feel good or bad? | 89% | 0% | 11% |
| Are people who give without expectation of return better than people who do not? | 60% | 38% | |
| TRUE OR FALSE: When I donate money to charity, I feel good about myself. | 71% | 27% | |
| Did you know that an educated girl in Africa is 3 times less likely to contract HIV/AIDs than an uneducated one? | 42% | 58% | |
| Did you know that an educated girl in Africa will marry later than an uneducated girl? | 53% | 47% | |
| Did you know that for each year of school that a girl in Africa completes, she will increase her income by up to 25%? | 27% | 69% | |
| Did you know that an educated girl in Africa will re-invest 90% of her earnings back into her family? | 24% | 76% | |
| Did you know a girl in Sierra Leone has only a 1 in 6 chance of attending high school? | 13% | 87% |
| Question | Happy | Sad | Angry | Excited | N/A |
|---|---|---|---|---|---|
| How do you tend to feel when you help someone in need? | 76% | 4% | 0% | 16% | 4% |
For the five questions adopted from the Personality Survey, respondents answered basically the same way with a slight variation (a drop in good feelings) on the question about how you feel when you donate to charity.
For the five questions adopted from the knowledge survey, respondents were decidedly more knowledgeable about the plight of young girls in Africa. Interesting.
What Survey Produced the Most Promised Donations?
If you remember, these surveys were set up with a “sneak attack question at the end—”Will you make a one-time donation of $10 to One Girl?”
Since this project is all about getting people to give more money to charity, we should start the analysis here. The results are fun to look at, but they’re meaningless without knowing how they affect a person’s desire to get their wallet out and do some good with it.
So which survey produced the most promised donations meaning, how many people answered the above question “yes”?
We have a winner! By a margin of 3%, the Knowledge Survey produced the highest percentage of promised donations.
Remember: Because a different number of people took each survey, we can’t accurately compare straight numbers. Instead, we must work in percentages to see which survey performed best.
We could stop here and say, “To get the best results for your project, ask your potential customers questions that test their knowledge about problems you aim to solve!”
But if we did that, we’d be wrong. Read on to find out why.
Which Survey Produced the Most Actual Donations?
Growing up, my parents taught me a very important life lesson:
Talk is cheap. If you want to prove you mean business, you do it with your actions, not just your words.
This lesson is applicable to all facets of life and, as it turns out, is especially applicable when it comes to charity.
After a respondent finished their survey, I presented them with a link to go make their donation. The link went to the donation page at One Girl where they could quickly enter their credit card number and make their donation.
The Knowledge Based Survey won the showdown for most promised donations, but how many people who said they would give money actually did so?
To find out, I used a click tracking service that would tell me how many people actually went to the donation page. Here are the results:
Respondents that took the Knowledge Survey were eager to commit to making a donation, but had the biggest drop when it came time to actually pony up. The conversion rate dropped 7%!
But respondents to the Hybrid Survey—who were less eager to promise a donation—were more likely to follow through. The conversion rate only dropped 3%.
So, at the end of the day, The Hybrid Survey produced the best results! Long live the Hybrid Survey!
Quick Analysis: Knowledge questions produce the highest amount of promised donations. Personality questions produce the worst result all around (Surprising!). The best result, however, comes from mixing the two together.
Recommendation: Based on these findings, to get the best real conversion rate when using a survey to increase revenue for your non-profit or business, ask questions that cover both categories—personality traits and knowledge.
Wait, Who Cares About A Measly 1% Improvement?
You’ll notice the difference in real donations between The Knowledge Survey and The Hybrid Survey is only 1%.
Here at Riskology.co, I like to encourage long-term thinking. You will always be better off making decisions that are best for you in the future than just right now, even if it’s more difficult to do so.
If you look at the results of this little study from a “What can I get right now in this very moment?” point of view, the answer is:
A whole lot! If you’re not using surveys to increase the income in your business or charity project, you’re leaving a lot of money on the table. Even the worst survey still converted at 36%, which is astounding.
But you may wonder still, “Who care’s how I do it as long as I just get started?”
Good question. Yes, you should just get started. But once you’re done with “just getting started,” you should work on “just getting a little better.”
In this little experiment, I tested three different survey styles. The difference between best and second best was only 1%. But thanks to the powers of Mathmagics™, you would realize that an increase from 41% to 42% is not just a 1% improvement but, in fact, a 2.4% improvement.
Still not the most exciting number to look at but, over time, a 2.4% improvement makes an enormous difference. Especially if you consider the Mathmagical power of compound interest.
We’ll use this experiment as an example.
Let’s say I decide to campaign full-time for One Girl, and I plaster the surveys we just used all over the site. Everyone that comes to Riskology.co will see it, some will take it, and some will go on to donate.
I’ll estimate conservatively and say that I could get 10,000 people to take the survey each year.
If I were lazy and just went with my first assumption that the Personality Survey would do best, I’d raise about $36,000 for One Girl in a year. That’s already big money. Nothing to sneeze at!
But if I got off my butt and tested the Knowledge Survey, I’d quickly find that it did much better. It would bring in $41,000—a $7,000/year improvement. Wow!
Now I’m feeling lazy again. I know that I could get a 2.4% improvement by using the Hybrid Survey, but that would only bring in $1,000 more each year. Who care’s about the difference between $41k and $42k?
If you’re a true Riskologist, the only correct answer is: I do! I do!
Over 10 years (remember, we’re thinking long-term here), that’s an extra $10,000!
And what if One Girl decided to invest that money prudently to grow their operation, gaining the historical stock market return of 7% over that same time?
I’ll tell you what.
The Hybrid Survey would produce $630,000 over a 10 year period. The Knowledge Survey would produce $606,000 ($24,000 less). And the Personality Survey would produce a paltry $532,000 ($98,000 less!).
If I were really dedicated, I’d test new questions and hypotheses regularly. And I don’t doubt that, if I committed myself to it, I could improve the donation rate by at least 1% each year.
This would raise over $1 million in just 14 years.
Now look at that. With just a bit of Riskologist elbow grease (okay, a lot of what would be “hard work”…), we took the potential for this little experiment from the lame $2,310 that I raised in a few days to $1 million.
I don’t claim that money is the answer to the problems of the world (it very much is not), but how many more people could you help with $1M than with $2,310?
This is why I prefer to think long-term and why I get very excited about even the smallest improvements, like a measly 2.4%.
Limitations and Opportunities of This Experiment
We covered a lot of ground here, and if you’re not excited about the opportunities that come from using surveys to raise money, well then, I give up!
But every experiment comes with a number of limitations. In the name of full-disclosure, I’m going to list a few of the biggest ones here:
- Actual, traceable donations were not tracked. Money was accepted by One Girl’s donation system, and there is not a good, clean way to track exactly how much money was actually donated. It’s possible (and likely) that some respondents who viewed the donation page did not actually submit a donation. And it’s just as likely that some who did changed the amount of their donation before submitting.
- Only three survey types were tested. For this experiment, I limited the variables to personality-based questions, knowledge-based questions, and a hybrid including both. There are many more categories of questions that need to be tested!
- All surveys were ten questions long. It would be interesting to see how surveys performed if they were shorter or longer.
- The donation amount was fixed at $10. It would be interesting to see how surveys performed when asking for different amounts or even no specific amount at all.
- The Hybrid Survey asked personality questions first and knowledge questions second. It would be interesting to see what happens if the order of questions were flipped or mixed up.
- Many others I’m not smart enough to recognize, but you’ll surely tell me of.
Of course, with every limitation comes an opportunity for someone else to carry the torch and improve upon my work. I sincerely hope that what you’ve read today inspires you to get started with your own experiment.
Take what you’ve learned here and test one of my limitations. And, of course, report your findings to the world; expand the body of knowledge!
Important: I have much more to say about this, but at more than 5,000 words, I’ve decided to cut it off here for now. I need another 5,000 words to explain all the other data I collected (and analysis I performed) for this experiment that’s going to help you get even better results.
There’s more to come. Stay tuned!