Data Science in Action: Explaining Psychology Using Statistics

Imagine you’re in a shower, enjoying hot water, and you can hear a distant ringtone of your phone in the other room. Just imagine getting off, drying off, all the running with still wet head and picking the phone up a fraction of a second later when the other side just hang up. What an event. It will definitely stay in your memory. What’s not going to stay is you taking a shower, enjoying the water and thinking “Hey, the phone isn’t ringing”. What’s not going to stay in your memory is amount of times throughout the day when your phone rings and you are not in the shower.

I guess, you definitely have heard from your friends or said to yourself “The phone rings every time I am in the shower”, “My bus is always late”, “Someone always messaging me when I write an article” (just happened :D). These one-sided events make a huge impact on ourselves. We tend to believe in prophetic dreams, we tend to give importance to rare, but random events and it’s OK. Because this is how our brain works, this is the cognitive process. We put familiar faces in our dreams, random events happen all the time, – we just have to recognize it and be more careful with our judgments. A lot of things can be explained by natural means. But let’s face the reality – if someone had a dream that tomorrow their dog will die and their pet actually dies, I won’t convince this person. Even by telling that dogs are not immortal and they may die and there is a probability that this particular dog can die in this particular day when you had this particular dream.

But if we modify this experience a little. Imagine, you wake up after a nightmare about your dog’s death, you stand up quickly, go to check it and find your fluffy mate is sitting in front of the window thinking about much more important things. All good. You come back to your bed and forget about everything.

It was an extreme example, but this kind of things happen all the time. For example, my colleague started a discussion saying: “Anytime I look at my phone, the numbers for hours and minutes are the same. Look it’s 19:19!”. So I decided to run a little experiment to find the probability of seeing “lucky numbers” few times per day. We all humans and we will say that we see something always when we see it 3-4 times in a row. Knowing about the effect described above it was not so complicated, as it is enough to see “lucky hours” few times in one day to make that claim, not even necessarily seeing it in a sequence.

Let’s start with an agreement on what we call the “lucky hour”. I decided to include into this category zero hour (14:00, 22:00), same hours and minutes (21:21, 02:02), and reverted hours and minutes (23:32, 15:51) – in total 64 occasions. As we know a day have 24 hours or 1440 minutes and by doing simple calculation 64/1440 = 0,04(4) we find that the probability to spot a “lucky hour” is around 4,4%. Not that easy as a flip of a coin with 50% chance or a dice with 1/6, but it is possible.

This process can be described using binomial distribution: I check the phone -> I see “lucky hour” or normal. True or false. And to apply this I needed a number of times a person checks their phone on average. Google is the best to answer this question. Different studies in different countries showed different results, from 20 to 150, so I gathered those numbers that were the most repeated in different sources. Also I took three different measures: 28, 47, 86 as minimum, average and maximum. Also I thought it’ll be interesting to see the differences.

The code below performs all the calculations.

import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns

sns.set()

np.random.seed(666)
a_min = np.random.binomial(28, 0.044, size=10000)
p_a_min_2 = np.sum(a_min > 1) / 10000
p_a_min_3 = np.sum(a_min > 2) / 10000
p_a_min_4 = np.sum(a_min > 3) / 10000
# print statements removed, link to github will be below

a_avg = np.random.binomial(47, 0.044, size=10000)
p_a_avg_2 = np.sum(a_avg > 1) / 10000
p_a_avg_3 = np.sum(a_avg > 2) / 10000
p_a_avg_4 = np.sum(a_avg > 3) / 10000
# print statements removed, link to github will be below

a_max = np.random.binomial(86, 0.044, size=10000)
p_a_max_2 = np.sum(a_max > 1) / 10000
p_a_max_3 = np.sum(a_max > 2) / 10000
p_a_max_4 = np.sum(a_max > 3) / 10000
# print statements removed, link to github will be below

And this is what I’ve got printed in the console:

=== Assuming average person checks their phone 28 times per day ===
Probability of seeing ‘lucky time’ two times per day: 0.3543, three: 0.1284, four: 0.0357
=== Assuming average person checks their phone 47 times per day ===
Probability of seeing ‘lucky time’ two times per day: 0.6115, three: 0.3404, four: 0.1449
=== Assuming average person checks their phone 86 times per day ===
Probability of seeing ‘lucky time’ two times per day: 0.8966, three: 0.734, four: 0.5271

As you can see, the probabilities are quite high! If we check our phones 86 times per day the chance to see 4 “lucky hours” is about 53%! And don’t forget about the feature in new Androids – always-on display (no clue if Iphones have the same). Of course, you will come up with the conclusion that you see only these “magic hours”.

And you may notice this type of event even more than 4 times, the simulation shows this number can reach even 12. Look at this histograms:

On the last figure we can see that if we check our phones very often it is less probable to NOT see a “lucky hour” at all than seeing it 8! times per day.

I tried to go further and find the probability of seeing these magic numbers 2 times in a row, but with 10 million simulations I’ve got nothing. Here is the code, if someone wants to try. (please, correct me if I made a mistake, that’s also probable :D)

n_sequential = 0
size = 28
sample = 10000000

for _ in range(sample):
rare = np.random.random(size=size) < 0.044
n_rare = np.sum(rare)
if n_rare > 1:
for i in range(size):
if i == size-1:
break
elif rare[i] is True & rare[i+1] is True:
n_sequential += 1

Conclusion: the probability of spotting 2 “lucky numbers” in a row is very-very low, given that you check the phone randomly of course. So how can we talk about 3 or 4? But because of those processes in our minds we tell people that we see ONLY “magic hours” when checking our phones.

It is a silly example, it won’t make any difference in your life if you will continue telling other people about having Dr. Strange’s superpower of managing time. I want us to think a little before making the judgments and claims, to use scientific methods to find answers to our questions, to open our minds and consider natural explanation to unnatural, from the first sight, things, consider other opinion which may differ from our own. Being stubborn is like being closed in your own room, yes it is comfortable, calm and safe, but what a beauty is outside! Wide calm lakes and rivers with fresh cold water, seas with lovely beaches and powerful oceans with big great waves, quiet forests with sun rays between the leafs and limitless colorful fields of flowers, mountains and hills with astonishing views, cities with their busyness and villages with their tranquility…

I guess, it’s worth leaving your room.

P.S. Link to Github with the code.


Photo by Riccardo Chiarini on Unsplash


Leave a Reply

Your email address will not be published. Required fields are marked *