Lazy, lazy, tests: They are fast, easy, and non-invasive but they should never be solely relied upon

     Once upon a time (about 30 years ago), I was going through various dating services trying to find a lifemate. (Found her at a mutual meeting on an outdoor walking excursion.) I decided to use a reasonably rated dating service in Seattle. I filled out, completely honestly and transparently, all about me and the most important things about the person I wanted to meet. It did not work out well.

      I am short (shorter as I age) and a bit pudgy. I am also very well muscled. I used to be able to do leg presses of heavier weights than most of the people on my high school football team (I didn’t make the team — another story).

The erroneous world of BMI

     My height and weight statistics were such that I had a BMI of 31.3 which is considered “obese”. My height and weight were almost exactly the same as the quarterback of the Denver Broncos. No, I won’t claim I was in as good of condition as the quarterback. As I said, I was pudgy. What I wasn’t was “obese”. But no woman in the dating records had any interest in dating an “obese” man. I offered to go into their office and show them. No go. They had their fast, easy tool that told them what they wanted to know and they didn’t care at all that it was totally inaccurate.

     I was self-employed at the time and had my own health insurance. I was penalized for my muscle. The insurance agent SAID that he understood and sympathized but could do nothing about it. (The BMI is known to only be accurate for about 85% of the time — due to “abnormal” body composition of too much muscle or thick bones or other non-average aspects.) The company used the BMI as their Bible. There are other tests for better accuracy such as a water displacement test and a caliper test. There is even a pretty easy measurement test that is a lot more accurate. It includes the circumference of your waist and wrist in addition to height and weight. No go. Fast, easy, and wrong.

The autocratic world of Reddit

     Once upon a time, when I was building my audience for my blog (https://technoglot.blogspot.com) I made use of some of the subreddits of Reddit. As you may know, each subreddit has its own owner, or owners. They control the topic as well as the rules for appropriate entries. They are the ruling folks. But, as the royalty, they have no obligation to follow their rules. The rules are there to meet the requirements of a subreddit. Following them is optional.

     I never succeeded in staying on a subreddit long because they didn’t like a blog to occupy a place there (I did get quite a few views). None of the rules said no blogs but … I had one person who had written their own tool for spam detection. The tool detected duplicate base URLs! (So blog.com/X was spam of blog.com/Y because they both were based on blog.com.) An article from newsweek.com would have registered as spam because other articles would also exist within newsweek.com. But, their tool was fast and easy and they didn’t care that it was wrong (I have a Master’s degree in Computer Science). Not important.

The world of AI detection following the easy, quick incorrect path.

     Now, we are entering the world of AI generating text and material. I am getting reports on the use of tools dictating what is AI and what is not. One of the rules is that of using the — (emdash). Apparently, AI detection tools have decided that only AI generated text makes use of the emdash. Balderdash. But, I am trying to wean myself away from using that indication of an extended pause because I would prefer to not be erroneously marked as AI. First BMI, now potentially AI. All out of laziness being more important than accuracy.

     So, what other items is an AI detector likely to flag? Formal, academic style writing, simple and clear sentences. If it is clear, direct, easy to understand then you are likely to be in trouble. We will all need to write like Lewis Carroll in “Jabberwocky”. We all need to make our prose as unpredictable and erratic as possible. Oh, there are some understandable authors that have a “unique” voice who should be safe. Coming from a non-fiction, technical, background, I am probably going to be in deep doodoo.

     Frankly (does anyone ever say “Georgely”?) I don’t think it can be done. So be prepared to be accused of being an AI if you love language and want to type it according to the rules of English. Oh, yes, ESL folks are also likely to be flagged because they are taught direct, straight-forward, sentence structure and “normal” writers don’t sound like that.

What’s going to happen?

     I would like to say that all of these folks that are using these tools will sit up and say “my goodness, are the tools that bad? We should immediately stop relying on them so much”. But, people are people and most want to follow the quick, easy, cheap route. Plus, here in the US, we love those numerical scores. If a tool can give us a number as output, we can’t help but fall in love with it.

     I’m “lucky”. I make almost no money from my books, blogs, and newsletters. So, it doesn’t matter if I am accused of being an AI. And I can find (and have found) other methods of distributing my blogs (and make use of SEO tools). And I am settled into a permanent “death do us part” relationship. So, as Bill Murray says so eloquently in “Meatballs” — “it just doesn’t matter”.

Sure, it doesn’t. Good luck to all of us.

Ideas & Interpretations is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.

Odds are for the future: a brief chat about statistics

 

     Something happens. Something unexpected. A person near you says "what are the odds of that happening?" Well, since it has already occurred, the answer is 100%. Without a parallel universe, the chances of it NOT occurring no longer exists.

     In statistics, this situation is expanded by the concept that the past has no bearing on the future. If the odds of something occurring are 5 to 1, then after the event has occurred in the past then the NEXT odds of something occurring are still 5 to 1.

     Every time you flip a coin (with a fair, properly balanced, coin) the odds are 50% heads 50% tails (yes, with that very small possibility of landing on edge left out). If you toss a coin and get heads seven times in a row then the next time you toss it the odds are still 50% heads/50% tails.

     Now, if you anticipate the future, you can determine the odds of getting heads seven times in a row (1/2 * 1/2 * 1/2 * 1/2* 1/2 * 1/2 * 1/2 == 1/256) for the future. But every time you flip the coin, the odds for that flip remain the same.

     The phenomenon of people expecting the past to affect the future exists in many people -- good luck/bad luck. It exists in all addicted gamblers (the scientific gamblers are a different matter). That lucky streak just HAS to affect the future. (It doesn't.) I had an uncle who won the state numbers lottery (not a huge amount at that time -- but significant) and he put the money all back into the lottery system until he no longer had any net winnings.

     There are situations where the odds do change with time. This is true for actuarial tables. Your odds for living to a certain age changes with each year you live. If you live until you are 75, your odds of living until 85 (10 more years) are better than the odds of you living until 85 from the age of 55 (30 more years). But, be assured, the insurance company has it well planned out such that, on average, they are not going to lose money. Those tables require ignorance of the future to follow the actuarial odds. Robert A. Heinlein's story "Life-line" indicates a way to beat the system -- but it may be the only one.

     There are also situations where the perceived odds -- what we think exists -- are not the same as the statistical odds. Professional gamblers are aware of these situations and, by gambling calmly and without emotion, succeed in taking advantage of them -- over the long run. This also happens in other areas of investment and helps certain investors increase their likelihood of beneficial returns.

     All venture capital firms rely upon both experienced analysis and spreading their risks. Five investments of 10K still gives a good profit if two are totally lost, two give back 15K each, and one brings back 30K.

     "May the odds be ever in your favor."

All in the Numbers: Popular Economic Numbers as Used by Politicians

     There was "good news" in the U.S. in October 2019 (see article). The U.S. added 128,000 jobs to the economy. Unemployment is down. But, wait, what ARE those 128,000 jobs? According to CNBC, "The Labor Department explained in a press release that much of the upswing in leisure and hospitality came from a surge in hiring at food and beverage establishments, which alone added more than 45,000 jobs for the sector’s best month since January" The article further indicated that 60,000 were from the leisure and hospitality sector while health care and social assistance added 34,200 jobs. Is there a pattern here? Ah -- 22,000 jobs were created for professional and business services which is defined as management consultant positions, computer system design, and architectural and engineering services.
     So, we have 105,000 jobs added in fairly low-wage areas, 22,000 added in relatively high-wage areas, 36,000 lost in medium-wage manufacturing, and a few dozen thousand added in low medium-wage areas. Is this a success? Or is it a failure?
     The point is that the number of jobs added is not very relevant unless one also knows the median wage paid for those jobs. In the U.S., and in most of the more industrialized areas of the world, "income inequality" continues to accelerate. Numbers such as the "number of jobs created" are meant to encourage and say things are great. Maybe they are -- but the analysis of the October 2019 job results don't tend to lend themselves to that conclusion. Lots of low-paying jobs and many fewer higher-paying jobs do not start to spread the economy out to more and more people.
     "The average wage for people in the U.S. has gone up!"  Sounds great. Wait a minute. If I have 10 people working and five earn $10, 3 earn $15, 1 earns $20, and 1 earns $100 -- we have an average wage of $21.50. What happens if that person earning $100 now earns $200? Wow. We now have an average wage of $31.50. Doesn't that sound great! probably not for the lower-earning 9 people.
     So average doesn't tell us much. How about median or mode? Median is the value given when, sorted in order, the number of instances of a value below a point is the same as that above that point. In our example above, with an even number of instances (10), the median is the average of the values at position 5 and position 6 -- so, for the above example, it would be $12.50. Note that, unlike for average, if the person in position 10 (highest earner) doubles their income, the median does not change.
     How about mode? Mode is the "most popular" instance. Since, in the above example, five people earn $10, $10 is the mode. Once again, when the most highly paid person gets paid double, the mode remains the same.
     There is a quote popularized by Mark Twain which I have always liked: "There are three kinds of lies: lies, damned lies, and statistics". Although this does call out the need to both understand, and interpret correctly, statistics -- it is not really fair to statisticians. But statistics are a politician's best friend largely because most people do NOT know what statistics mean or how they are calculated.
     "The average wage has doubled!" Sounds good, right? Not necessarily. "500,000 jobs have been produced in the past month!" Wow, things are completely rosy. Once again, not necessarily. If 400,000 jobs were lost then it is actually terrible. Usually, what is reported is NET gain so if 500,000 jobs were added and 400,000 were lost it would be reported as 100,000 jobs added. Sound good? Well, with that great of a turnover of labor, the chances are not wonderful that those 400,000 (actually, probably not the same people) who lost and got a job improved their wages and benefits.
     "$70 million dollars of welfare fraud found". Horrible, right? Well, yes, it should certainly be reduced if possible but total welfare funding is about $67 billion. So, this atrocious fraud is actually part of 1/1000 of all of the welfare funding -- or 999/1000 are NOT committing fraud. In most businesses, this would be a resounding success. In Florida, politicians stamped the ground and screamed about all of the welfare fraud -- and spent MORE money investigating the fraud than the net amount of the fraud. A net taxpayer loss.
      Let's take the above $67 billion in welfare funding mentioned, once again. A lot of money isn't it? It certainly is and I would not mind getting my $200 share of that money. But, within the context of overall spending, the $67 billion is insignificant in comparison to the overall federal budget -- about 0.5% or 1 part out of 2000 [note that it is about 21% of the federal social support budget but that is also not a high percentage of the overall federal budget]. The $67 billion is not significant overall but politicians blow it up to seem significant so that people will ignore the other 99.5%.
     Another side track of statistics from the politician's mouth is rankings. We are number one! OK. We probably are -- but number one in ranking for what characteristic? Hopefully, the people of most countries think that their country is the best -- a subjective ranking that has multiple truths based on who is saying it and where they are saying it. But, when we get to rankings based on quantitative (countable) characteristics, the record isn't so great. Yes, if a specific formula is applied to the countable numbers, there will be a most highly ranked (# 1) and a most lowly ranked (# xxx). But not many countries have multiple "# 1 rankings" for their country. Given a large enough population and land mass, most countries will have A (one or more) "#1 ranking". But, even in this case is being first in numbers of prisoners equivalent to being first in the number of books per household? So, relative rankings are fairly useless -- dependent on many different factorsl 
     Numbers do mean something -- but not always what they are implied to mean.

Is Being Average "Normal"?

     Many people are fascinated by surveys and tables of statistics. They can be fun -- and even useful, when interpreted by people who know what the data does, and does not, indicate. They can also be misused by presenting data in such a form that people will jump to conclusions that are not really supported by the data. This is done directly by many politicians and for political purposes and causes.
     I have mentioned before the quote popularized by Mark Twain -- "There are three kinds of lies: lies, damned lies, and statistics." I like statistics because I am fascinated by numbers. Their relationships and the ability to use various formula upon the data is a lot of fun. But I don't have sufficient training to use (or misuse) them properly.
     But this particular blog is not about such weighty matters. It is more about how people can take "everyday" data and make more of it than what they deserve. In particular, it is about averages and the definition of normal. An average is a fact when you are dealing with numbers. The average of two, five, and eleven is six.  The average height of a human male in the United States is approximately 69.7 inches (177 cm) -- but there are not that many men in the United States who are exactly 69.7 inches tall. In other words, few men are of "average" height.
     Averages only make sense for characteristics when you examine them within a distribution. A normal distribution curve looks like:

     Normal means, basically, that the curve has a uniform change throughout the graph -- there are just as many above the average as there are below the average and the percentages change consistently on both sides of the "bell curve". This ideal curve doesn't occur that often in nature. One problem that happens is that there are often assumptions that a distribution of data will meet a normal distribution without doing a sufficiently wide sampling of data to justify it. For example, the data for men and women's heights in the U.S. can look like:

     Note that the peak for men ends up around 70 inches, while the peak for women is at about 64.5 inches (164 cm). The peak is higher than for men and the curve is narrower. This indicates that women's heights don't vary as much as men and concentrate at the average. However, in both cases the curve is very "smooth" and symmetrical. For height, it seems that the data really does seem to support this -- with a longer "tail" at the tall end indicating a slightly greater number of very tall people versus the number of very short people.

     Any characteristic that can be measured can have an average. Skin color is based on the amount of pigmentation from different types of melanin in the skin. There could be an average value for this. The average weight for a U.S. male (in 2015) was 195.5 pounds (88.5 kg) -- an increase of 30 pounds (13.5 kg) since 1960. The number of hair follicles per square inch can be measured -- thus, there is an average value for "hairiness". Ear size can be measured from top to bottom (or front to back or distance out from the skull).

     How many people have an average height, weight, skin color, hairiness, and ear size? Probably only a half dozen or so within the United States. Are these six people the "normal" ones? No, not even from a statistical sense because there are going to be many other measurements that are potentially able to be done -- eye color, IQ, foot size, hand span, distance from bottom of nose to top of lip, and so forth. Everything that can be measured can have an average -- but none of them are "normal" because humans are not just one characteristic. They are combinations of many, many characteristics and there may be some that are genetically linked (if you have one then you also have the other) but most appear to be totally independent. Skin color is totally independent of IQ. Eye color appears to have little to do with height. Weight and height do have some correspondence but it is possible to have a tall thin person as well as a short heavy one.

     Since being average in every possible measurable area is highly unlikely,  it is certainly not normal to be average.

Magic numbers: Society and what is "normal"

I was involved in a thread once upon a time and got trolled (expected if you talk about anything of significance -- and sometimes even if you are just talking about the weather). This was a thread talking about one of society's "magic numbers".

These are part of a group of numbers which we use based on statistical information. As Mark Twain once said (he said that the British Prime Minister Disraeli said it first): "There are three kinds of lies: lies, damned lies, and statistics."

For example, in current society a number called the Body Mass Index (BMI) is used to determine whether you are overweight, underweight, or in a "good range". This number is a ratio of height to weight and provides reasonable results for 80 to 85% of the population. For the other 15% to 20% of the population, the number just doesn't work well. If you are an athlete (or work out a lot and have more muscle) it doesn't work well. If you are a total couch potato with almost no muscle it will actually give you a better result than you "deserve". If you truly do have "thick bones" you will be at a disadvantage. This doesn't matter that much except if people are basing other things on that number -- insurance companies and computerized social services for example.

Why do we use this "magic number"? It is quick. It is easy. It is cheap compared to other, more accurate methods of body fat percentage calculations. However, even if one used a more accurate measure than that of the BMI, it would STILL not be accurate for everyone because there is no "one body shape fits all". For some, to hit that "ideal", one part of their body would have to be UNDER-weight in order for the average to be correct.

There are other "magic numbers" used. The "age of consent" is a magic number which, in more technical society, indicates that a child has become an adult. They are able to sign contracts, do things without permission from parents or guardians, get married and so forth. This magic number ranges from 13 in Japan to 21 in Bahrain. In some non-technical societies, it depends on the age when menarche sets in for women and "rites of passage" for men.

Many people get the direction of "age of consent" backwards -- thinking that a higher age of consent protects the child more from society. In reality, the family, church, or government usually have many options to do what they want with children at the age they think is right. The "age of consent" is what gives people the right to control, for themselves, decisions that affect their lives.

These numbers are set for two principal reasons. One is to prevent abuse of others by pushing them into activities (such as marriage or other things) before they can really make good decisions for themselves. The other is to determine an "age of emotional and mental maturity" which is taken to be an indication that they can make good decisions.

This type of magic number is determined almost solely by societal norms. It is usually lower in agricultural societies and higher in societies that require a longer period of education and social adjustment. It is also higher in societies where familial, and religious, control of women is greater. But, as is true with the BMI, it is (at best) a statistical reflection. Some will not be ready at age 30 to properly make decisions for themselves. Some might be ready at a very young age. There have been no psychological studies and are unlikely to be such.

Even the "age of consent" is not for everything. There are separate "magic numbers" for voting, being drafted for war, being able to drive, being able to work full-time, for purchasing and using legal drugs, and so forth. In many instances, the society would like to prohibit the activity but do not have sufficient backing from the populace to do such. Therefore, they set an age which most of society agrees is appropriate.

In all of these areas (and more), the "magic number" is sometimes determined by a statistical averaging and sometimes determined by societal norms. It is rare that the number is backed up by thorough, and consistent, studies -- which is why it is "magic".

What other "magic numbers" are you aware of within society? Do you know of any that have a researched background reason? Others that do not have such a background?

Simplified is not necessarily better -- the tyranny of the average

There is a strong tendency in our society to try to make things "simple". I don't know whether it is because we are so time rushed or because we have such an imbalanced system of education. Sometimes having it simplified works to a specific person's benefit -- sometimes it is quite unfavorable to a specific person.

Simplification is often linked closely with statistics. As Mark Twain is quoted as saying -- "there are liars, damned liars, and statisticians". Statistics are applied math so they should be accurate -- however, the actual use of the formulas and numbers are decided by people and people make decisions based on what they want to have be true (either consciously or subconsciously).

One example of this are the figures quoted by politicians during election time. "The average income tax went down during my administration". Hmmm. Well, this statement could be "true" if the average income also went down during the time. It could be "true" if the tax rates went down (this is the interpretation the politician probably hopes you will make). It could be "true" if tax rates went down for one segment of the people (usually the high-income group) but went up slightly for other segments (and this has been happening in the U.S. for the past 12 years or so).

Three different realities -- all "supported" by the same "facts".

One area that hits hard for me is the Body Mass Index (BMI) number. The BMI is a fast, easy, "simple" method to indicate whether a person is overweight. It is reasonably accurate (+/- 5% or so) for about eight out of ten people. Who are the other 20% of the population? They are people who are particularly tall (more than 15% above average) or short (20% or more less than average) or people who have large amounts of muscle tissue -- yes, the "fit" are most at risk from inaccuracies on the BMI.

All of this would be academic if it wasn't so easy to fit these simplistic numbers into other formulas -- such as actuarial tables used by insurance companies. So, if you happen to be a body builder then be prepared to pay more from private insurance companies (on life and health) for being "too fat". If you are very short, then you can be rather overweight and still have the insurance count in your favor. If you are very tall, however, then you need to be prepared to pay more once again. Oh -- and I forgot to add -- computerized dating systems tend to like to use BMI in their calculations so expect body builders to be matched up with others of ample dimensions.

The reason the BMI is used is that it is easy and cheap -- take your weight (in kg) and divide it by your height (in meters) squared and you have a handy, dandy, all-purpose number. In order to really find out an accurate number, it is necessary to find real body fat percentage. There are formulas that measure different areas on one's body and then use those in conjunction with weight to get a number that works for 95%+ of the population. But it takes more time and more time means fewer patients seen and that means smaller profits. It is also possible to do a submersion test (where your body is submerged into a tank of water to accurately determine volume) that is the most accurate way to measure density (weight divided by volume equals density).

These are two methods where the average can hurt those who don't fit. There are many others. But do your best to understand the implications of statistical statements -- it may not mean what it seems.