Friday, July 24, 2020

Data Silos: not just storage


     Many moons ago, I had a summer part-time job building grain silos in southern Kansas. Grain silos are buildings -- presently usually built with aluminum and concrete -- which allow storage of grain (corn, wheat, barley, oats, ...) such that they are contained, out of the potential rain, and on a dry surface. This allows them to stay good for an extended length of time and not rot.
     I haven't worked on a grain silo now for a long time. But I spend a lot of time trying to break down data silos. Data silos also keep things around for a long time. Some (mis)information within data silos may have been circulating for a decade or more. They also keep the data contained -- the data is circulated amongst the other sites and organizations that are within the data silo.
     The silo is delimited by internal consistency. Stories and sites within the silo refer to each other, repeat what others say, and report or create new information based on the information already being circulated within the silo. If you believe "A" to be true then you will only see information about "A" being true. If you believe "C" to be false, then you will only see information about "C" being false.
     As is true in many areas of life, there are actually a spectrum of truths that are happening. Event A is happening. Event B is happening. G said Y. G also said Q. One silo may report on A and G saying Q -- but nothing about B or G saying Y.  Another silo may report only on B and G saying Y. Thus, at times, multiple data silos can each contain A reality -- but a carefully maintained subset of reality.
     I mention creation of information. In other words -- fantasies, misdirection, lies. Why would a person or site deliberately create false information? I am not talking about someone who has "passed along" something false because they don't have the motivation to check to see if it is true first. I am talking about deliberate creation of false information. Why would a person or site do that?
     One reason is a desire for self-promotion. For an individual, their name gets associated with "information" that is exciting or motivating for others to remain in their data silo. For sites, there are often factors of advertising dollars or political inclination. Some "news" sites have teams of data creators -- very similar to the teams of script writers used for comedy shows or other variety shows that used to be popular. And, of course, there are sites and individuals who are not who they say they are -- international or domestic organizations who desire to maintain and intensify the divisions created by data silos.
     There are tools available to help a person break out of a data silo -- to get a more balanced (and, overall, much more accurate) view of what is going on. There are sites that evaluate general reliability. and bias ("right" or "left") of information such as that published by Ad Fontes Media. There are various "fact checking" organizations such as PolitifactSnopes, and FactCheck. There are two problems about these tools being used to break out of a data silo.
     The first, and most confining one, is that you have to have some degree of trust that the site, itself, isn't biased and deliberately giving a distorted view. Most of the more trusted sites, as listed above, are well liked AND disliked by many points in the spectrum of thought. That is probably a decent yardstick by which to measure. Deliberate information may, however, be more concentrated in certain areas -- the Ad Fontes Media graph can only point this out for well-known public information sites. The private sites that report, or create, information are not evaluated.
     The second, and possibly most difficult to change, reason that people do not try to break out of their data silos is that they don't want to. They are comfortable. The information may comfort them, or make them angry -- but only in the ways that they already are oriented. No confusion -- only yes/no, true/false -- and no spectrum of items.
     I talk about people breaking out of their data silos. Can't an outside person break another person out of their silo? No, I don't think so. Change of a person's ideas, or behaviors, has to be rooted in their own desire. That desire may be based on outside things (a new significant other, something so outrageous and blatantly false that it makes the ground shake, a drastic change of environment, ...) but the desire must be internal.

Interrupt Driven: Design and Alternatives

       It should not be surprising that there are many aspects of computer architecture which mirror how humans think and behave. Humans des...