Monday, April 28, 2025

Data Silos: circular references within a boundary

     Occasionally, people will talk about “data silos”. More often, there is just a complaint that people don’t listen to enough sources of information to be able to have an accurate view of what is going on. That is a description of the effects of being trapped in a data silo but it isn’t a definition.

     A silo is designed to contain things, to gather them for storage and dispersal. When I was working the summer before going to college, one of my jobs was to help build grain silos back in Kansas near my hometown. It was hot work and I was really itchy by the time I got home but I believe I did my job correctly except for once when a tool malfunctioned. My biggest concern was for my hearing as using a power socket wrench inside the enclosed space caused huge, very loud, echos. Perhaps that experience happened to someone who later thought about how appropriate the name was for what was going on with information — an echo chamber within a data silo.

     A data silo provides a bordered, protected, area within which the data are “safe” from contamination, or influence, from outside sources. But an echo chamber describes how data found, or created, within the data silo will rapidly echo to fill all of the data sources. And, whether false or true, that will become an accepted fact.

     Wouldn’t people worry about the validity of information that they see only in one small area of the possible sources? No, not necessarily. One huge danger signal is when data sources say “don’t trust anything outside of our information sources”. It is very similar to “don’t look behind the curtain” for the Wizard of Oz. If they were truly confident about, and proud of, their data sources no such warning would be needed. The fact that they are saying “trust us and don’t trust anyone else” is a huge “red flag” and everything should be very carefully checked.

     One great example of a well-defined data silo concerns David Duke’s book “The Awakening”. He professes that it is an academic book with references, footnotes, and detailed justification for his views (which, thank goodness, are not mainstream views). But those references point to people who agree with his basic feelings and their references point to others who agree with both of them and THEIR references point back to David Duke; this is an instance of “circular reasoning”. There is never any “grounding” for any of the data or information. It is all self-created with a close group of colleagues all agreeing to support one another.

      This happens with many social media streams also. They refer to another media stream within the same data silo which — eventually — will point back to the first source. To a certain extent, our self-censorship will create a situation where most of the things we read, see, or hear reinforce our general views. Once again, however, when any data source warns against paying attention to other sources — LOOK ELSEWHERE!

     There may never be a single “truth” — but there can be facts as long as those facts are carefully detailed (I saw this in this context from this angle after having eaten this and so forth). Well-defined facts should remain the same no matter what the point-of-view of others may be. When you are checking out whether or not you are caught within a data silo first ask “is this an interpretation or a fact?”. If it is an interpretation then you should recognize that the more interpretations you can find the more likely you can come up with an analysis that is close to reality.

     But, if it is supposed to be a fact, you must cast your net out. If you encounter a single instance where they disagree with the fact then it is vital you determine which is the one that is true. And for facts, there should be only one correct answer.

     There are information sources that I have repeatedly checked and found their data to be wrong a large percentage of the time. After a bit, I just discard all of the information presented by them. Some may be true but it is so likely to be false that my time and energy can be better used checking other sources.

     Always keep an eye out for “the person behind the curtain” and beware of warnings to not check on reality elsewhere.

Ideas & Interpretations is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.

Strata: Layers don't apply only to geology

 “If you can convince the lowest white man he's better than the best colored man, he won't notice you're picking his pocket. Hel...