In statistics, the number of degrees of freedom is the number of values in the final calculation of a statistic that are free to vary. The number of independent ways by which a dynamic system can move, without violating any constraint imposed on it, is called number of degrees of freedom. In other words, the number of degrees of freedom can be defined as the minimum number of independent coordinates that can specify the position of the system completely.
However, I wanted to quickly provide you a different interpretation than most statistics instructors. This discussion is from The Choice by Eliyahu M. Goldratt.
Goldratt informs us that when most think of complexity, they define it as:
"The prevailing definition for complexity is this: the more data elements one has to provide in order to fully describe the system, the more complex the system is."
Yet, Goldratt provides us with a better definition:
"But there is another definition of complexity. If you are a scientist or a manager, you are not so much interested in the description of the system. You are more interested in the difficulty of controlling and predicting its behavior, especially when changes are introduced. Your definition of complexity is - the more degrees of freedom the system has the more complex it is."
By "degrees of freedom" Goldratt informs us:
"Look at the system and ask yourself, What is the minimum number of points you have to touch in order to impact the whole system? IF the answer is one, THEN the system has only one degree of freedom."
"IF the answer is four, THEN the system has four degrees of freedom. By the way, a system that has four degrees of freedom is many orders of magnitude more complex - harder to control and predict - than a system that has only one degree of freedom."
Goldratt explains the concept known as Inherent Simplicity and that it is hard for people to accept that it (Inherent Simplicity) exists in what is apparently so complex.
"Newton came up with his three laws of motion. Newton did not invent his three laws, he discovered them. He revealed the Inherent Simplicity that was there. Newton was probably one of the first people to dare to seriously ask the question 'why?' By seriously, I mean to ask why and not be satisfied with an answer that is actually not an answer."
"When we ask why something exists, searching for the cause of that something, we usually get more than one answer or we get an answer that contains more than one component."
"What Newton tells us is that the system converges; common causes appear as we dive down. If we dive deep enough we'll find that there are very few elements at the base - the root causes - which through cause-and-effect connections are governing the whole system. The result of systematically applying the question "why" is not enormous complexity, but rather wonderful simplicity."
"Tell me how you measure me and I'll tell you how I will behave."
"The cause is the Measurement, and the effect is the Resulting Behavior."
Goldratt called this cause-and-effect analysis through simple common-sense logic.
The image above is a reinterpretation of Goldratt's diagram in Chapter 4 of The Choice. Goldratt informs us:
"Here are two systems. The one on the left, let's call it system A, is represented by four circles [Plectica Cards/rectangles]. And the nightmare of circles [Plectica Cards] and arrows on the right is system B. Which of these two systems is more complex?"
Using what we now know as degrees of freedom, which one possesses the minimum number of points you have to touch in order to impact the whole system?
Using an example from the Kansas foster care system, we can see using Goldratt's definition, that system B has one degree of freedom, while system A has four degrees of freedom. Therefore, system A is vastly more complex.
By impacting the bottom Plectica Card in system B, you can alter the entire system. However, you would have to alter four Plectica Cards to alter system A.