It is a pretty simple concept really. Anyone who tests knows that they really want
some level of confidence that the results of the test actually reflex the
reality of what was being tested. You
test students and you probably have some level of confidence that the scores
produced actually reflect what the student does or does not know or
understand.
However, what about a teacher’s own personal judgment of
that same student’s knowledge or skill?
What if it does not agree with the results of the test? Do they believe as a trained observer that
the student really knows the material and yet they for some reason struggled
with the test? Or the opposite where they
are reluctant to believe the student knows something even though they passed
the test. Do they have the confidence to
believe their own judgment over that of the exam?
And then there is the student’s confidence in their own
knowledge or ability? We have all
probably said or at least considered saying, “Look, I know what I know”. More important however is the implication
from such a thought or statement, that if you indeed do “know what you know”, you
probably also have a pretty good handle on what it is that you do not
know.
So
what does all this have to do with the confidence of competency? Well a lot actually. As schools slowly start to transform
themselves into more student based and confidence based entities, knowing when
you have sufficient confidence to actually mark a student competent and begin
building on that knowledge or skill with new curriculum can be critical. Legacy systems just assumed that a student
passing the tests and doing all the other prescribed learning work somehow
magically produced confidence in the ability to build on that work in the
future. But that has proven to be a very
poor assumption.
One
way to look at the problem is by first asking yourself if a student would say I
“think” I know this stuff, or simply… I know this stuff! Would they defend their answer with an
explanation if challenged? Would they
properly identify errors in their understanding of future conceptual models
because they believed that past foundations were sound, therefore the problem
could only be with the new learning and not the old?
Dr. Robert Marzano is an established expert in the field of
standards based and competency based education.
In most of his publications he calls for a “Preponderance of Evidence”
when evaluating student performance for competency. The need for such a “preponderance of
evidence “ is to confirm some reasonable level of confidence and reliability in
the final declaration of competency. But
how do you know what that is? When has
such a “preponderance” level been established?
Most likely it is slightly different for each thing being measured, who
or what is doing the measurement and who is being measured. Yet no system of tracking is designed to handle
that extreme level of diversity. But
there are some good ways to think about it.
The best way is probably to just take those three dimensions
of the tools, the evaluators and the learners, and somehow apply a good ordered
estimate of competency or understanding to each learning goal. Knowledge and skill also have to be looked at
separately with different priorities.
For example if you can demonstrate that you can hammer a nail and that
was the goal, then the assessment is complete via just the observation by a
trained observer. Yet if the goal is the
addition of 3 digit numbers, it is impractical to test all of the combinations
of two 3 digit numbers being added together.
You really need three things. You have to have a tool that challenges a
variety of options, plus a trained observer to try and visualize that a process
of addition has been captured by the learner when presented with the challenge,
and the opinion of the learner that they indeed have the confidence that they
could complete any of the million options if required.
If we can get these three views of the competency combined
in proper proportion for the competency being considered, the preponderance of
evidence we are looking for should be maximized. So new competency tracking systems will need:
- The ability to organize a variety of different and independent competency measurements.
- Estimate the level of confidence each offers into the real level of understanding by the learner.
- Logical and user friendly ways of adjusting those estimates from the structure of the goal itself.
And they will also need user friendly ways of illustrating
these unique pathways to competency measurement as they apply to the equally
unique ways students will have of completing their competency based
learning. It is quite possible that not
only are the Grades as we have traditionally used them slowly finding their way
to the history books, but with them will go the spreadsheet of grades that
simply average out how well a student is meeting expectation rather than
describing specific knowledge or skill.
If we don’t need the formulas, tracking the same measurements for the
same students at the same time, do we still need complex spreadsheets to
average the results?
Probably a more practical way would be a simple individual
learning path map, adopted from and tied to a larger competency based map with
options and requirements. Where
students are merging their maps would indicate opportunities for
collaboration. Where they are
diversifying their maps, opportunities would exist for re-grouping. Progress along their map simply becomes their
personal “grade book” or portfolio of competency sign-offs, confirmations,
badges, evidence and enabled opportunities for future adoptions of learning
pathways into their individual map.
