observations
* there is an approximately finite set of learning inputs for a given "course"
* there is an infinite, unbounded set of possible learning outputs
* amount and nature of learning is a function of the individual and external context
* how much and for how long people remember is also extremely variable.
* trying to control the output is generally a great way to kill the learning spirit.
* quantity and quality are different types of learning outputs.
* do we want to use assessment to reward a level of knowledge, or the completion of a process?
* if a tree falls in the forest... if someone learns something and has no way to communicate or demonstrate it, is it learning?
* in assessment, there is an expected homogeneity of output (what was learned) from input (course content).
thoughts:
* perhaps we should think about using a statistical system instead of a rule based one. not "i AM or am NOT certified in X" or "I did or did not pass this course", but something more continuous. but what would that mean?
* do we want to measure process or output? eg. time spent? or skills learned? the latter is necessarily subjective, dependent on the context of the learner (among other things), and certainly not homogenous.
* "learning to the test" is bad, but so is teaching to the test-- it's worse in fact, because students in general look to the "teacher" or even facilitator to define the expectations of the learning experience. so no wonder they learn to the test.
what do we use assessment for? really, the role of assessment isn't, for example, "who should i hire," although it's often marketed that way. practically, it plays more of a culling or curation role, along with other factors, in choosing who to consider for some opportunity. it's an interior node on a decision tree. but ultimately, people make decisions based on many other factor we haven't learned how to measure yet, and maybe never will or maybe never want to. but the curation axis is relative and subjective-- that's why there's so much room for curation.
is there any objective, open ended definition of learning? what if we could measure the number of neural pathways formed? even then you'd have to identify which of those pathways were a function of the learning experience and which were a function of the rest of life :).
properties of some improved system:
* continuous instead of discrete
* heterogenous instead of homogenous
a proposal
actually one interesting idea would be to have people
assert for themselves what they learned in a course. of course anyone can assert something, but imagine using something akin to
TCP's 3-way handshake. Let's call it the
assert-challenge-confirm workflow.
student --> facilitator/course participants/community: "i learned x from this course"
facilitator/course participants/community --> student: issues challenge: "well then probably you should be able to do Y"
student --> facilitator/course participants/community: why yes, indeed i can, and here's an example.
this keeps the possible outputs unbounded, scales to the individual, and is declarative rather than passive. it leaves room for students who do not wish to assert any specific learning outcomes, without preventing or punishing their participation, or even needing an alternative model for it. in fact i think it's really natural that the "assessment" portion is an added burden on both the student and the assessors (whoever they are). it seems like it would focus assessment on those times or areas where it's particularly valuable. one could imagine gaming this system by initiating a challenge for a bazillion things until they find some they're capable of completing. aside from the human reaction which would likely limit this, you could also use a measure of
precision which considers correct claims and incorrect claims, instead of correct claims alone.
the alternative when assessment is overkill would be to issue simple participation badges that make little claim about assessment, but could still be used as an indicator the individual's interests and activities-- as well as a starting point for more custom/personal assessments and investigation of outcomes gained.
