Paul, thanks for your comments. You make a good point. "Intangible" factors like integrity, loyalty, organizational and politics play a significant role in the outcome of the assessment process. Such factors must be decoupled or accounted for in both the assessment model and the associated process. This has to be done with painstaking persistence and rigor. Note the use of quotes around the term "intangible". We believe that these factors are not in fact as impossible to quantify (albeit a bit fuzzily) as they are perceived to be. Measurement of intangibles would necessarily be a rough proxy of reality, but to account for them would be vastly better than ignoring them completely. For example, by analyzing the "time to respond" of emails over time one could potentially determine closeness of a person to a manager or the work ethic of the person being measured (correlate the results to grade/rank of the email sender - a proxy for power, the results would be even more interesting). In an utopian fully transparent organization, a staff evaluation could have a closeness score of the manager attached to their evaluation.
Our goal in writing this series is to trigger an open and informal discussion on "assessment", its use and its potential misuse. We plan to propose principles for assessment models, get some ideas going and maybe if we are lucky enough, start a debate. After the series on assessment we plan to delve into the assessment of organizational transparency - and that will address the kind of issues that you are bringing up. Please stay tuned.