The 15 year programme, research method and findings are described in the downloadable paper:
Walker developed a consistent, standardised assessment method for measuring steering cognition, which he has used repeatedly since 2002 across large, diverse, independent cohorts of up to 6,500 participants /study to produce commensurate, replicable data.
His study populations have been drawn from large primary and secondary schools as well as some university and adult cohorts. School populations allow cross-correlations against known factors such as age, gender, ethnicity, CAT score as well as intra-school social, group, cohort, welfare and academic data.
The consistency of this assessment method, data model model and standardised conditions makes the steering cognition database highly reliable. Item, scale and factor structures have been repeatedly tested. As of 2015, the database contains more than 80,000 completed datasets enabling each of the seven steering cognition model factors to be analysed against more than 11,000 factor scale scores. It grows each month as schools repeat assessments.
"This is a high quality scientific database, which is the result of a standardly applied data model, consistent testing conditions, repeated candidate measures and standardised responses. As a boinformatician used to working on chomatin influence on gene expression from the Global Genome Initiative, analysis of data in the steering cognition database is relatively simple because both participant and response variables are so tightly controlled and known. Measurement error effects are small and correlation effects have proved to be replicable."
Dr Rosa Karlic, Bioinformatics Department, University of Zagreb, project data analysis
Research claims that are made are based upon repeated results, reproduced consistently, across multiple, large, independent and diverse populations and over multiple years.
The technology, data model and method of testing has been subjected to 80,000 trials with 11,5 00 candidates over a 14 year period within standardised conditions.
Technology design, development and tesing
Walker designed the research technology to be immune to collecting algorithmic data.
4.1. To avoid inadvertently collecting algorithmic cognitive data along with non-algorithmic data, Walker first designed as assessment which involved no computational calculation, deduction or other algorithmic process. Candidates opt for multiple-choice answers which would not be aided by prior knowledge of computational ability. In this way, the risk of algorithmic cognitive processing leaking into the assessment is removed.
4.2. Second, Walker exploits this correlation between heuristic cognition and the imagination. Using an imagination exercise, in which the candidate imagined performing a learning task Walker activates and then assesses the first-person cognitive response of the candidate rather than an abstracted response. In this way, associative processing rather than algorithmic processing is engaged.
4.3. Third, candidates complete the exercise without formal guidance as to the shape, structure, kind or approach to take to the imaginative task. By cueing up an undefined ‘white world’ in the candidate’s imagination, Walker removes the potential priming biases about what kind of answer was required, in order to overcome Stanovich's criticism of current ‘heuristic measures’ that are, in fact, closed and prescriptive.
4.4. Fourth, Walker standardises the candidate’s white world imagination against a data model consisting of 7 validated factors. Standardised scoring consists of a set of multiple choice questions on a Likert scale. Walker measures the candidate’s response to a series of real-world and unpredicted scenarios. In this way, the capacity to adjust and regulate heuristic cognition in response to an epistemically varied set of scenarios is measured against a baseline score. Walker refers to this 7 factor model of steering cognition as CAS state – cognitive affective social state.
4.5. Fifthly, Walker has conducted a multi-year programme of experiments with secondary school students across 20 schools, in which he compared student heuristic CAS scores with academic outcomes and general intelligence (algorithmic cognition measured by CAT or MiDYIS). In so doing, Walker has been able to identify that there is statistically negible relationship between CAT and CAS in large populations.