Assessment and Evidence Audit

Assessment and Evidence Audit
When the Instruction Is Not the Problem: A Hypothesis About Methodology
Patricia Silva
Lamar University · EDLD 5315 — Assessing Digital Learning and Instruction
Dr. Kristi Meeuwse · October 2026
Note to the reader. The pages that follow open with material written for myself, before the assessed components begin. The opening note and the hypothesis essay that follows it are companion material: they record what I was reasoning about at this point in the program, so that I can return to them later and remember how I arrived here, not only where I landed. They are not part of the deliverable. A second heading marks where the assessed work begins.
Companion Material — Written for the Author's Own Record
Not part of the assessed submission. §1 The Hypothesis and §10 Justification of the Hypothesis belong here.
1. The Hypothesis
I began this audit believing the problem was instruction. I compared two sets of mathematics materials used with the same learner — one from his school and one from a supplementary program — expecting to find that one taught well and the other taught poorly. I did not find that. Both instruct clearly, both sequence the content correctly, and both give the learner the procedure and mark the steps. The instruction is not where the problem is.
The instruction is not the problem. The problem is that the instruction carries no meaning for this learner — and the examination of that gap is what methodology actually means.     
The word is not decorative. Methodology is not a synonym for method. Meta means beyond; hodos means the road; logos means the examination of the road. What this learner receives, in both settings, is method without methodology: a well-taught route that no one examines for him, at this moment.
The hypothesis, stated plainly: a learner can be correctly instructed, work enormously, complete everything, and still not progress — because the route is never examined for meaning. This is a hypothesis, not a finding. The evidence available can suggest it and cannot establish it, and the final section states where that boundary runs.
10. Justification of the Hypothesis
I am still examining this, and I have not reached a conclusion. What follows is the reasoning that produced the hypothesis above — a line of thinking about the word itself, prompted by what I saw.
I went looking for what methodology means, and the word came apart: meta, beyond; hodos, the road; logos, the examination of the road. A method is the road without its examination. Methodology is the road and its examination together. This is where my question began.
Then I looked at what this learner has done. The photograph below is a portion of the material completed across two settings in the first term of 2026. Every task is complete. Not one lesson is skipped. The teacher's verification appears on the pages, in her own hand. The volume of work is what made me stop, because I could not reconcile it with his result in the same subject.

Figure 6. A portion of the learner's completed material, first term of the 2026 school year. Reproduced with the written permission of the learner's mother; no institution is identified and the learner's name is withheld.
Worksheets, workbooks and handwritten exercises, all completed. The volume is not an impression; it is what the term actually contains.
So I ask the question I cannot yet answer. If he has walked the road this many times and the road has been verified as correct every time, why has neither the walking nor the verification produced understanding? The volume is not the problem. The instruction is not the problem. What is missing is not more walking — it is the examination of the road for this learner, at this moment, which is what methodology means and which, in both settings, no one is doing.
The final piece came from a conversation. During an informal family meal, I asked this learner what multiplication was for — what he would use it for in his own life. The learner is a family member; his name is withheld for his privacy. He could perform every operation correctly and complete every page. He could not give me one example. He knows the point. He does not know the road.
That conversation is evidence of a particular kind, and it belongs in this audit for a reason. Critical reflection on one's own practice is not an ornament added to practitioner inquiry; it is one of the features that defines it as systematic inquiry at all (Mertler, 2024). The learner's inability to give one example is what converted a suspicion about instruction into a question about the route.
The context for this observation matters. In the final four modules of the program I stopped working in order to concentrate on my studies, and I came to Brazil to spend a season with my family. That is how I was able to sit with this learner, watch him play, and talk with him at length. What follows was observed in that period, in his home, in Portuguese, and is reproduced here in the language in which it was said, with the English translation beneath it.
The observation session
He spends hours playing video games, building structures, and talking with friends online, while his mother works. He finishes his homework quickly and completely because finishing it is what earns him the game. So I sat beside him and watched him play, and we talked while he built.
— Olha, isso aqui não cabe aqui, porque é grande.
— Look, this one doesn't fit here, because it's too big.
— Mas isso aqui é quantas vezes o tamanho desse?
— But how many times the size of this one is that one?
— Eu acho que é umas duas.
— I think about two.
— Você não vê a tabuada aqui no que você está fazendo? Aqui está cheia de tabuada.
— Don't you see the multiplication table in what you're doing? There is multiplication all over this.
— Não, não tem não. Não tem dois vezes dois.
— No, there isn't. There's no two times two.
— Tem sim. Quantos blocos tem aqui?
— There is. How many blocks are here?
— Quatro.
— Four.
— Então, se você precisa preencher esse espaço aqui, isso aqui é tabuada.
— So if you need to fill this space here, this is the multiplication table.
— Ah, aqui tem tabuada. Eu não sabia.
— Oh, there is multiplication here. I didn't know.
He was doing mathematics and did not know it. He had measured the wall and the furniture against each other, answered in proportion, and counted the blocks he needed — and none of it connected to the subject he studies every day in two institutions. The knowledge is not missing. The connection between the knowledge and his own life is missing.
If methodology means meta, hodos, and logos — beyond, the road, and the examination of the road — then where is the road when everything is fragmented? What this learner receives, in both institutions, is not a road. It is a set of fragments that do not touch each other, and no one is examining them for him.
I brought the observation to his mother. Her words were:
— Nossa, que falta de conexão com o mundo real.
     — Wow, what a lack of connection with the real world.
She also described what happened after the pandemic. In 2019 classes moved online: the teacher wrote on the board at school and the children copied it at home, on a computer screen. When they returned to the classroom, she said, the children could not write and could not do mathematics. For this learner, writing recovered. Mathematics did not.
He engaged. Engagement measures the point of the activity. Engagement is not learning.
BEGIN: Assessment and Evidence Audit (Assessed Components 1–9)
2. Context and Problem of Practice
"Oftentimes, educators mistakenly equate educational problems with problems of practice… By definition, this would not be considered a problem of practice because it is simply too large and too complex to be investigated" (Mertler, 2024, p. 3).
That distinction makes this a problem of practice: an achievement gap cannot be investigated, a single learner's evidence trail can. My narrow, practice-based question uses existing records (Mertler, 2024).
This professional context is described without identifying details.
What is happening is measurable. He completes every assigned exercise correctly, at home and in a supplementary program. His first-term 2026 mathematics result is 6.2/10, against 9.0 in Spanish.
Who is affected: the learner, whose effort is not becoming progress; the teacher, accountable against an indicator unable to distinguish completion from understanding; and the family.
What is known: work is completed in two settings; the result does not move. Everyone, including me, assumes more or better instruction is the answer. What needs examination: whether the difficulty lies in teaching.
3. How the Problem Presented Itself
The problem entered my thinking in three stages; the third produced the hypothesis.
The first was earlier in this program: a student struggling with an assigned Spanish activity spoke Spanish fluently within minutes of ordinary conversation. Recorded performance and demonstrated capability did not match.
The second was reading CAST UDL Guidelines 3.0 during this course. Guideline 9 describes recognising emotions and choosing how to proceed. My question remained: are we seeing understanding, or the conditions allowing learners to act on it?
The third was the comparison: I set out to examine instruction and finished by eliminating it. That left the hypothesis.
This audit's method began with the photograph below. On 10 October I photographed the electrical pole across the street and applied filters. The pole did not change; what I could see did. A platform would have recorded no clicks.
Figure 1. The electrical pole, photographed in São Paulo, 10 October 2026. Personal archive of the author.
The object did not change. Zero digital activity was recorded while I reasoned.
Figure 2. The same photograph, reinterpreted through visual filters, 12 October 2026. Personal archive of the author.
The same evidence, read differently by a human interpreter. Activity records cannot say which reading is happening.
4. Focused Question
For a learner receiving instruction in mathematics across two settings, what do existing records of completed work reveal about demonstration of understanding over time — and what can they not establish?
A negative answer remains valid. The question asks not whether he is learning, but what the records can establish.
5. Existing Evidence Sources
Three different sources available to me can speak to each other.
Source 1 — A continuous series of completed handwritten exercises
What it is: many handwritten exercises across successive levels, at home and in a supplementary program.
Provides: his responses, corrections, and level sequence.
Represents: his output, not his understanding.
Collected: during ordinary study; the program uses it for advancement.
Relevance: the longest continuous record of his work.
Source 2 — A comparison of two instructional designs
What it is: two institutions' mathematics materials, with the same purpose.
Provides: what each teaches, and in what order.
Represents: instructional design, not the learner.
Collected: examined directly, as issued.
Relevance: eliminated my original explanation — both teach clearly; design is not failing him.
Source 2b — The learner's own account
In the interview, he was asked whether his mathematics connected to his on-screen activity.
He could not, although he measured and answered in proportion while building.
He performed mathematical operations without recognising them; he did not know multiplication was present.
Relevance: the only source carrying his account; it shows the gap most directly.
Source 3 — Summary assessment records
What it is: periodic subject results, first term of 2026.
Provides: one number per subject.
Represents: an end point.
Collected: through each institution's own assessment.
Relevance: the evidence that carries consequences and travels furthest.
Figure 3. Anonymous assessment record, first term of the 2026 school year. Reproduced with the written permission of the learner's mother. The learner's name is withheld for his privacy; no institution is identified.
Mathematics 6.2; Spanish 9.0. The same learner, term and effort — yet the subject receiving the most instruction has the lowest result.
Figure 4. Anonymous instructional material, first term of the 2026 school year. Reproduced with the written permission of the learner's mother.
A completed page, every exercise correct: evidence of what the instruction produced, and of what it did not ask.
Figure 5. The lesson preceding the completed page. Reproduced with the written permission of the learner's mother.
The procedure, taught clearly, with steps marked. The instruction is not the problem. Nothing asks what it is for.
6.2 When additional information would be needed
Some points permit no evaluation at all.
Source 1 — completed work. Credibility cannot change without knowing whether work was independent, assisted, or under correction. Nothing records which.
Source 2 — instructional designs. The design is fully available and directly examined. Establishing either design's delivery as written requires observing instruction, not reading material.
Source 3 — summary assessment records. Assessment construction, administration conditions and who marked them are undisclosed. The number can therefore be reported but not judged for fairness.
7. Analysis Across the Evidence
Together the sources produce what none produces alone: a finding about the route, not about the learner.
Source 2 eliminates the obvious explanation: both teach clearly; instruction is not what separates him from progress.
Source 1 shows he performs what is asked; Source 3 shows unchanged results.
Together they show instruction not failing, and progress at a rate it does not explain.
However, the collection leans on output. Completion, correctness and results measure production; outcomes remain inferred — and the inference is in question.
Nevertheless, the main absence is a perspective, not a document. Reported engagement predicts learning weakly and inconsistently (Hiver, 2025). Activity versus understanding is a documented measurement problem, not my preference.
6. Evaluation of Each Source
My ratings follow the Evidence-Quality Criteria and Rating Guide; reasoning matters more than rating.
6.1 What the three ratings have in common
Each source does its job: work records completion; designs record teaching; results record a summary.
None can move from its record to a claim about learning.
The risk is not an inaccurate record, but an accurate record carrying an unsupported claim.
Accessibility is the criterion I can least judge. Universal Design for Learning asks for multiple means of engagement, representation, and expression (CAST, 2024). If writing is the only route, evidence says more about it than the learner.
Ethics: not only privacy, but using the right data for the right purpose and accepting interpretation's consequences (Mandinach & Jimerson, 2021).
The authors describe a similar case: a platform reported a student's reading far below her demonstrated level, contradicting teacher observations.
Here the conflict remains: sources record something about the learner, not the circumstances.
Fairness is what this collection can least satisfy. Equitable assessment lets learners show knowledge; ranking rather than teaching has lasting consequences for identity and equity (Heritage & Wylie, 2018). A completed page offers that chance only if he can use writing. Nothing establishes this.
8. Gaps and Missing Perspectives
Five gaps each carry a specific consequence.
Evidence of meaning. Nothing asks him to explain, apply or transfer; a completed page cannot distinguish understanding from reproduction.
1.The learner's own perspective. His account now shows the gap most directly.
2.Evidence of the medium. Nothing records hand or keyboard. They engage different cognitive processes; longhand is associated with better conceptual performance even when typing produces more words (Mueller & Oppenheimer, 2014; Van der Weel & Van der Meer, 2024).
3.Evidence of conditions. Records show output, never time, fatigue, correction practices, or assistance.
4.Any examination of the route. No source asks whether the route fits this learner now.
5.The purpose is not more measurement: unearth underperformance's root causes and centre the learner's long-term good (Mandinach & Jimerson, 2021).
9. Recommendations
The recommendations follow practitioner inquiry: identify the problem, specify its scope, and choose appropriate data before drawing conclusions (Mertler, 2024).
Applying the required labels:
Source 1 — completed work: Retain and Supplement. Keep his only continuous output record unchanged, adding periodic learner check-ins.
Source 2 — instructional designs: Retain. It eliminated the obvious explanation, removing instruction from possible causes.
Source 3 — summary assessment records: Retain. It answers a legitimate institutional question; the problem is misuse, not its existence.
9.1 Steps that would strengthen the evidence base
Add brief, regular learner check-ins to Source 1: what was easy, difficult, and usable outside the page. This missing perspective costs nothing.
Confirm written guardian consent for professional review, withholding the learner's name and institutional identification.
Finally, a systematic review found data literacy training prioritised analysis over application, while effective approaches included structured decision-making (Sandoval-Ríos et al., 2025).
11. Overall Conclusion
What the evidence may suggest is narrow: instruction does not separate him from progress; two independent designs teach clearly, yet his result does not move.
What it can responsibly establish is narrower: what is recorded. No source can show what he understands. It cannot establish that he is not learning, that either institution is responsible, or that another method would work.
The first two sources are most useful: one accumulates, one eliminated an explanation. The third demands caution: it says least and travels furthest.
The audit refined my understanding: I set out to examine instruction and finished by eliminating it as the cause. What remains is the route, and examining that route for one learner at one moment is what methodology means.
Disclosure of AI Use
Prepared under AI Use Level 1 (Preparation Only). This submission was written by me: I selected and evaluated the evidence sources, applied the evidence-quality criteria, assigned and justified every rating, identified the gaps and missing perspectives in my context, and wrote every sentence of the text. I used generative AI only to review the meaning of the evidence-quality criteria and to check my draft against the assignment requirements, where it functioned as a completeness check by pointing out places where a required element appeared to be missing or incomplete; every correction made in response to those observations was located, written, and decided by me.
References
CAST. (2024). CAST Universal Design for Learning Guidelines version 3.0. https://udlguidelines.cast.org/
Heritage, M., & Wylie, E. C. (2018). Reaping the benefits of assessment for learning: Achievement, identity, and equity. ZDM Mathematics Education, 50(4), 729–741. https://doi.org/10.1007/s11858-018-0943-3
Hiver, P. (2025). From task motivation to L2 learning: Understanding links through learners' task engagement. TESOL Quarterly. Advance online publication. https://doi.org/10.1002/tesq.3410
Mandinach, E. B., & Jimerson, J. B. (2021). Data ethics in education: A theoretical, practical, and policy issue. Studia Paedagogica, 26(4), 9–26. https://doi.org/10.5817/SP2021-4-1
Mertler, C. A. (2024). Action research: Improving schools and empowering educators (7th ed.). SAGE Publications.
Mueller, P. A., & Oppenheimer, D. M. (2014). The pen is mightier than the keyboard: Advantages of longhand over laptop note taking. Psychological Science, 25(6), 1159–1168. https://doi.org/10.1177/0956797614524581
Sandoval-Ríos, F., Gajardo-Poblete, C., & López-Núñez, J. A. (2025). Role of data literacy training for decision-making in teaching practice: A systematic review. Frontiers in Education, 10, Article 1485821. https://doi.org/10.3389/feduc.2025.1485821
Van der Weel, F. R., & Van der Meer, A. L. H. (2024). Handwriting but not typewriting leads to widespread brain connectivity: A high-density EEG study with implications for the classroom. Frontiers in Psychology, 14, Article 1219945. https://doi.org/10.3389/fpsyg.2023.1219945