Reliability and validity are different
Reliability concerns how consistently a test measures; validity concerns the evidence supporting what a score means and how it is used. A consistent result is not automatically a correct interpretation. When asking “Is this test accurate?”, first name the questionnaire, version, language, population and decision you want it to support.
Reliability concerns whether results are stable: for example, whether similar items produce relatively consistent results under similar conditions, or whether a person's results are broadly similar when they answer again within a short period. Validity concerns whether evidence and theory support the interpretation of assessment scores for a specific use: for example, whether a career interest assessment observes interests rather than abilities, or whether personality dimensions interpret behavioral tendencies rather than make moral judgments.
An assessment can be fairly stable without necessarily being valid; it may also have a clear purpose while its results lack stability. Reliability and validity must therefore be considered separately.
Common questions about reliability
Reliability is not a single indicator. Common discussions include internal consistency, test-retest stability and scoring consistency.
Internal consistency concerns whether items within the same dimension broadly observe similar content. Test-retest stability concerns whether results remain relatively stable after a period of time. Scoring consistency is more relevant when human scoring is involved.
Look for reliability estimates tied to the named instrument, version, sample and testing conditions. Published scoring formulas, response-quality flags and evidence for the general model are not substitutes for those estimates. This page does not inventory the evidence for every FermatMind tool.
Common questions about validity
Validity also has different aspects. Content validity concerns whether items cover a reasonable range of the intended construct. Structural evidence concerns whether the result structure supports the expected interpretation of scores. Criterion-related validity concerns whether results have a reasonable relationship with external indicators.
For example, a career interest assessment claiming to interpret work environment preferences should organize its content around activity preferences, environment preferences and career exploration, rather than suggest that it can judge abilities or predict high-risk career outcomes.
Specific validity, sample and norm information must be checked separately by tool, version, language and intended use. This page does not compile evidence for all tests and does not treat published scoring or norm status explanations as validity validation.
Why norms and comparison groups matter
Some results require a reference group for interpretation. A score alone is not necessarily meaningful: what matters is the population, version and language context against which it is compared.
Population rankings require a clear reference sample, language version, scope of applicability and conversion rules. Without these, do not infer percentiles or general rankings. Raw scores can still be read according to the tool's explicit scale and interpretation rules.
Error is not failure, but it must be acknowledged
Assessments involve error. It may arise from how questions are understood, the state in which answers are given, language differences, cultural experiences, recent events and model limitations. Acknowledging error helps users understand how to use results, rather than denying the value of assessment.
Treat results as reference clues, and continue testing them against real experiences and ongoing observations. Major decisions should not rely on a single assessment result.
FermatMind's current principles for stating evidence
Without public validation materials, FermatMind should not claim that a test is ‘absolutely accurate’, ‘absolutely authoritative’, ‘proven on a large scale’ or ‘able to predict outcomes’. Check each tool's version, scoring and norm status, and validation evidence for the relevant use separately. The absence of information from this page does not mean all public information is unknown.
For the general framework, see the Standards for Educational and Psychological Testing. For tool-specific reading, review the current Big Five, Holland Code or EQ-60 method notes. General standards do not validate a particular FermatMind questionnaire.
Frequently asked questions
Does high reliability mean a test is valid?
Not necessarily. Reliability indicates relatively stable results, while validity concerns whether evidence supports the expected interpretation and use of scores. The two must be considered separately.
What is internal consistency?
Internal consistency concerns whether items in the same dimension observe similar content. It is one aspect of reliability, rather than its entirety.
Can I still read results without norm data?
You may use them as references for self-observation, but should not interpret scores as general rankings or strict comparisons. Without a clear reference basis, do not supply percentiles or population rankings yourself.
Where should I check reliability and validity evidence for a FermatMind test?
Check the current method notes for the particular test, version and language. This overview does not list every tool’s evidence. A published formula or a provisional norm label is not a completed reliability or validity study; when the relevant sample or method is not established, keep that part of the interpretation uncertain.
Does error mean the test is useless?
No. Error indicates boundaries on interpretation. Assessments can still help raise questions and support reflection, but cannot be the sole basis for judgment.
