« Back to Assessment Instruments

Hall’s Levels of Use Scale

Qualitative/Quantitative:
The assessment instrument uses quantitative and/or qualitative data

Qualitative
Quantitative

Type of Instrument:
The type of the assessment instrument

Individual Interview

Number of Items:
Number of items in the assessment instrument

27 branches depending on previous answers

Subscale Information:
Names of each of the subscales and the number of items for each of the subscales

User (8)
LoU V Probes (7)
Not a User (8)
Past User (3)

Language Availability:
Language(s) in which the assessment instrument is available

English

Brief Description:
Brief summary description of assessment instrument

The Levels of Use is one of three components that make up the diagnostic dimensions of the Concerns-Based Adoption Model. Determining Levels of Use is done with the use of a series of questions that a facilitator asks a staff member. The Levels of Use interview protocol enables educators to know not only the extent to which staff are using a new program but also whether individuals are at a beginning stage—still working through the challenges associated with grasping the program—or at a more advanced level, where he or she has expertise in using the program. With this knowledge, leaders can provide the support necessary to help all staff members use the program fully and effectively.

Citing Literature - Development/Original:
Reference for publication describing the development of the assessment instrument

Hall, G. E., Dirksen, D. J., & George, A. A. (2013). Measuring implementation in schools: Levels of use. Southwest Educational Development Laboratory.

Kim, C., Kim, M. K., Lee, C., Spector, J. M., & DeMeester, K. (2013). Teacher beliefs and technology integration. Teaching and Teacher Education, 29, 76–85. https://doi.org/10.1016/j.tate.2012.08.005.

Citing Literature - Empirical Use/Application:
Reference for publications on the application of the assessment instrument

Rusmawaty, D., Hermagustiana, I., & Sunardi, S. (2023). An exploration of EFL teachers’ perceptions and experiences of e-learning implementation through the concern-based adoption model. Journal of English as A Foreign Language Teaching and Research, 3(1), 31–43. https://doi.org/10.31098/jefltr.v3i1.1439.

Website:
Website providing access to and/or describing the assessment instrument

Levels of Use │Concerns-Based Adoption Model

Attachments:
Related files uploaded (instrument if directly available) including descriptions for each

Instrument.pdf (445.02 KB) - Instrument Attachment
Measuring Implementation in Schools: LEVELS OF USE.pdf (2.87 MB) - Supporting Document
Definitions.pdf (2.46 MB) - Supporting Document
Rating Sheet and Rating Guide.pdf (478.95 KB) - Supporting Document
SAS Syntax for Reliability Analysis.pdf (445.33 KB) - Supporting Document

Version:
Number/name of the most recent version of the assessment instrument

Not Found

Related Instrument:
Indicate if assessment instrument is related to another instrument in the repository.

Not Found

Implementation Science Considerations

Intended Focus

Psychometric Properties

Pragmatic Properties

Assessment Instrument Characteristics

Characteristic Title	Characteristic Description	Data Type
Brief Description	Brief summary description of assessment instrument	Open Text
Citing Literature - Development/Original	Reference for publication describing the development of the assessment instrument	Open Text
Citing Literature - Empirical Use/Application	Reference for publications on the application of the assessment instrument	Open Text
Website	Website providing access to and/or describing the assessment instrument	Open Text
Attachments	Related files uploaded (instrument if directly available) including descriptions for each	Open Text
Version	Number/name of the most recent version of the assessment instrument	Open Text
Qualitative/Quantitative	The assessment instrument uses quantitative and/or qualitative data	Multi Select Qualitative Quantitative See Values
Type of Instrument	The type of the assessment instrument	Multi Select Checklist Documentation Guide EMR Form Ethnographic Assessment Focus Group Interview Individual Interview Observation Guide Periodic Reflection Reporting Tempate Survey Template/Worksheet Undisclosed online tool Worksheet See Values
Levels of Data Collection	The level(s) from which the assessment instrument collects data	Multi Select Community Implementer Individual (Patient, Community Member) Organizational Policy System See Values
Language Availability	Language(s) in which the assessment instrument is available	Multi Select Albanian Chinese Danish Dutch English French German Italian Portuguese Spanish Swedish See Values
Constructs Assessed	Constructs assessed by the assessment instrument (linked to constructs included in the D&I models webtool)	Relationship
Theories, Models, Frameworks Assessed	The D&I TMFs relevant for the assesment instrument based on constructs assessed	Relationship
Implementation Outcomes	The relevance of the assessment instrument to various implementation outcomes	Multi Select Acceptability Adaptation Adoption Appropriateness Effectiveness Feasibility Fidelity Implementation Implementation Cost Penetration Reach Sustainment used with IAM and FIM to assess APPROPRIATENESS and FEASIBILITY See Values
Implementation Strategies	The implementation strategy/ies evaluated by the assessment instrument	Multi Select Access new funding Adapt and tailor to context Alter incentive/allowance structures Alter patient/consumer fees Assess for readiness and identify barriers and facilitators Audit and provide feedback Build a coalition Capture and share local knowledge Change infrastructure Change physical structure and equipment Change service sites Conduct educational meetings Conduct educational outreach visits Conduct local consensus discussions Conduct local needs assessment Conduct ongoing training Create a learning collaborative Create or change credentialing and/or licensure standards Develop a formal implementation blueprint Develop academic partnerships Develop educational materials Develop implementation blueprint Develop resource sharing agreements Develop stakeholder inter-relationships Develop stakeholder interrelationships Distribute educational materials Engage consumers Identify and prepare champions Identify early adopters Inform local opinion leaders Make billing easier Make training dynamic Mandate change Model and simulate change Not Found Obtain and use patients/consumers and family feedback Obtain formal commitments Organize clinician implementation team meetings Place innovation on fee for service lists/formularies Prepare patients/consumers to be active participants Promote adaptability Promote network weaving Provide clinical supervision Provide interactive assistance Provide local technical assistance Provide ongoing consultation Purposely reexamine the implementation Recruite, designate, and train for leadership Revise professional roles Shadow other experts Stage implementation scale up Support clinicians Tailor strategies Train and educate stakeholders Use advisory boards and workgroups Use an implementation advisor Use capitated payments Use data experts Use data warehousing techniques Use evaluative and iterative strategies Use mass media Use of evaluative and iterative strategies Use other payment schemes Use train-the-trainer strategies Utilize financial strategies Visit other sites Work with educational institutions See Values
Intended Priority Population	Intended priority population from whom data are collected using the assessment instrument	Multi Select Administrator Clinician Community Members/Patients Employer Public Health Practitioner Researcher/Evaluator Teacher/Trainer See Values
Intended Priority Setting	Intended priority setting in which the assessment instrument is used	Multi Select Clinical Inpatient Clinical Outpatient Community Organization Public Health Agency Residential Care School Workplace See Values
Phase of Implementation Process	Phase of implementation process when the assessment instrument can be used	Multi Select Implementation Pre-Implementation Sustainment See Values
Policy	Assessment instrument is relevant to policy	Multi Select Clinical Practice Healthcare Financing/Reimbursement Public (Laws, Regulations) See Values
Equity Relevant	Assessment instrument includes at least one equity-focused component	Multi Select Yes See Values
Uses Secondary data	Assessment instrument relies on a secondary data source	Multi Select Yes See Values
Time to Administer	The amount of time required to complete the assessment instrument	Open Text
Number of Items	Number of items in the assessment instrument	Open Text
Subscale Information	Names of each of the subscales and the number of items for each of the subscales	Open Text
Scoring	The assessment instrument produces a composite score	Multi Select No Yes See Values
Norms	Measures of central tendency and distribution for the total score are based on small, medium, large sample size	Multi Select Yes See Values
Responsiveness	The ability of the assessment instrument to detect change over time (i.e., sensitivity to change or intervention effects).	Multi Select Yes See Values
Cost	Cost associated with access to assessment instrument (Some insturments might require login.)	Multi Select Cost Free See Values
Literacy	Readability of the items reported on.	Multi Select Yes See Values
Interpretation	Expertise needed for interpretation of data is reported.	Multi Select No Yes See Values
Training	Expertise needed to use the assessment instrument is reported	Multi Select No Yes See Values
Resources Required to Administer	Resources needed to administer the assessment instrument (FTE for data collector, equipment, etc.)	Multi Select High None/Low See Values
User Guidance	Guides are provided to support administration of assessment instrument/data collection, and/or analysis of data from the assessment instrument, and/or interpretation of data, and/or action/decision on how to use data	Multi Select Guidance for Action/Decision Guidance to Administer Guidance to Analyze Guidance to Interpret See Values
Obtrusiveness	Degree of intrusion the participants will experience because of the data collection when using the assessment instrument (e.g., assessment instruments that rely on use of secondary data or automated data will be less obtrusive)	Multi Select High: Observation and syncronous collection of data Low: Automated collection of data or secondary data Medium: Asyncronous collection of data See Values
Interactivity	Data collection and/or result generation involves interactive components.	Multi Select Adaptive Instrument Generative Outcome See Values
Related Instrument	Indicate if assessment instrument is related to another instrument in the repository.	Multi Select Acceptability of Intervention Measure CFIR Interview guide webtool (2009) Cost of Implementing New Strategies (COINS) Diagnosis Related Group (DRG) Policy Survey 2 FIM Feasibility of Intervention Measure Framework for Reporting Adaptations and Modifications to Evidence-based Implementation Strategies (FRAME-IS) IAM Implementation Checklist Implementation Leadership Scale Intervention Appropriateness Measure Iterative RE-AIM Local Wellness Policy Implementation Checklist Local Wellness Policy Survey Measure of Innovation-Specific Implementation Intentions (MISII) NIATx implementation strategies Not Found PRISM PRISM survey eScreening PSAT PSAT Short Program Sustainability Tool (PSAT) RAPICE Instruments Readiness Thinking Tool - Observation Guide Readiness Thinking Tool - Survey Research Engagement Survey Tool (REST) Research Engagement Survey Tool (REST) 9 Item Condensed Version Some domains related to PRISM Stages of Implementation Completion (SIC) Tool The Evidence-based Practice Attitude Scale (EBPAS) The Implementation Citizenship Behavior Sclae (ICBS) The Implementation Climate Scale (ICS) The Implementation Leadership Scale UES- Short Form User Engagement Survey (UES) User Engagement Survey (UES) - Short Visual output similar to precis 2-PS See Values
Validity	The extent to which an instrument measures what it is intended to measure accurately.	Multi Select Concurrent Validity (Definition) Convergent Validity (Definition) Cross-Cultural Validity (Definition) Discriminant Validity (Definition) Known Group Validity (Definition) Predictive Validity (Definition) Unspecified Validity See Values
Reliability	The extent to which results are consistent results over time, across raters, across settings, or across items intended to measure the same thing.	Multi Select Yes See Values
Factor Analysis	A statistical method that uses the correlation between observed variables to identify common factors.	Multi Select Yes See Values

Comments

Hall’s Levels of Use Scale

Qualitative/Quantitative: The assessment instrument uses quantitative and/or qualitative data

Type of Instrument: The type of the assessment instrument

Number of Items: Number of items in the assessment instrument

Subscale Information: Names of each of the subscales and the number of items for each of the subscales

Language Availability: Language(s) in which the assessment instrument is available

Brief Description: Brief summary description of assessment instrument

Citing Literature - Development/Original: Reference for publication describing the development of the assessment instrument

Citing Literature - Empirical Use/Application: Reference for publications on the application of the assessment instrument

Website: Website providing access to and/or describing the assessment instrument

Attachments: Related files uploaded (instrument if directly available) including descriptions for each

Version: Number/name of the most recent version of the assessment instrument

Related Instrument: Indicate if assessment instrument is related to another instrument in the repository.

Implementation Science Considerations

Constructs Assessed: Constructs assessed by the assessment instrument (linked to constructs included in the D&I models webtool)

Theories, Models, Frameworks Assessed: The D&I TMFs relevant for the assesment instrument based on constructs assessed

Implementation Outcomes: The relevance of the assessment instrument to various implementation outcomes

Implementation Strategies: The implementation strategy/ies evaluated by the assessment instrument

Phase of Implementation Process: Phase of implementation process when the assessment instrument can be used

Intended Focus

Levels of Data Collection: The level(s) from which the assessment instrument collects data

Intended Priority Population: Intended priority population from whom data are collected using the assessment instrument

Intended Priority Setting: Intended priority setting in which the assessment instrument is used

Policy: Assessment instrument is relevant to policy

Equity Focus:

Psychometric Properties

Scoring: The assessment instrument produces a composite score

Norms: Measures of central tendency and distribution for the total score are based on small, medium, large sample size

Responsiveness: The ability of the assessment instrument to detect change over time (i.e., sensitivity to change or intervention effects).

Validity: The extent to which an instrument measures what it is intended to measure accurately.

Reliability: The extent to which results are consistent results over time, across raters, across settings, or across items intended to measure the same thing.

Factor Analysis: A statistical method that uses the correlation between observed variables to identify common factors.

Pragmatic Properties

Time to Administer: The amount of time required to complete the assessment instrument

Secondary Data:

Cost: Cost associated with access to assessment instrument (Some insturments might require login.)

Literacy: Readability of the items reported on.

Interpretation: Expertise needed for interpretation of data is reported.

Training: Expertise needed to use the assessment instrument is reported

Resources Required to Administer: Resources needed to administer the assessment instrument (FTE for data collector, equipment, etc.)

User Guidance: Guides are provided to support administration of assessment instrument/data collection, and/or analysis of data from the assessment instrument, and/or interpretation of data, and/or action/decision on how to use data

Obtrusiveness: Degree of intrusion the participants will experience because of the data collection when using the assessment instrument (e.g., assessment instruments that rely on use of secondary data or automated data will be less obtrusive)

Interactivity: Data collection and/or result generation involves interactive components.

Assessment Instrument Characteristics

Comments

Qualitative/Quantitative:
The assessment instrument uses quantitative and/or qualitative data

Type of Instrument:
The type of the assessment instrument

Number of Items:
Number of items in the assessment instrument

Subscale Information:
Names of each of the subscales and the number of items for each of the subscales

Language Availability:
Language(s) in which the assessment instrument is available

Brief Description:
Brief summary description of assessment instrument

Citing Literature - Development/Original:
Reference for publication describing the development of the assessment instrument

Citing Literature - Empirical Use/Application:
Reference for publications on the application of the assessment instrument

Website:
Website providing access to and/or describing the assessment instrument

Attachments:
Related files uploaded (instrument if directly available) including descriptions for each

Version:
Number/name of the most recent version of the assessment instrument

Related Instrument:
Indicate if assessment instrument is related to another instrument in the repository.

Constructs Assessed:
Constructs assessed by the assessment instrument (linked to constructs included in the D&I models webtool)

Theories, Models, Frameworks Assessed:
The D&I TMFs relevant for the assesment instrument based on constructs assessed

Implementation Outcomes:
The relevance of the assessment instrument to various implementation outcomes

Implementation Strategies:
The implementation strategy/ies evaluated by the assessment instrument

Phase of Implementation Process:
Phase of implementation process when the assessment instrument can be used

Levels of Data Collection:
The level(s) from which the assessment instrument collects data

Intended Priority Population:
Intended priority population from whom data are collected using the assessment instrument

Intended Priority Setting:
Intended priority setting in which the assessment instrument is used

Policy:
Assessment instrument is relevant to policy

Scoring:
The assessment instrument produces a composite score

Norms:
Measures of central tendency and distribution for the total score are based on small, medium, large sample size

Responsiveness:
The ability of the assessment instrument to detect change over time (i.e., sensitivity to change or intervention effects).

Validity:
The extent to which an instrument measures what it is intended to measure accurately.

Reliability:
The extent to which results are consistent results over time, across raters, across settings, or across items intended to measure the same thing.

Factor Analysis:
A statistical method that uses the correlation between observed variables to identify common factors.

Time to Administer:
The amount of time required to complete the assessment instrument

Cost:
Cost associated with access to assessment instrument (Some insturments might require login.)

Literacy:
Readability of the items reported on.

Interpretation:
Expertise needed for interpretation of data is reported.

Training:
Expertise needed to use the assessment instrument is reported

Resources Required to Administer:
Resources needed to administer the assessment instrument (FTE for data collector, equipment, etc.)

User Guidance:
Guides are provided to support administration of assessment instrument/data collection, and/or analysis of data from the assessment instrument, and/or interpretation of data, and/or action/decision on how to use data

Obtrusiveness:
Degree of intrusion the participants will experience because of the data collection when using the assessment instrument (e.g., assessment instruments that rely on use of secondary data or automated data will be less obtrusive)

Interactivity:
Data collection and/or result generation involves interactive components.