aea365.org
Open in
urlscan Pro
38.106.217.170
Public Scan
Submitted URL: http://feedproxy.google.com/~r/aea365/~3/mVG-rjWfees
Effective URL: https://aea365.org/blog/implementation-process-evaluation-strategies-to-identify-potential-areas-of-focus-by-tamara...
Submission: On August 24 via manual from US
Effective URL: https://aea365.org/blog/implementation-process-evaluation-strategies-to-identify-potential-areas-of-focus-by-tamara...
Submission: On August 24 via manual from US
Form analysis 3 forms found in the DOM
POST https://aea365.org/blog/wp-comments-post.php
<form action="https://aea365.org/blog/wp-comments-post.php" method="post" id="ast-commentform" class="comment-form">
<p class="comment-notes"><span id="email-notes">Your email address will not be published.</span> Required fields are marked <span class="required">*</span></p>
<div class="ast-row comment-textarea">
<fieldset class="comment-form-comment">
<div class="comment-form-textarea ast-col-lg-12"><label for="comment" class="screen-reader-text">Type here..</label><textarea id="comment" name="comment" placeholder="Type here.." cols="45" rows="8" aria-required="true"></textarea></div>
</fieldset>
</div>
<div class="ast-comment-formwrap ast-row">
<p class="comment-form-author ast-col-xs-12 ast-col-sm-12 ast-col-md-4 ast-col-lg-4"><label for="author" class="screen-reader-text">Name*</label><input id="author" name="author" type="text" value="" placeholder="Name*" size="30"
aria-required="true"></p>
<p class="comment-form-email ast-col-xs-12 ast-col-sm-12 ast-col-md-4 ast-col-lg-4"><label for="email" class="screen-reader-text">Email*</label><input id="email" name="email" type="text" value="" placeholder="Email*" size="30"
aria-required="true"></p>
<p class="comment-form-url ast-col-xs-12 ast-col-sm-12 ast-col-md-4 ast-col-lg-4"><label for="url"><label for="url" class="screen-reader-text">Website</label><input id="url" name="url" type="text" value="" placeholder="Website" size="30"></label>
</p>
</div>
<p class="comment-form-cookies-consent"><input id="wp-comment-cookies-consent" name="wp-comment-cookies-consent" type="checkbox" value="yes"> <label for="wp-comment-cookies-consent">Save my name, email, and website in this browser for the next time
I comment.</label></p>
<p><label for="subscribe-reloaded"><input style="width:30px" type="checkbox" name="subscribe-reloaded" id="subscribe-reloaded" value="yes"> Notify me of followup comments via e-mail. You can also
<a href="https://aea365.org/blog/?page_id=99999&srp=24171&srk=1477236ba1741ee52e21a312017db756&sra=s&srsrc=f">subscribe</a> without commenting.</label></p>
<p class="form-submit"><input name="submit" type="submit" id="submit" class="submit" value="Post Comment »"> <input type="hidden" name="comment_post_ID" value="24171" id="comment_post_ID">
<input type="hidden" name="comment_parent" id="comment_parent" value="0">
</p>
<p style="display: none;"><input type="hidden" id="akismet_comment_nonce" name="akismet_comment_nonce" value="efc58c15d9"></p><textarea name="ak_hp_textarea" cols="45" rows="8" maxlength="100" style="display: none !important;"></textarea>
</form>GET https://aea365.org/blog/
<form role="search" method="get" class="search-form" action="https://aea365.org/blog/">
<label>
<span class="screen-reader-text">Search for:</span>
<input type="search" class="search-field" placeholder="Search …" value="" name="s">
</label>
<input type="submit" class="search-submit" value="Search">
</form>GET https://aea365.org/blog
<form action="https://aea365.org/blog" method="get"><label class="screen-reader-text" for="cat">Posts Related to AEA Topical Interest Groups</label><select name="cat" id="cat" class="postform">
<option value="-1">Select Category</option>
<option class="level-0" value="192">Advocacy and Policy Change (66)</option>
<option class="level-0" value="208">Arts, Culture, and Museums (55)</option>
<option class="level-0" value="197">Assessment in Higher Education (34)</option>
<option class="level-0" value="196">Behavioral Health (41)</option>
<option class="level-0" value="198">Business, Leadership, and Performance (33)</option>
<option class="level-0" value="200">Cluster, Multi-site and Multi-level Evaluation (5)</option>
<option class="level-0" value="201">Collaborative, Participatory and Empowerment Evaluation (165)</option>
<option class="level-0" value="202">College Access Programs (12)</option>
<option class="level-0" value="1642">Community Development (6)</option>
<option class="level-0" value="875">Community Psychology (85)</option>
<option class="level-0" value="203">Costs, Effectiveness, Benefits and Economics (12)</option>
<option class="level-0" value="204">Crime and Justice (1)</option>
<option class="level-0" value="595">Data Visualization and Reporting (256)</option>
<option class="level-0" value="1398">Democracy and Governance (18)</option>
<option class="level-0" value="1396">Design and Analysis of Experiments (14)</option>
<option class="level-0" value="205">Disabilities and Underrepresented Populations (78)</option>
<option class="level-0" value="206">Disaster and Emergency Management Evaluation (6)</option>
<option class="level-0" value="207">Distance Education and Other Educational Technologies (12)</option>
<option class="level-0" value="158">Environmental Program Evaluation (78)</option>
<option class="level-0" value="209">Evaluation Managers and Supervisors (16)</option>
<option class="level-0" value="210">Evaluation Policy (8)</option>
<option class="level-0" value="211">Evaluation Use (65)</option>
<option class="level-0" value="212">Extension Education Evaluation (40)</option>
<option class="level-0" value="213">Feminist Issues in Evaluation (61)</option>
<option class="level-0" value="214">Government Evaluation (66)</option>
<option class="level-0" value="215">Graduate Student and New Evaluators (123)</option>
<option class="level-0" value="216">Health Evaluation (75)</option>
<option class="level-0" value="1606">Health Professions Education Evaluation and Research (7)</option>
<option class="level-0" value="217">Human Services Evaluation (9)</option>
<option class="level-0" value="218">Independent Consulting (165)</option>
<option class="level-0" value="219">Indigenous Peoples in Evaluation (40)</option>
<option class="level-0" value="220">Integrating Technology into Evaluation (119)</option>
<option class="level-0" value="221">Internal Evaluation (66)</option>
<option class="level-0" value="222">International and Cross-cultural Evaluation (138)</option>
<option class="level-0" value="1399">Latinx Responsive Evaluation Discourse (43)</option>
<option class="level-0" value="223">Lesbian, Gay, Bisexual & Transgender Issues (53)</option>
<option class="level-0" value="1395">Military and Veteran’s (23)</option>
<option class="level-0" value="224">Mixed Methods Evaluation (28)</option>
<option class="level-0" value="225">Multiethnic Issues in Evaluation (93)</option>
<option class="level-0" value="226">Needs Assessment (52)</option>
<option class="level-0" value="227">Nonprofits and Foundations Evaluation (115)</option>
<option class="level-0" value="228">Organizational Learning and Evaluation Capacity Building (97)</option>
<option class="level-0" value="229">Prek-12 Educational Evaluation (190)</option>
<option class="level-0" value="1397">Program Design (16)</option>
<option class="level-0" value="230">Program Theory and Theory Driven Evaluation (9)</option>
<option class="level-0" value="231">Qualitative Methods (129)</option>
<option class="level-0" value="232">Quantitative Methods: Theory and Design (87)</option>
<option class="level-0" value="233">Research on Evaluation (36)</option>
<option class="level-0" value="234">Research, Technology and Development Evaluation (22)</option>
<option class="level-0" value="1479">Social Impact Measurement (48)</option>
<option class="level-0" value="1330">Social Media Tools & Updates (35)</option>
<option class="level-0" value="235">Social Network Analysis (54)</option>
<option class="level-0" value="236">Social Work (29)</option>
<option class="level-0" value="1138">STEM Education and Training (77)</option>
<option class="level-0" value="237">Systems in Evaluation (42)</option>
<option class="level-0" value="238">Teaching of Evaluation (98)</option>
<option class="level-0" value="239">Theories of Evaluation (27)</option>
<option class="level-0" value="1368">Translational Research (19)</option>
<option class="level-0" value="1">Uncategorized (1,571)</option>
<option class="level-0" value="876">Youth Focused Evaluation (102)</option>
</select>
</form>Text Content
Skip to content
Main Menu
* About
* AEA365 Curators
* Archive
* Contribution Guidelines
IMPLEMENTATION/PROCESS EVALUATION: STRATEGIES TO IDENTIFY POTENTIAL AREAS OF
FOCUS BY TAMARA YOUNG
1 Comment / Uncategorized / By Sheila Robinson / August 22, 2021 June 20, 2021
I’m Tamara Young, an associate professor in Educational Evaluation and Policy
Analysis at North Carolina State University. Today, I’m sharing a heuristic I
use with students to help them generate a comprehensive list of potential areas
of focus for an implementation evaluation (of course, with the understanding
that stakeholders, preferable primary intended users, ultimately select the
focus).
HOT TIP:
There are six approaches that can help identify potential implementation
evaluation focus areas. Below, I briefly describe each approach and provide
examples of focus areas that stem from the approach. Certainly, there is some
overlap.
1. Implementation Syntheses or Frameworks
Factors that influence the implementation process that are described in several
implementation syntheses and frameworks, such as The Quality Implementation
Framework (QIF), are potential foci.
Example: QIF’s Phase One: Initial Considerations Regarding the Host Setting
* What types of assessments (e.g., needs, resources, fit, and readiness) were
employed prior to implementation?
* How was buy-in obtained from critical stakeholders?
* What actions were undertaken to foster a supportive climate?
2. Bollman and Deal’s Four Frames
The Four Frames Model describes organizations from four perspectives:
structural, human resource, political, and symbolic. Gallos suggests that each
frame can be used to develop a checklist for gathering data, for example:
a. Structural – regulations and operations
Examples: rules, regulations, policies, and vertical and horizontal coordinating
mechanisms
* What policies facilitate or impede implementation?
* Are there incentives in place to support implementation?
b. Human Resource –the fit between individual and organizational needs
Examples: needs, skills, relationships, morale, motivation, and training
* What types of training is supporting implementation?
* Do staff feel motivated to implement the innovation?
c. Political – power and conflict
Examples: influence, coalitions, stakeholders, interests, and resource
allocation
* Are there coalitions supporting or opposing the implementation of the
innovation?
* Does implementing the innovation intentionally or unintentionally alter the
allocation of resources?
d. Symbolic- ensuring people have a sense of meaning
Examples: values, stories, culture, myths and symbols
* Is the innovation aligned with the organization’s values?
* What symbols are used to facilitate implementation?
3. The Program’s Logic Model
After explicating the logic model for a program, I generate questions from the
items specified in the resources/inputs, activities, and outputs components.
4. Logic Model Components
I keep a list of common questions associated with the first three components of
logic models that can be applied to different programs, for example:
Resources/Inputs
* What is the leader’s role in facilitating implementation?
* What is the nature of coordinated activity with community partners or other
agencies?
Activities
* What are the actual services being provided?
* What activities are used to recruit and retain participants?
Outputs
* What is the quantity of services or products delivered?
* What is the participation rate of different subpopulations, including the
neediest groups (chronic, severe, or highest risk)?
5. Fidelity of Implementation
Saunders, Evans, and Joshi’s how-to-guide explains how FOI can be used to
evaluate implementation. Here are a few standard FOI questions:
Fidelity
* To what extent does each program activity adhere to the intended program
design?
Dosage/Exposure
* What is the quantity and strength of services delivered or products produced?
Coverage/Reach
* To what extent are the participants of the program the intended target
population?
Service Quality/Delivery
* To what extent does service delivery reflect the underlying philosophy of the
program?
* Is service quality undermining or supporting the achievement of outcomes?
* Are participants satisfied with the services or products?
Participant responsiveness
* To what extent are participants engaged with program activities?
Adaptation
* What changes were made to the program during implementation?
RAD RESOURCE:
Implementation Monitoring and Process Evaluation is a comprehensive guide on
implementation evaluation.
Do you have questions, concerns, kudos, or content to extend this aea365
contribution? Please add them in the comments section for this post on the
aea365 webpage so that we may enrich our community of practice. Would you like
to submit an aea365 Tip? Please send a note of interest to aea365@eval.org.
aea365 is sponsored by the American Evaluation Association and provides a
Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365
blog are solely those of the original authors and other contributors. These
views and opinions do not necessarily represent those of the American Evaluation
Association, and/or any/all contributors to this site.
0
0
New
0
0
POST NAVIGATION
← Previous Post
Next Post →
1 THOUGHT ON “IMPLEMENTATION/PROCESS EVALUATION: STRATEGIES TO IDENTIFY
POTENTIAL AREAS OF FOCUS BY TAMARA YOUNG”
1. Mikkel Møldrup-Lakjer
August 23, 2021 at 2:55 am
For implementation evaluation, I use the VICTORE framework of Pawson (Pawson
2013, which elaborates on earlier publications) where context (the C of
VICTORE) is understood as layered on principally four levels: the
individual, the interpersonal, the institutional and the infrastructural
level. The implementing organisation should work to make forces on these
four levels support the workings of the programme. I use the framework to
identify which forces are important for the programme to work.
Reply
LEAVE A COMMENT CANCEL REPLY
Your email address will not be published. Required fields are marked *
Type here..
Name*
Email*
Website
Save my name, email, and website in this browser for the next time I comment.
Notify me of followup comments via e-mail. You can also subscribe without
commenting.
This site uses Akismet to reduce spam. Learn how your comment data is processed.
TAKE ACTION
* AEA Digital Knowledge Hub
* Go to AEA Home
* Subscribe to AEA365 via Email
* Subscribe to AEA365 via RSS
* Subscribe to Comments via Email
* Subscribe to Comments via RSS
SEARCH
Search for:
TRANSLATE
Select
LanguageAfrikaansAlbanianAmharicArabicArmenianAzerbaijaniBasqueBelarusianBengaliBosnianBulgarianCatalanCebuanoChichewaChinese
(Simplified)Chinese
(Traditional)CorsicanCroatianCzechDanishDutchEsperantoEstonianFilipinoFinnishFrenchFrisianGalicianGeorgianGermanGreekGujaratiHaitian
CreoleHausaHawaiianHebrewHindiHmongHungarianIcelandicIgboIndonesianIrishItalianJapaneseJavaneseKannadaKazakhKhmerKinyarwandaKoreanKurdish
(Kurmanji)KyrgyzLaoLatinLatvianLithuanianLuxembourgishMacedonianMalagasyMalayMalayalamMalteseMaoriMarathiMongolianMyanmar
(Burmese)NepaliNorwegianOdia
(Oriya)PashtoPersianPolishPortuguesePunjabiRomanianRussianSamoanScots
GaelicSerbianSesothoShonaSindhiSinhalaSlovakSlovenianSomaliSpanishSundaneseSwahiliSwedishTajikTamilTatarTeluguThaiTurkishTurkmenUkrainianUrduUyghurUzbekVietnameseWelshXhosaYiddishYorubaZulu
Powered by Translate
POSTS RELATED TO AEA TOPICAL INTEREST GROUPS
Posts Related to AEA Topical Interest Groups Select Category Advocacy and Policy
Change (66) Arts, Culture, and Museums (55) Assessment in Higher
Education (34) Behavioral Health (41) Business, Leadership, and
Performance (33) Cluster, Multi-site and Multi-level Evaluation (5)
Collaborative, Participatory and Empowerment Evaluation (165) College Access
Programs (12) Community Development (6) Community Psychology (85) Costs,
Effectiveness, Benefits and Economics (12) Crime and Justice (1) Data
Visualization and Reporting (256) Democracy and Governance (18) Design and
Analysis of Experiments (14) Disabilities and Underrepresented
Populations (78) Disaster and Emergency Management Evaluation (6) Distance
Education and Other Educational Technologies (12) Environmental Program
Evaluation (78) Evaluation Managers and Supervisors (16) Evaluation
Policy (8) Evaluation Use (65) Extension Education Evaluation (40) Feminist
Issues in Evaluation (61) Government Evaluation (66) Graduate Student and New
Evaluators (123) Health Evaluation (75) Health Professions Education
Evaluation and Research (7) Human Services Evaluation (9) Independent
Consulting (165) Indigenous Peoples in Evaluation (40) Integrating Technology
into Evaluation (119) Internal Evaluation (66) International and
Cross-cultural Evaluation (138) Latinx Responsive Evaluation Discourse (43)
Lesbian, Gay, Bisexual & Transgender Issues (53) Military and Veteran’s (23)
Mixed Methods Evaluation (28) Multiethnic Issues in Evaluation (93) Needs
Assessment (52) Nonprofits and Foundations Evaluation (115) Organizational
Learning and Evaluation Capacity Building (97) Prek-12 Educational
Evaluation (190) Program Design (16) Program Theory and Theory Driven
Evaluation (9) Qualitative Methods (129) Quantitative Methods: Theory and
Design (87) Research on Evaluation (36) Research, Technology and Development
Evaluation (22) Social Impact Measurement (48) Social Media Tools &
Updates (35) Social Network Analysis (54) Social Work (29) STEM Education and
Training (77) Systems in Evaluation (42) Teaching of Evaluation (98) Theories
of Evaluation (27) Translational Research (19) Uncategorized (1,571) Youth
Focused Evaluation (102)
ADMINISTRATION
* Site Administration Login
* Site Administration Logout
The views and opinions expressed on the AEA365 blog are solely those of the
original authors and other contributors. These views and opinions do not
necessarily represent those of the American Evaluation Association, and/or
any/all contributors to this site.
Copyright © 2021 AEA365 | Powered by Astra
ORIGINAL TEXT
Contribute a better translation
--------------------------------------------------------------------------------
ShareThis Copy and Paste