Case Coordinator: Justin Walls and Jo-Anne Kelder

University: University of Tasmania

• Priority Focus and explanation of PATS variation
• Key Outcomes
• Learning

Priority Focus and explanation of PATS variation

The focus was teaching teams and quality enhancement of each course delivered in the Faculty of Health. More specifically, to encourage the adoption of a distributive leadership model within teaching teams as a mechanism to achieve a systematic and planned approach to evidence-based design and delivery of all curricula. The term "Quality Enhancement" in our context refers to any curriculum or teaching activity that is oriented towards quality improvement QI), quality assurance (QA) and scholarship in learning and teaching (SOTL).

The Faculty has two programs that function as core elements of a quality framework: the Peer Assisted Teaching Scheme (PATS) and Peer Assisted Course Enhancement (PACE) are designed for teaching staff members to work together on quality-related activities. The goal is to ensure that changes are evidence-based and strategic.

The University of Tasmania’s institutional requirements for quality improvement and quality assurance of curriculum are specified in the University Standards Framework and Teaching Performance Expectations (which additionally articulates expected scholarship activities). Institutional requirements are themselves in response to legislative and regulatory drivers and compliance is mandatory.

The quality framework includes organisational processes and structures to encourage and recognise participation in PATS and PACE, and is aligned to centrally administered reporting and reward systems.

PATS is for individual teachers: focusing on a unit or aspect of their teaching, treated as a specific project for a specified period of time. PATS can be varied from the ‘classic’ PATS design but key components required of a proposal are:

• Plan (project activities, milestones, deliverables)
• Peer Engagement
• Goal setting
• Professional development
• Reporting

PACE is for teaching teams: focusing broadly on the course curriculum, with capacity to target specific components of curriculum (e.g. single unit, sequence of units) or support specific teachers (e.g. novice teachers). Conceptually, a PACE program can incorporate PATS projects for specific units in a course, or to provide targeted peer mentoring for teachers within the team.

A foundational activity for a PACE teaching team is to establish an ethics approved evaluation-research plan. Ethics approval for whole-of-course research means student data (including feedback surveys, assessment items, grades and demographic information) can be collated by an independent third party as re-identifiable data (enabling data matching) and then used by teaching academics for evaluation and research purposes. An “Embedding Evaluation Research into Units and Courses” workshop is a resource for teaching teams that provides a generic ethics application for a course and for a unit with advice on how to customise it for their specific course or unit.

The goal of PATS and PACE is to enhance teaching academics’ individual and collective employment experience through encouraging intentional peer engagement focused on improving curriculum and teaching and thereby to positively affect the quality of units/courses and teaching in the Faculty of Health.

Why

The ‘classic’ version of PATS is a structured, project-based program that provides professional development and peer support to a teacher responsible for a unit that will benefit from intentional reinvigoration or remediation. The Faculty of Health has adopted PATS for the context of 1) units that are flagged as low performing by centrally administered student evaluation surveys, 2) units in need of reinvigoration and 3) teachers who self-nominate for peer mentoring to guide a planned quality improvement.

PACE extends the concept to incorporate these PATS variations into the context of a planned approach to quality enhancement of a course curriculum. Teaching academics are encouraged to have a strategy for data collection to provide evidence for changes to curriculum and teaching practice that can underpin reporting against external standards, including evaluative judgments by peers. Distributive leadership means that all team members can use and develop their skills in scholarship and quality enhancement as they have motivation and opportunity, sharing the workload and rewards for outputs such as teaching awards and publications.

People

All employees who have a role in curriculum delivery within the Faculty of Health (teaching academics and academic administration staff) are invited and encouraged to participate in PACE and/or PATS. Participation in PATS is strongly encouraged for units that are flagged by the centrally administered student evaluation survey; PATS is mandatory if there is no improvement after a subsequent delivery.

Course coordinators are encouraged to use PACE as a means of engaging and leading their teaching team in quality activities. The Faculty provides resources and support to set up their courses for ongoing evaluation-research and organize peer partnerships, professional development and support in translating evaluation and research into measurable outputs (e.g. for performance reporting, institutional quality assurance reports, accreditation applications, grants, awards).

Timeframe

PATS: single semester but allowed to be stretched over 12 months if requested.

PACE: suggested duration is 5 years. This timeframe is aligned to the centrally mandated course review cycle.

Scope: Faculty

The scope of the quality framework incorporating PACE and PATS is all units and courses delivered within the Faculty of Health.

Key Outcomes

PATS Variation – outputs and outcomes

The quality framework with a distributive leadership model for course teaching teams has been incrementally developed over three years. During 2013-2014, staff members from the Faculty’s Quality Evaluation Learning and Teaching (QELT) unit provided peer support and expertise in educational evaluation/research to three teaching teams. The outcome for each team was an evaluation-research plan underpinned by an ethics application. In 2014, each of these teams used the evidence generated by their research program, including peer reviewed research publications, to write successful applications for Vice Chancellor’s teaching excellence awards.

In 2014/15, QELT staff members were awarded a “Projects of Institutional Significance Grant” to further develop and then disseminate the resources cross institutionally. In 2015, the resources developed for the three PACE teaching teams were reviewed and refined to make available:

• generic course-level ethics application (and unit-level version) to enable student and staff interactions and outputs (e.g. assessment items) to be used for research purposes
• method for recruiting, informing and establishing consent of participants that complies with National Ethics Standards – using the University’s online Learning Management System
• professional development workshop “Embedding Evaluation and Research into Courses and Units” to support teaching teams to design and implement the approach for their course
• Faculty quality framework setting out organisational processes and structures to embed PATS and PACE into quality assurance of learning and teaching in the Faculty and link to central processes and reporting requirements.

System level impacts

Within Faculty of Health, University of Tasmania, the Quality Framework with PATS and PACE aimed to have impact at all IMPEL levels and to date has achieved levels 1, 2, 3, 4, 5, 6 and 7.

1. Team members:
   1. PATS: in 2013 nine staff members peer reviewed ten units in the Bachelor Dementia Care (BDemCare); a team of over twenty staff developing a Global Perspectives Program (GPP) for inclusion in a first year unit for Health Professional courses peer reviewed all aspects of the GPP and three teachers in the Bachelor Paramedic Practice (BPP) reviewed two units with support from two staff in a mentoring role. Since 2014 the whole of teaching team approach continued in the BDemCare and GPP; BPP teachers and their mentors extended their focus to all units in the course. Since 2013, the course coordinator of a fully online suite of post-graduate nursing degrees has used PATS to provide peer support to six to eight teaching staff, most of whom are casual/sessional teachers who work off-campus.
   2. PACE: In 2013/14 three teaching teams (BDemCare, GPP and Understanding Dementia MOOC) were supported to establish an evaluation-research program resulting in peer reviewed publications and team teaching awards. A shared leadership model within teaching teams enabled sharing workload, peer professional development and career opportunities for staff interested in emphasising scholarship.
2. Immediate students: the core of PATS and PACE is evidence-based changes to curriculum and teaching and commitment to communication with students about what changes are made in response to their feedback and learning outcomes.
3. Spreading the word: QELT, led by the Associate Dean Learning and Teaching (ADL&T) disseminates information about PATS, PACE and the resources via a QELT newsletter and through Faculty and School Learning and Teaching committees.
4. Narrow opportunistic adoption: Several variations of PATS were piloted in the faculty 2013-2014 (presented as case stories in this Guide). During this period, three teaching teams were supported to develop an evaluation-research framework with ethics. This variation of PATS was named PACE to reflect the focus on shared leadership within a teaching team to operationalize a planned approach to course quality.
5. Narrow systemic adoption: PATS is embedded in the Faculty of Health Learning and Teaching strategic plan in relation to ‘problem units’ and improving teacher professionalisation and quality. Adoption of PACE is slowly gathering momentum. In addition to the teaching teams engaged in 2013/14, during 2015, teaching teams for the Bachelor Nursing and Postgraduate Medicine have decided to adopt PACE for redesigned courses in 2016. 21 staff members indicated interest in PACE for 2016 in response to a workshop on “Embedding Evaluation and Research into Courses and Units”. In 2016, five courses are signed up to set up their courses to include an evaluation-research plan for evidence collection and schedules quality improvement, quality assurance and SOTL activities (including teacher, student and peer lenses). The goal is for 20% of courses in the Faculty of Health to have PACE as a mechanism for organised quality enhancement by end 2016.
6. Broad opportunistic adoption: The quality framework, specifically PATS and PACE, are embedded in the Faculty’s Learning and Teaching Strategic Plan. Currently participation in PATS and PACE is voluntary, however processes are in place to flag units that may need remediation or reinvigoration via the student satisfaction survey and other metrics (e.g. grade profile). Course coordinator and unit coordinator roles / position descriptions are being refined to articulate responsibility for the quality of curriculum. Information and resources are provided on a staff-specific site on the University’s Learning Management System. It functions as an online training space and communication channel to encourage and equip staff members to engage in planned and collegial quality enhancement activities with measurable, reportable outcomes.
7. Broad systemic adoption: the Projects of Institutional Significance project is spreading the word to encourage PACE adoption in other Faculties. The concept and resources are being disseminated via a presentation to the Associate Deans Learning and Teaching Advisory Group; “Embedding Evaluation and Research into Courses and Units” workshops and a presentation at the University’s annual Teaching Matters conference. The resources will be made accessible as a Learning Object Repository (LOR) on the University’s Learning Management System.

 

Learning

Barriers and Opportunities

Two barriers to engagement in PATS identified prior to the 2013 pilot were workload (lack of time) and perceived inflexibility of classic PATS. A decision to allow individuals to negotiate the details of their PATS program (flexibility) and the Faculty providing formal recognition of the time required for collaborative, peer supporting teaching were reported as key motivators for participation.

Institutional drivers for participation in PACE were mandatory requirements related to evidence for course accreditation, course and unit review reports and individual academic’s concerns about meeting the University’s Teaching Performance Expectations. The focus of PACE is planned collection of data that everyone in the teaching team can use for quality enhancement and performance outcome reporting purposes. PACE is pitched to staff as a collaborative and strategic solution to the problem of managing expectations of producing high quality research outcomes as well as the routine requirements of assuring curriculum and improving teaching in response to student feedback.

Academic teaching staff wanted tangible commitment and support from the Faculty in terms of formal recognition of time (workload) and that a commitment to engaging in PATS should be linked to performance management. A current barrier is the University information systems do not yet support integrated monitoring of learning and teaching activities (e.g. recording professional development, rewards, grants) which makes it is difficult to monitor and reward participation and evaluate outcomes if PACE and PATS are systematically adopted across the Faculty.

What worked well

The initial stigma associated with classic PATS (perception that it was for ‘poor performing’ units and teachers) is being gradually overcome by a strategy of prioritising communication about PACE, and the concept of teaching teams, collaborating on a joint evaluation-research project, distributing leadership responsibilities for quality improvement, quality assurance and scholarship amongst the team based on members’ motivation, capacity and time available.

Classic PATS is intended for teachers who need targeted resourcing and mentoring to either 1) remediate/reinvigorate a unit, addressing a well-defined problem (e.g. low student satisfaction) or 2) implement an innovation, minimizing risk to students of a poor learning experience.
Linking participation in PATS and PACE to the University’s Teaching Performance Expectations (TPE) is a strong motivator for engagement. The introduction of the TPEs generated perceived pressure on staff members to have evidence of their engagement in teaching (quality improvement of curriculum, quality assurance and scholarship).

The faculty staff member responsible for developing and encouraging participation in PACE/PATS met with course coordinators and teaching teams, articulating the concept and benefits of a teaching team approach to meeting the TPEs. Course coordinators were encouraged to work towards a culture of distributed (shared) leadership within their teaching teams, develop an evaluation-research plan and gain ethics approval. The Faculty staff reviewed the research plans and ethics developed with three award winning teaching teams and collaboratively developed a ‘generic ethics application’ suitable for course level or unit level scholarship. It provides a well-designed plan and method for routine data collection from students and peers. It complies with the National Statement on Ethical Conduct in Human Research (NHMRC 2007, updated 2015) and the University’s Research Ethics policy. Face to face training workshops and online resources are available for course coordinators and their teaching teams.

PACE and the supporting resources are being promoted and disseminated beyond the Faculty as part of a University of Tasmania Projects of Institutional Significance Grant (2014/15).

What didn't work well

The Faculty of Health has around 90 courses and the uptake of PACE and PATS is small in comparison. Some progress has been made to integrate teachers’ PACE/PATS activities with performance management and reward systems but engagement is still largely based on the ‘carrot’ of a simple method to engage in evidence-based teaching and scholarship. To date we have supported those teaching teams that were already motivated to achieve excellent outcomes. We are putting in place mandatory PATS for units institutionally flagged as needing remediation but the central student evaluation survey is a blunt instrument and does not identify many units that are known to be in need of peer supported remediation or reinvigoration.

What was learnt

The process of developing an ethics application is a good proxy for developing an evaluation-research plan: it provides a structured process of questions in which each section requires team discussion to think through the research problem and rationale, research questions, data to be collected, selection and recruitment of participants, data management and outputs management (including publication plan and authorship). This builds team culture and shared understanding.

Set up for PACE takes time. It is sensible for a teaching team to take up to six months to develop their research plan and adapt the ethics application for their course.

Faculty provided resources for PATS and PACE need to be accessible, with clear guidelines on how to use the resources plus how to adapt for specific needs and context. A site on the Learning Management System is used to communicate with staff members and provide access to resources and online training materials as well as function as a central repository for uploading reports.

Pearce’s (2004) article, The Future of Leadership: Combining Vertical and Shared Leadership to Transform Knowledge Work guided the Faculty staff in designing organisational systems to support embedding PATS and PACE into teaching team work. The concept of shared leadership within teaching teams is very attractive to academics, who value the simplicity of a course-level evaluation-research plan with opportunity to insert individual projects where warranted and sharing data collection, analysis and writing for publication. Pearce’s (2004) advice on the role of vertical leaders in maintaining shared leadership has been confirmed in the experience of the Faculty of Health team:

• Training and development systems – need to be developed and accessible to all staff members.
• Reward systems – need to have clear processes established with links to Faculty reporting requirements and recognition embedded in workload allocations for course coordinators.
• Cultural systems – take time to develop and require trust from all participants. Faculty (vertical) leadership needs to support teaching team (shared) leadership without being controlling; promote its importance and value, measuring performance and providing training.

National System Impact

IMPEL Level 3: Contributions to knowledge in the field: growth or spread of disseminated ideas; serendipitous adoption/adaptation by people beyond the project’s intended reach.