University of Virginia School of Medicine                 
Systems Leadership Community
April 13, 2011
Minutes

Present (underlined) were:  Brian Behm, Alan Binder, Robert Bloodgood, Elizabeth Bradley, Megan Bray, Chris Burns, Mary Bryant, Troy Buer, Addeane Caelleigh, Eva Casola,  Randolph Canterbury, Bobby Chhabra, Pam Clark, Bruce Cohen,  Eugene Corbett, Alan Dalkin, John Dent, Terri Ellison, Allison Innes,   Don Innes, John Jackson, Kambiz Kalantarinia, Karen Knight, Darci Lieb,   Stuart Lowson, Jen Marks, Jim Martindale, Melanie McCollum, Veronica Michaelsen, Mo Nadkarni, Bart Nathan, Selina Noramly, Sabrina Nuñez, Richard Pearson, Chris Peterson, Sue Pollart, Ellen Ramsey, Mitch Rosner, Mark Russell, Neeral Shah, Allan Simpson, Nassima Ait Daoud Tiouririne, Jonathan Truwit, Linda Wagoner-Fountain, Mary Kate Worden, Elizabeth Wright, Zhen Yan

Convenors:  Bart Nathan

1. Faculty attitudes expressed to students.  Bart Nathan reminded System Leaders that faculty should be careful about the attitudes they express in front of students.  For example, referring to the new curriculum as the “experimental curriculum” and to the students as “guinea pigs” undermines the students’ confidence in the faculty and the curriculum.
   
   
2. Sabrina Nuñez’s workshops for Systems faculty.  Sabrina Nuñez, working with Kimberly Joyce, is developing workshops for faculty of upcoming Systems.  They will soon be offered to CV and Pulmonary faculty and will be available to other faculty.  They have developed an instructor packet of useful information and templates to help instructors develop their course materials; both are available on the Collab site for System Leaders.
   
   
3. Organization of upstairs-downstairs schedule must be finalized by June.  The general schedule for use of the downstairs (Learning Studio) and upstairs (Lecture Hall) has been set:  Class of 2015 will be downstairs 8-10 a.m.; Class of 2014 will be downstairs 10-11 noon.  Systems Leaders may swap those times by mutual agreement; that final schedule must be completed by June.
   
   
4. Upcoming TBL session on Friday, April 15, 8:00-10:00 a.m.  This session, focused on mild traumatic brain injury and sports-related concussion, would be a good one for faculty of upcoming systems to observe.  It is shorter than many others and yet will illustrate the full TBL structure and process.
   
   
5. Faculty evaluations.  Elizabeth Bradley reported for the working group that is developing a faculty evaluation instruction that reflects the NxGen curriculum and works for P&T.  She distributed a draft of the new evaluation form.  Comments and suggestions were made about revisions, including that
   
   

   (a)	some sessions are multi-faculty, multi-discipline, integrated sessions, and the form would need to accommodate them,
   (b)	there is a great deal of advance planning, and the person who does much of that work may not be the person in the classroom during the session,
   (c)	the graduate school is interested in the form because it will also need to use the form,
   (d)	if the evaluation focuses on single faculty members in single sessions, faculty will be discouraged because the curriculum has been designed, and often delivered, in teams.

6. MBB System Leaders:  Lessons learned and tips.  Bart Nathan, Mary Kate Worden, and Nassima Ait Daoud Tiouririne noted lessons based on their experience with MBB:
   
   

   (a)	Students are very vocal.  About what they like and dislike; comments come fast from all directions.  The most vocal may not reflect the whole class (example: after 10 days of complaints about having so many assessments, the MBB leaders had students vote on whether to have a scheduled weekend assessment, and 68% voted yes).
   (b)	Importance of learning objectives.  The students have bought in to the LO system and philosophy, using them to study before class.  So they want everything beforehand and resent any later changes.  Reminder:  if something in not in the LOs, instructor cannot test for it; faculty may talk about other things but must make clear that they will not be assessed.  A participant noted that this reaction by the students made him uncomfortable.  Bart Nathan commented that the point is that you can teach it but cannot test it unless it is specifically in the LOs.
   (c)	Students do more than the LOs.  They ask a lot of questions by email that demonstrate that they are going beyond the LOs; they often acknowledge it in the questions.
   (d)	Faculty are not ready to produce materials beforehand.  The faculty are having trouble adapting to the new curriculum, both in terms of submitting the materials for review and processing before posting for the students—even having trouble preparing the materials before class begins.  A participant asked about materials for active-learning activities in which the instructor does not want the students to see the materials beforehand.  The response was that the instructor may create two versions, one for the students to use in preparing for class (without the active-learning items) and the other, fuller version with all the materials. Alternatively, if the instructor may request that no materials be distributed
   (e)	If you want students to be prepared on something, assign learning objectives that cover it.  The students will do it and can be relied upon to work on it at a high level.
   (f)	Challenges on exams and quizzes.  Mary Kate Worden noted that the present system is completely dysfunctional.  It takes two weeks to deal with challenges, and students are very unhappy at the wait for scores.  A new/different system is needed.  In response to a question, she said that about half of the challenges are valid and need to be considered.  She also noted that when she reviews the students’ exam answers, she can immediately see likely problems unusually large proportions of students will give wrong answers; she can analyze the answers and decide on grade adjustments.  If a new system allowed her to handle those before the students submitted challenges, the total of challenges would be much lower.   Linda Waggoner-Fountain commented that there needs to be a different system for challenges, if possible one in which students can submit the challenge online at the time of the exam.  John Jackson commented that such a system could be built into the online assessment system if the goals can be described and studied.
   (g)	The students have been very stressed in MBB.  The students are expressing their stress in complaints about classroom assignments and activities.  Nassima Ait Daoud Tiouririne commented that she approves of the System Leaders’ not getting the Listening Post comments. Students can still email directly and so there is no need for Listening Post comments by students.
   (h)	Students are preparing beforehand and really learning the material.  They are conducting high-order discussions of difficult material.
   (i)	Observers.  Mary Kate Worden noted that she checks with the classroom observers to learn their perception of the amount of active learning, to see whether their perception is the same as hers.

    
7. X-Credit.  John Jackson demonstrated the X-Credit software application, explaining that is needed to design and manage the overall curriculum and its components, to respond to LCME visit and questions the survey team may ask, and to analyze students’ performance across topics and years.  He showed how the system allows analysis of curriculum systems and threads.  He also demonstrated how instructors will upload and submit their classroom materials.  There was considerable discussion of the issues of “one learning objective – one test item,” which John emphasized had been demonstrated in the literature as the educational best practice.  Darci Lieb’s group is backloading the learning objectives for earlier systems in 2010-2011, and the system will soon be ready for instructors to upload their materials from this point onward.
   
   
8. “Course Design Course 2011” at the Teaching Resource Center.  Bart Nathan urged Systems Leaders to take the course being offered through the TRC.  Several participants, including Don Innes, comments about how helpful it had been to those who have already taken it.
   
   
9. Next meeting.  The next scheduled meeting is Wednesday, May 11, 2011, at 4:00, in the Learning Studio of the Medical Education Building.

'