When I began my work at the Charles E. Smith Jewish Day School, my first responsibility was to develop greater consistency across our curriculum and improve student achievement in reading and writing. Many schools wrestle with how best to deliver a rigorous general education within the time constraints of a Jewish day school schedule. Data must be at the heart of the decisions we make as educators to ensure that our classroom work aligns with our desired outcomes.

In the current state of education, standardized tests have become a highly controversial means of analyzing student achievement. This attitude is not unfounded. Nationally, standardized tests that are currently being implemented, created by PARCC or Smarter Balanced, could arguably be seen as imperfect tools to measure teacher efficacy as opposed to student growth. However, Jewish day schools have the opportunity and the flexibility to use standardized assessments for their truest purpose: to  improve student outcomes. Using assessment data can help to ensure that the intended curriculum, the taught curriculum and the learned curriculum are in alignment.

Over the last several years, part of my work at CESJDS has been analyzing data to make informed decisions about our curriculum, our teaching and our priorities. Although we do use formative and summative assessments during the school year to inform instruction, our students also take the ERBs in 3^rd and 5^th grade. Each year when the ERB scores come in, we look not only at individual student data, but also at gradewide trends to see where the gaps in our students’ learning lie. When I first began this work, it was clear from our test scores that there was not enough focus on writing mechanics and writing concepts and skills.

Using the data, I returned to the grade-level teams to discuss units of study and the timing of each unit to ensure that

1. We were covering all necessary concepts

2. We had the time to cover each skill fully

3. The skills we were expecting built on the previous years’ work

4. We had the resources and teacher training to improve our student outcomes.

Difficult decisions had to be made. Integration of content and skills across disciplines was necessary to provide enough time in our truncated day for students to practice and hone their skills. Together, the teachers and I documented the skills that were necessary in each grade and ensured consistency of skill work across classrooms through collaborative teams and examination of student work. This work was not easy; many teachers had to give up units that they had been teaching for years or change their methods or instructional practice. However, when looking at the data, it was clear these changes were necessary.

In the end, the hard work of data analysis and reflection on our practice based on these tests have produced excellent results, as our students have made significant gains. Standardized tests are certainly not the only way to measure student achievement. However, they provide a window on a school’s performance, enabling adjustments to be made that ensure the highest levels of achievement.

When our students purchase new technological devices, often they quickly become capable of creating professional looking slide presentations, videos and musical compositions, bringing research and personal questions together, wowing us with their clever humor and with their technical proficiency. In many Jewish studies and Tanakh classes that I have taught and observed, students have produced work at high levels of creativity, using these and other media:

Apple Keynote, Microsoft PowerPoint and Prezi slide and image presentations about medical ethics

Videos produced with iMovie software to dramatize a passage in the Tanakh

Lecture-style whiteboard presentations made with Explain Everything and Educreations to share moral dilemmas in intellectual property law

Voice-over public service announcements made with Apple’s Garage Band or with a built-in microphone and recorder to encourage young people to guard against Lashon ha-Ra (inappropriate speech)

A guide to promote the ethical treatment of animals in a brochure format available through word processing software such as Microsoft Word or Apple Pages

Newspaper comic style representations of different schools of Zionist thought through Comic Life or GoAnimate

Newspaper broadsides using Microsoft Publisher to call people to come to Israel on one of the early waves of aliyah

Original non-technological artwork in pastel, pencil or pen and ink on paper that presented a menu in a restaurant inspired by the philosophy of Judah haLevi

Original songs about the biblical period of the Judges, played on live musical instruments

Board games created to challenge 7th grade students to reach ever-higher levels of tzedakah

As rewarding as it is to see creative work from our students, sometimes it is easy to feel lulled into confusing our students’ proficiency and creativity with mastery of the knowledge we aspire for them to acquire. Teachers encounter a number of questions when they consider how to evaluate such student work:

How can we be certain that a fun video, a clever comic strip, and a newspaper-style broadside add up to proficiency and mastery?

Do we all agree on what we mean by “creativity”? Is one student’s creativity the same as another student’s creativity?

Does creativity in one medium add up or correspond to creativity in another medium? In other words, can I judge and compare a public service announcement with a board game and with a video?

Some teachers may feel, as I have felt, that they don’t feel capable of grading for creativity, so they look at a student’s mastery of the content and they simply leave creativity as something praiseworthy to be mentioned but not assessed. Creativity, they reason, is one of those areas that is deeply personal, a matter of taste rather than objective measure.

I would argue, however, that we shortchange our students by withholding important feedback that could come from assessing their creative efforts. An article about creativity, published by the Association for Supervision and Curriculum Development (Sarah McKibben, “Creativity Isn’t a Free-for-All”), helped to change my thinking and gave me a valuable tool to evaluate creativity.

The rubric helps teachers to assess students by addressing several issues:

Does the work demonstrate mastery of the subject matter content by bringing together concepts from a variety of disciplines and sources?

Does the student do the synthetic work of bringing together ideas in unusual and novel ways?

Does the project communicate something new that makes an original contribution?

In order to succeed, a student also will have to answer the following questions:

Does the finished product demonstrate proficient use of the medium of the presentation? (In other words, is the student proficient with slide presentation software or with creating something with pen and paper?)

Does the project somehow communicate the student’s curiosity and stimulate the curiosity of the project audience?

I would agree with Tikvah Wiener (“Taking Creativity Seriously,” HaYidion, Summer 2015) that it is helpful to establish standards for creativity by requiring documentation of sources of information; both generation and selection of meaningful and significant ideas; and presentation of work to users or a target audience.

The creativity rubric allows a teacher to establish a framework for assessing any creative project, regardless of the medium and the assignment. We know that our students are capable of echoing and carrying forward the creative genius of their predecessors from the writers, prophets and rabbis of ancient times to the philosophers, state-builders, artists and poets of our time. We play a crucial developmental role by not being shy about assessing our students. It is for us to challenge and to inspire our students to reach excellence by helping them to shape their creative voices as they mature from children into young adults.

Rabbi Judd Kruger Levingston PhD is the director of Jewish studies at the Jack M. Barrack Hebrew Academy, Bryn Mawr, PA

As a K-12 school, David Posnack Jewish Day School has a unique opportunity to use assessments to analyze the academic curriculum at all grade levels. When standardized testing or summative assessments identify areas in which students demonstrate a specific weakness or lack of prerequisite knowledge, Posnack teachers are able to use this data not only to remediate instruction, but to proactively ensure that academic gaps are closed before moving forward to the higher level grades.

This approach is most easily illustrated by the change that took place in Posnack’s lower school (K-5) math curriculum over the last three years. The school administers standardized testing twice a year to all students in the lower school and middle school for the purpose of tracking student progress and to assist in determining appropriate class placement. Using an adaptive norm-referenced assessment provides valuable data for teachers, including individual growth projections and normative group data to compare student performance with others outside of the school population.

Several years ago, standardized testing results indicated that many middle school students were lacking key foundational math skills, specifically in the area of operations and number systems. Lower school students were similarly scoring below the desired mean in areas of number sense, operations and algebraic thinking. Classroom observations and assessments in both the elementary and middle grades also determined that a large percentage of students were unable to consistently apply previously learned problem-solving strategies to multistep word problems. As the movement towards greater rigor and relevance in math education was taking hold, addressing these issues was particularly important to the leadership at Posnack. The concern was that not addressing these areas of weakness would ultimately affect the students’ success in high school level algebra and beyond.

The lower school math curriculum was then reviewed to determine how to best address these deficiencies and for which specific grade level(s) intervention would be most successful. The decision was made to implement a more rigorous math curriculum for the entire lower school by introducing Singapore Math strategies and changing the instructional methodology for problem solving. This new method of delivering math instruction focused on a constructivist approach to teaching mathematics whereby students were actively engaged in the lessons and used concrete and pictorial representations to solve word problems. An emphasis was also placed on the importance of all students discovering for themselves, which strategies and computational methods worked best for their individual learning preferences. In essence, teachers no longer delivered a “one size fits all” approach to teaching mathematics, but rather encouraged students to think through the problem-solving process and combine together their existing knowledge and newly acquired skills as they searched for solutions.

After only two years of implementing these changes to the math curriculum, this teaching methodology has translated into a measurable increase in both standardized assessment scores as well as classroom assessment grades. Even more important is the change in students’ attitudes and perceptions of their math abilities. This positive approach to learning mathematics at the lower grade levels is encouraging, and we believe that the benefits will continue to unfold as the students work their way towards higher-level math courses.

Is your school’s today the same as its yesterday? Enrollment dropping—staff morale sinking—parents frustrated—do you know why?

One thing’s for certain: we cannot afford to stand still in today’s demanding, competitive educational environments. When every Jewish child counts and the tectonic plates of the educational landscape continue to jostle and shake, how do you know what moves to make to ensure you’re delivering the best your system can provide?

The answer lies in research. Data-based decision making is as relevant to schoolwide reform as it is to effective instruction.

At the Tucson Hebrew Academy, we implemented a comprehensive process to seek input on the thoughts, feelings, perceptions and experiences of our constituents. This process may just be something that assists your school’s evolution as well.

Establish a sense of urgency and hunger for data with the board of trustees. Once the board understood the need, its strategic planning committee launched an effort to rebrand the school that has become one of the most powerful agents of reform THA has seen in years. Our board [1] hired a local firm for this work. With or without a consulting firm, the following steps will support any school’s efforts to gather this critical data.

Identify your constituents. Whom do you want to learn from? We selected current students, parents and employees, alumni and alumni parents, parents who moved their children before they graduated, parents who looked at THA but chose another school, and parents of current preschoolers in our local synagogue and JCC programs. We then branched out to lay and professional community leadership, including every rabbi with a synagogue in town, Jewish federation and foundation leadership, and others.

Determine what you want to learn. A subcommittee worked with the consultants to develop questions for input. Asking the right questions is essential.

Collect data. We used focus groups, interviews and online surveys. We invited participation via email, the school website and social media. 388 people participated.

Analyze and report data. A comprehensive report was presented to our board and strategic planning committee. From there, data was shared with employees by the head of school and will be formally shared with families.

Rally the troops! Based on what we learned—our strengths and opportunities—I was empowered to move forward with key community and school leadership to identify reform initiatives for THA. Here was the moment to develop a shared vision to address our needs. This included a great deal of face time and the establishment of a new entity at THA: the site Leadership Team, consisting of teachers and other school employees.

Make the data your driver. Our decisions are based on two things: what is best for children, and what the data tells us. This empowers leadership to focus intently on what is in the best interest of the future of the school.

What did we learn, and what are we doing about it? Essential findings included great positives: relationships with teachers and students, our sense of community, and the quality of our academic program. Past efforts and accomplishments provide us a powerful platform for renewal. Our key priorities revolved largely around communication and outreach, as well as the (real or perceived) need to improve the quality of our secular, Judaic and Hebrew programs. Cultural reform on many levels was urgent, including professional standards and expectations, inclusive and effective problem solving, and a refocus of energy and efforts towards a team mentality to support students, teaching and learning as top priorities.

To that end we implemented strategies to reconnect administration with faculty and parents, and the school in general to Jewish community agencies and leadership. We adopted routinely scheduled communication practices that focused on practical information and school/student happenings. An open door policy was established and initiated with a widescale “listening tour” to reach out to parents, employees, clergy, and agency and lay leadership. We expanded programming with partner organizations and synagogues. We established higher standards for customer service and professionalism across the board, increased teacher collaborative time as expectations for excellence and accountability were raised, and emphasized professional development for individuals and the instructional team as a whole.

We continue to work towards refinements that take time to develop and implement. We’re revising our mission and retooling our image. Leadership has established a seismic cultural shift that the community has embraced with enthusiasm. A sense of renewal, energy, capacity and collaboration feeds our spirits. Standing on the great strengths and accomplishments of the past, armed with powerful data and vision, we build towards a brighter future for our students and Am Yisroel.

Once there was a gentile who came before Shammai, and said to him, “Convert me on the condition that you teach me the whole Torah while I stand on one foot.” Shammai pushed him aside with the measuring stick he was holding. The same fellow came before Hillel, and Hillel converted him, saying, “That which is despicable to you, do not do to your fellow. This is the whole Torah, and the rest is commentary—go and learn it.” Babylonian Talmud, Shabbat 31a

In this classic text, we often focus on Hillel’s universal message, understood colloquially as behave toward others as you want them to behave toward you, seemingly reducing the Torah to that basic etiquette lesson, applicable as much to a children’s playground tussle as it is to geopolitics. Yet, in thinking about this issue of HaYidion on measurement, I found myself drawn to a simple detail in the text: Shammai’s measuring stick (translated from the Talmudic Hebrew amat habinyan).

The Talmud generally doesn’t use language extraneously; Shammai’s use of a measuring stick to push away the seeker is significant. In the telling of this story Shammai is clearly the bad guy in opposition to Hillel, the good guy. But what makes Shammai so problematic? I’d suggest that his behavior offends us precisely because he chooses to use his own measuring stick as a means to judge the worthiness of another. Shammai establishes an exclusive set of standards and measures everyone else against those standards.

Hillel, in contrast, is able to see the person standing before him and respond accordingly. Perhaps his message is less a message to the seeker, and more a rebuke to Shammai: “Would you wish to be measured against another’s standards? For surely you might not measure up either.” Thus, “That which is despicable to you, do not do to your fellow.” This undergirds the whole Torah, because when we measure others with standards that are not reflective of who they are or what they want to be, we prevent them from engaging in the most meaningful ways with Torah. Other famous rabbinic statements claim that the Torah has “seventy faces” and speaks in “seventy languages”; the face and language that one person encounters, even someone as respected as Shammai, cannot be imposed on others.

The use of data and measurement has become standard operating procedure across all sectors of society—business, nonprofit, social services and education alike. Yet data collected and measurements assessed can only be valuable in relationship to what an organization is trying to accomplish. As simple as this seems, it’s not always the practice to ask “What are we trying to accomplish?” before we ask “What we should measure?”

I’ve often been struck by conversations over how we should measure the “Jewishness” of a day school, with the implication that the value or success of that school is then calculated by a set of predetermined Jewish data points. The definition of a Jewish day school then hinges on certain measurable criteria: how much tefillah takes place in the school and what sort; the number of hours devoted to Jewish studies subjects; the strictness of a kashrut policy or admission decisions; how a school celebrates Yom Ha’atzma’ut; what the dress codes are, or who teaches there. The motivation in these debates is by no means insidious; with the best of intentions, researchers and practitioners seek to identify success stories in Jewish education in order to replicate them and engender further success stories. The challenge becomes when we expect all schools to be able to be successful in the same ways.

When we define Jewish day schools using standard measuring sticks, we run the risk of becoming Shammais. We ignore the diversity of the day school field and eschew a deeper understanding that the goals of a Jewish day school and its function will differ according to its own community. Form should thus better follow function than vice versa.

By no means do I intend to suggest that there should be no standards or that schools should not strive to be passionately, profoundly Jewish places. Capitalizing the J in Jewish day schools and supporting the lay and professional leaders of schools to live deeply within their Jewish missions has been at the core of the work that RAVSAK has done with our schools for more than two decades. But I am suggesting that we as a field need to be more intentional about what we mean when we talk about Jewish schools and how we help schools first and foremost articulate what they are trying to accomplish. Only then can we understand what kind of Jewish schools they need to be in order to simultaneously serve and elevate their communities. Only then can we turn to schools and together define standards and benchmarks that ask schools to measure themselves against their own goals and their own community values.

Hillel doesn’t say, do to others as you would do to yourself (the Golden Rule), because that is precisely what Shammai has suggested: that his standards should be others’ standards. Rather, he says, don’t do to others what you wouldn’t want done to you (the Silver Rule). The same goes for Jewish day schools. Let’s collect data, set standards and find ways to measure schools. Let’s just do so with the full awareness that engaging each institution in its own place may be what Hillel deemed to be kol hatorah kulah (the whole Torah).

Classroom observations hold great potential to improve teaching and learning. In an effective evaluation and feedback system based on mutual trust, observations can clarify expectations for teaching, support teachers in elevating their practice, and provide essential information for professional development decisions. Moreover, when teachers receive regular, meaningful, and actionable feedback on their practice they are more willing and better equipped to make the instructional shifts called for as schools increasingly aim to become models of innovation.

Unfortunately, regardless of the nature of other innovations taking place at schools, teacher performance evaluations today often remain stiff, formal, generic and even fearful proceedings. The same forms and evaluation criteria are often used for all teachers. Teachers hold their breath when the supervisor enters the room, they put on their best dog and pony show, and then breathe a sigh of relief when the supervisor leaves. They hope the supervisor was duly impressed, and that they will be able to keep their teaching jobs. The process does not demonstrate the level of trust between teacher and supervisor that is essential for a teacher’s professional growth.

During my time as principal, I once had a second-year teacher approach me with concerns about her fourth grade social studies class. She said, “Dr. Solomon, I’m bored while I’m teaching! And if I’m bored, then I know the students are bored.” She was seeking guidance in developing engaging lessons that held meaning and relevance. And because this was one among numerous conversations and interactions that focused on her growth as a teacher, she trusted that I would not view this revelation of her weakness in a punitive manner, but rather as an opportunity to continue to improve her pedagogy, with the students reaping the benefits. We worked together to plan creative and interesting lessons and activities for the upcoming unit. Naturally, when it came time to conduct a formal evaluation, her progress in promoting student engagement in general, and in social studies in particular, was an area of focus and a discussion topic. Had we not had a relationship built on trust, she most likely would not have sought my assistance in developing what she recognized as a weak area.

Too often, however, teachers fear exposing their weaknesses with supervisors, for fear of losing their job. They will put on their best performance in an effort to appear perfect. A lack of trust in their supervisor’s intentions will drive them to conceal or overcompensate for shortcomings. Therefore, when the formal evaluation process commences, teachers are stressed and supervisors can be stymied in trying to accurately evaluate teacher effectiveness. Even when the formal evaluation is part of an ongoing conversation between teachers and supervisors, as it should be, how can we know if teaching and learning are really going on through these evaluations? How can we help teachers come to understand that performance evaluation is not about “catching them”?

The answer is, we can’t, at least, not through this commonplace process. A system that truly measures instructional effectiveness should be meaningful, respectful (to both teacher and supervisor), trust-based, and faithfully adhered to by all personnel. In addition, when conducted properly, it is clear to all parties that performance evaluations are focused on teacher growth and development, as well as student learning. They are simply part of a continuous growth process.

If we Google the phrase “teacher performance evaluation form,” we will find a seemingly endless supply of generic forms that are currently in use for teacher evaluations. However, we do not have generic schools, and our teachers are not all the same. How can one form provide a complete and accurate picture of who your teachers are and what they and their students have accomplished? And what happens when that form goes into the dreaded “permanent file”? (More on that later.)

The evaluation process, along with any accompanying forms, should be mission- and values-driven. As Jewish day schools, we have a unique set of values that we work to impart to our students. Shouldn’t our teachers be a part of that process? All of our teachers, not only those who teach religious subjects, should regularly communicate and model the middot that are valued by our schools. Therefore, the evaluation process should be developed and conducted with the intent of developing teacher middot as well as pedagogical skills. Ultimately, teachers should feel valued and enriched by the process, not only judged.

However, the best intentions will still yield largely unproductive results if a climate of trust does not pervade the school. In an environment where teachers do not trust their supervisors, and do not feel that they have the teachers’ best interests in mind, a culture of self-preservation will prevail. Megan Tschannen-Moran, a professor and researcher in educational leadership, who has conducted numerous research studies on trust in schools, defines trust as one party’s willingness to be vulnerable to another based on the confidence that the other is benevolent, reliable, competent, honest, and open. She believes that in the absence of trust, people are increasingly unwilling to take risks (read: to innovate), are likely to withhold information, and will view even benign actions of others with mistrust. Essentially, without trust, a supervisor will be unable to accurately measure teacher performance.

How do we know if our staff members trust their supervisors? It can be difficult to determine. Annette Baier, in Moral Prejudices: Essays on Ethics, writes, “We inhabit a climate of trust as we inhabit an atmosphere and notice it as we notice air, only when it becomes scarce or polluted.” Nevertheless, there are some clear, research-based steps that you can take to establish and build trust in your school. (For information on measuring trust in schools, visit the Research Tools page on Tschannen-Moran’s website, http://wmpeople.wm.edu/site/page/mxtsch.)

While teachers’ honesty and integrity in interactions with the principal are important, it is the responsibility of the supervisor—the person with more power in the relationship—to set the stage for trusting relationships with teachers. Tschannen-Moran has referred to teacher empowerment and shared decision-making in much of her work, and found evidence of their positive relationship with trust. Phyllis Gimbel, in Solutions for Promoting Principal-Teacher Trust, agrees. Supervisors need to be willing to trust teachers to make decisions, allowing them a voice in issues of consequence, Gimbel believes, and not just in simple matters such as selecting field trips. She writes, “To earn trust, you have to be willing to extend trust.”

Researchers have consistently found that principals who are trusted and respected take an interest in the well-being of all members of the school community. Effective shared decision-making comes with knowing your staff well, and understanding their individual areas of strength and expertise. As a teacher, I had been given generic forms to fill out about my own performance, often as a precursor to a formal observation. There was often no indication that my evaluator was aware of or interested in hearing about the innovative ideas and student learning experiences that were unique to my classroom. Instead, I was asked to respond to generic questions about classroom management and lesson plans. The supervisors would use the same form to evaluate my performance. Unfortunately, this process of questionable value is still in wide use today. Instead of handing a middle school math instructor evaluation documents that look the same as those handed to the second grade science teacher, consider composing questions and criteria that reflect your own awareness of and interest in each teacher’s individual experience.

The value in composing and conducting a highly individualized performance evaluation is twofold. First, it is a clear demonstration that the supervisor actually cares about each teacher’s growth and effectiveness. When you walk into classrooms in your schools, it is apparent that each one has its own set of dynamics. Let your teachers know that you notice and appreciate each classroom’s strengths and challenges. These should serve as a focal point for constructive feedback. Second, it builds trust in the notion that information and suggestions related to the performance evaluation were given for the benefit of the students and the teacher. It is clearly demonstrated that the process has value, and is not just a means to “catch” them.

And what of the dreaded permanent file? Bob Hoglund, an educational consultant and the author of Intervention Strategies and The School for Quality Learning, suggests in a blog post that the sole purpose of an evaluation is to enable teachers to see their strengths, identify areas of growth, and articulate strategies for accomplishing goals. Therefore, keeping evaluation documents in a permanent personnel file serves no useful purpose. He suggests turning over all data and records related to performance evaluation directly to the teacher, who then decides if he or she wants any of it in the permanent personnel file. He states, “This gesture is the principal’s ultimate statement of trust and closes the book on the teacher’s fear of dismissal relative to the evaluation process.”

The role that trust plays in effective teacher and staff performance evaluations cannot be overstated. In a trusting relationship between staff and supervisors, there is a free flow of communication which engenders productive responsiveness, risk-taking, and honest evaluations which highlight both strengths and areas of growth. In addition, as mission- and Jewish values-driven organizations, our teacher-supervisor relationships should reflect the same levels of respect and middot that we work to instill in our students. 

Dr. Rebecca M. Solomon is the director of curriculum and instruction at Hochberg Preparatory School in North Miami Beach, Florida. [email protected].

American education is awash in evaluation these days. The driving notion seems to be that if we specify the outcomes we are after and test for them, good instruction will follow. This is both good news, and bad news, for Jewish schools. The bad news first. Vast resources are being directed away from teachers and students, and towards the development of tests: tests for students, tests for teachers, tests of administrators. One only has to glance at the newspaper to see the lively conversation all this testing has generated among parents, teachers and kids. Last year Detroit teachers reported that they spent 49 days of school (out of 180 days in the school year) administering standardized tests. That shifts the aims of instruction and takes time away from it as well. One perspective on our national obsession with assessment and evaluation is that it has depleted the teaching profession and made few improvements in student learning.

The good news? The emphasis on testing has yielded some important resources for educators when used judiciously. For example, we now have very detailed specifications of high-quality teaching that can be used by Jewish schools. For example, Marzano’s Teacher Observer Protocol provides clear descriptions of proficient teacher actions across subject matter areas; the TRU Framework is an example of a robust tool for articulating the components of mathematics teaching. And Jewish schools generally have the great advantage of being able to use these resources, as they choose. Free from federal mandates, Jewish schools can take the best of what these systems offer and leave behind some of their damaging side effects. In this article, I discuss some of the resources that are currently available specifically for teacher evaluation, and consider how they might be useful in Jewish educational settings. From my research and my experience in schools, I offer five guidelines for making productive use of assessment and evaluation resources.

Pick the right tools

My graduate student research group is conducting a study wherein we watch a number of videotaped lessons repeatedly, each time using a different observational assessment tool to appraise the quality of instruction. What we have learned is that, although teachers’ global performance is mostly level across the instruments, each instrument emphasizes different aspects of instruction. This means that school leaders can choose instruments strategically depending on their school’s instructional needs. So, for example, if a school’s priority is to build a positive climate for children, the principal might pick Charlotte Danielson’s Framework for Teaching. It’s estimated that this instrument is used in 90% of US public schools. Teachers and administrators alike feel that it captures the work of teaching well, and emphasizes important features of high-quality teaching. For example, “proficient” teaching is described this way in the Domain of “Classroom Environment”:

Teacher-student interactions are friendly and demonstrate general caring and respect. Such interactions are appropriate to the ages, cultures, and developmental levels of the students. Interactions among students are generally polite and respectful, and students exhibit respect for the teacher. The teacher responds successfully to disrespectful behavior among students. The net result of the interactions is polite, respectful, and businesslike, though students may be somewhat cautious about taking intellectual risks.

Danielson is relatively quiet on teaching particular content, though. A school that is more concerned with mathematics content, or literacy content, or the teaching of Bible, would do well to choose an instrument that makes those content areas more visible. While the Tanakh Standards from The Jewish Day School Standards and Benchmarks Project are not written to address Tanakh teaching, they do provide a strong framing for curriculum. These kinds of specification help schools develop a shared vocabulary about the teaching of content and the implications for good teaching. The TRU (Teaching for Robust Understandings of Mathematics) Framework describes high-level mathematics content instruction this way:

Classroom activities support meaningful connections between procedures, concepts and contexts (where appropriate) and provide opportunities for building a coherent view of mathematics.

Choosing an instrument for teacher evaluation that is aligned with the precise areas identified for a school’s instructional improvement is the first important decision in using evaluation tools wisely.

Make it collaborative

Across the US, teacher evaluation instruments are used increasingly to make “high-stakes” decisions about hiring, firing and retention. That has some unintended consequences: if teachers are worried that their performance on these observational assessments can sink their career prospects, the potential for these tools to help teachers improve is diminished. My research, and my experience working with teachers, indicate that these evaluation tools can be used much more fruitfully if they are anchors for collaborative conversations around instruction, rather than a mallet or a set of directives from on high. When teacher assessments are used to reward and punish teachers, teacher buy-in is threatened. Discussions about the hard work of teaching can be silenced if teachers are reticent to share their challenges. But teacher evaluation does not have to be an exercise in compliance rather than learning. Instead, teacher observation instruments can be touchstones to structure ongoing work that teachers do in collaboration with their peers and administrators to get better at what they do.

A great model for this is approach is drawn from a principal I studied in a large research project on teacher evaluation. “Raymond” (not his real name) creates verbatim transcriptions of each lesson he observes, and he then provides those transcripts to the teacher the following day. During their post-lesson conference, Raymond and the teacher sit together with the transcript and look at the teacher evaluation rubric adopted by his school system. During this conference time, the teacher uses evidence from the transcript to appraise her own performance. Together, Raymond and the teacher choose a focus for the next observation, based on what they learn from this post-lesson conference. Raymond’s implementation of teacher evaluation gives the teacher ownership of the process, encourages the use of evidence instead of opinion and memory, and prompts collaborative conversations around facets of instruction. In this way, teacher evaluation becomes a structure for teacher learning instead of a method for making consequential decisions about employment.

Stick with it

Teacher evaluation rubrics are typically many pages long and cover a wide range of teaching domains. As an example, the 2013 version of The Framework for Teaching by Danielson is 111 pages long and contains 22 components of instruction in four domains! It does little to improve instruction when such tools are frequently swapped out under teachers’ feet. My advice: pick an evaluation instrument and stick with it for a number of years, building a shared understanding of its view of instruction. It takes time to grow into the language that an instrument uses to describe instruction, and that language can animate conversations about teaching for many years. The reform-du-jour atmosphere causes innovation fatigue among teachers; besides, any reform worth its salt should be comprehensive enough to require long-term investment. Stay with a few big ideas, and resist that temptation to jump to the newest shiny thing that comes along.

One bite at a time

Once a school picks an instrument, select a couple of instructional foci within it and work on them over time, as a whole school. “Lizette,” another principal we studied, decided along with her faculty that they would work on instructional questions for two years. Teachers began by studying the following passage from one observational instrument:

Good teachers use divergent as well as convergent questions, framed in such a way that they invite students to formulate hypotheses, make connections, or challenge previously held views. Students’ responses to questions are valued; effective teachers are especially adept at responding to and building on student responses and making use of their ideas. High-quality questions encourage students to make connections among concepts or events previously believed to be unrelated and to arrive at new understandings of complex material. Effective teachers also pose questions for which they do not know the answers. Even when a question has a limited number of correct responses, the question, being nonformulaic, is likely to promote student thinking. (From Danielson’s Framework for Teaching)

Lizette organized a series of monthly professional development sessions focused on studying instructional questions together, and her weekly faculty meetings touched on this topic as well. Teachers read about instructional questions, watched videos of lessons and analyzed the kinds of instructional questions posed, and they designed lessons together using their new knowledge about instructional questions. The teachers gathered evidence in their own classrooms from these lessons, and they pored over what they saw as a result of their new questioning techniques.

Lizette’s observations of individual teachers highlighted planning for and evaluating teachers’ use of instructional questions, and she was able to document teachers’ progress on this dimension over time. In two years, the school saw real growth in teachers’ skill in this area, and student achievement rose along with it. What was key here was that this topic was worthy of ongoing study over time, and that beginning teachers and more experienced teachers alike would stand to gain from their joint study of this topic. Lizette chose not to address every domain in the observational instrument, which meant that other dimensions of instruction got relatively little attention while teachers developed a strong repertoire of instructional moves in questioning. This is a long-term strategy and underscores the fact that serious teacher growth involves career-long, collaborative effort.

Collect the right data

Teachers these days are encouraged to make “data-driven decisions,” in part because schools have more data than ever at their disposal. But the use of new teacher evaluation instruments depends on choosing the right data for decision-making. Often these are the data most proximal to a lesson: students’ work, a record of what students said in discussion, photographs of the whiteboard at the end of the lesson, a videotaped excerpt from the lesson. In visiting schools in Japan, we saw educators from a range of occupational identities (principals, teachers, professors, coaches) gathering together around a stack of student work or a photograph of the chalkboard following a lesson in order to appraise the quality of that lesson and how to improve the next one.

What we noticed was the close linkage between an observed lesson and the data from it, data that are relevant to student learning in that particular lesson. Imagine the kinds of productive conversations that ensue around components of an observational instrument when data are there to illustrate student learning. This conveys a number of important ideas about teaching: that each lesson matters, that the basis for judging the success of a lesson is evidence of student learning, that a single lesson is the crucial unit for appraising instruction, and that lessons are important enough that educators from every level of the system gather to study them together.

Like all forms of assessment and evaluation, teacher evaluation stands to improve instruction, but it can also stand in the way of instructional improvement and the building of professional capital if we don’t mind the details. Because they can skirt the high-stakes uses of assessment and evaluation, Jewish schools can select the instruments and the modes of engagement that best match the needs of their students. We’ve assumed that good teaching is obvious, but in fact these instruments make visible the intricate work and prodigious skill that is required. Jewish schools stand to benefit from the careful articulation of teaching practice. Done well, teacher evaluation systems can be the cornerstone of elevating the profession of teaching. Because Jewish schools opt into these systems, they are uniquely positioned to craft programs of assessment and evaluation that are tailor-made for each school’s needs.

Dr. Jennifer Lewis is an assistant professor of mathematics education at Wayne State University and works with the Mandel Foundation for Jewish Education. [email protected]

As a first year college student enrolling in an advanced Spanish course, I found myself sitting in class with students from diverse backgrounds. Many students, like myself, had barely visited a country with Spanish as its native language. Others in the class had actually lived in Spanish speaking countries and spoke—at least to my ear—fluent Spanish. This situation presented a challenge to the teacher in terms of grading. Students were entering the class with vastly different fluency levels in Spanish. Judging people at the end of the course on their Spanish fluency without taking account of their starting place wouldn’t be fair to those entering the class at the lower end of the fluency spectrum. The teacher’s solution was simple: Instead of judging us based solely on our attained fluency at the end of the course, she would judge us based upon our growth.

The situation in this class is in many respects a microcosm of efforts being undertaken in education nationally over the last decade. As part of the “big data” movement where efficient data collection and management is being leveraged to glean insights previously invisible across almost every sector, including education, we are now “connecting the dots” and looking not just at how students perform at the end of a class, but also how much they’ve grown—taking account of where they started. Unique student identifiers together with modern data management systems allow teachers, principals, administrators and policy makers to follow the child and understand not only their level of mastery at a point-in-time but also the progress they have made along the way.

The transition, however, has not been instantaneous. Education is replete with nomenclature which describes the point-in-time measures of student level of mastery of a topic. In US public schools, for example, the term proficiency has described the manner in which level of mastery has been characterized in reading and mathematics based upon performance on state standardized assessments. More recently, career ready and college ready are terms widely in use. These and similar terms are characterizations of a student’s performance that don’t take account of where the student began. The benefits of going beyond simple point-in-time views of student performance are profound. Taking account of where a student starts allows practitioners to tailor instruction to the student and where they are at. Simultaneously, by monitoring their progress, mid-course adjustments can be implemented based upon the real-time needs of the student.
students perform at the end of a class, but also how much they’ve grown—taking account of where they started. Unique student identifiers together with modern data management systems allow teachers, principals, administrators and policy makers to follow the child and understand not only their level of mastery at a point-in-time but also the progress they have made along the way.

If one thinks of education as a journey, educational leaders are concerned, even obsessed, about whether students reach their destinations. How does one go beyond level of mastery to characterize a student’s educational journey? The answer is to look at student growth in conjunction with student level of mastery. As the formula distance = rate x time implies, the distance a student travels in the education journey is a function of where they start (initial level) and the rate they travel along the way (i.e., growth). Examining student growth is simply a view of level of mastery over time—a level of mastery timeline for each student, monitoring their academic progress as they move through the education system.

This picture is a student level of mastery timeline that communicates the progress of a public school student. The main features of the figure, however, are relevant to monitoring progress of a student in any subject and in any educational context. In the picture, the grey-scale background indicates the criterion-referenced level of mastery levels in reading against which student level of mastery is judged (Unsatisfactory, Part Proficient, Proficient, and Advanced), and the white dots show the level of level of mastery of the student in the indicated grade and year. For example, in grade 3/2009-2010, the student’s level of mastery was judged Unsatisfactory. In grade 4/2010-2011, the student’s mastery was judged as Part Proficient followed by Part Proficient level of mastery in grade 5/2011-2012 and Proficient level of mastery in grade 6/2012-2013. Relative to the level of mastery levels, the student’s progress between grades 3 and 6 can be characterized as “catching-up.”

Note that regardless of content area, a criterion-referenced examination of level of mastery over time can be constructed. Whether for physical education or social studies, one need only have level of mastery standards and tests anchored to those content areas to produce the figure. What the figure provides in addition to criterion-referenced level of mastery over time is a norm-referenced interpretation of that progress, indicated by the arrows and fan. Because so many students take the same assessments in the public schools, it is possible to calculate growth norms indicating the relative growth of students in addition to their standard based/criterion-referenced growth. For the student in the picture, their growth in level of mastery is exemplary in a norm-referenced sense with student growth percentiles of 69, 85, and 74 in 2009-2010, 2010-2011, and 2011-2012, respectively.

Progress monitoring encompasses two complementary views of time-based level of mastery: a retrospective view and a prospective view. The retrospective view in the picture above shows a student demonstrating exemplary growth and catching up from what was considered unsatisfactory to proficiency in reading. The student has experienced three consecutive years of solid progress, taking them from what would generally be considered at-risk status to a student thriving in the subject. Retrospective views of a student’s level of mastery allows parents, teachers and even students to identify strengths and weaknesses and ideally leverage any information gleaned to maximize progress going forward.

By contrast, a prospective view of student level of mastery looks to the future and lays out what the next steps on the student’s education journey might look like. If the goal for all students is to put forward and accomplish ambitious yet reasonable goals, the fan in the figure above highlights the range of possible outcomes based upon the data of students just completing that year. The upper end of the fan indicates the upper range of achievement observed by previous students across the academic year while the lower end of the fan indicates the lower range of achievement.

To establish growth norms, states utilize data on vast numbers of students. In contexts without the thousands of students necessary to create growth norms, it is a bigger challenge to know whether the growth demonstrated by a student is impoverished or exemplary. For example, a student might go from Unsatisfactory to Part Proficient in a year in one content area. But is that fantastic progress or typical among students? A parent might believe that their student is making progress (which they are relative to the standards). But if the parent were informed that it is typical for students to progress from unsatisfactory to proficient in a year, then the progress the student made doesn’t look as exemplary. Norms help us to anchor what ambitious yet reasonable progress is. But without data to establish such norms, educators are left to apply their expert judgment (likely based upon their many experiences) on what “exemplary” looks like.

One benefit of individual level student growth data is that it enables those interested to aggregate the data and look for patterns. Do some curricula support greater levels of student growth than others? Are there schools where students grow faster than others? If so, why? Student growth provides another lens through which to understand student performance, with the hope that corrections can be made early enough to maximize the chances for a student to reach their level-of-mastery goals.

Examinations of student growth are often considered confusing because of the complicated calculations that take place to create the results. The analyses utilize regression techniques and all the data available to model expected levels of growth of students. Monitoring students’ progress throughout their educational journey is not as complicated as the calculations might suggest. Often the most difficult parts are the record keeping required to keep track of students as they pass from grade to grade in the education system (changing teachers and schools) and the creation of grade-level performance standards against which to monitor student progress. Once those features are in place, one has all the necessary components to chart student progress along their education journey.

Damian Betebenner is senior associate at the Center for Assessment. [email protected]

Figure 5: Sample NAEP Science Interactive Task (http://www.nationsreportcard.gov/science_2009/ict_tasks.asp).

The rapid changes in education brought about by contemporary technology and the drive toward “21^st century education” challenge educators to develop new tools for assessment. With the growing consumption of cheaper technology, many schools have begun to focus on STEM (or STEAM), design thinking, engineering, innovation, and the like. How do we measure the ways in which our students are interconnecting ideas, making meaning and designing new prototypes? NAEP has developed a tool to measure the way students apply their knowledge in Technology, Engineering and Literacy (TEL) (www.youtube.com/watch?v=uexguF1674k). In this 60-minute interactive scenario-based task, students have the challenge of fixing a broken well in a remote village. They have to play the role of an engineer, and get at the root of the problem to help the villagers fix their well. They conduct research to get background on wells, talk to people in the village about the well, and identify the cause of the problem to fix it. As we embark on the administration of NAEP for the first time at Yavneh, we will have the advantage of getting feedback about how well our new approach to design thinking and STEAM learning in the 21^st century can be measurable.

Answering our Stakeholders

For individual student reports that are shared with parents, students and teachers, we use the standard reports provided by NWEA, CPAA and MSMRI. These reports are relatively easy-to-read and interpret. They can help us answer our stakeholders in the following way (names and data changed):

Parent: Maya is well above grade level. She is in the top 95th percentile for reading, and top 80th percentile for math, so she is eligible to apply for the Center for Talented Youth program, offered by Johns Hopkins University. (from the NWEA MAP)

Teacher: Ava identified the sentence with the correct capitalization of the first word and names of people, but only after a hint. Perhaps, I can reinforce this concept in my work with her and the other students who struggled with this same concept. (from CPAA)

It seems that Eli should work on understanding how ½ could be used as a benchmark number when reasoning with fractions. (from the MSMRI)

Student: I grew 15 points since fall on the math assessment. But I could still use some more practice in measurement and geometry.

As we embark in analyzing aggregate reports, at the school and grade level, it is important to have someone who understands statistics. In addition to having some statisticians on staff at Yavneh, we have also had the benefit of working with Hanover Research, which helps K-12 organizations make data-driven decisions to impact and improve student learning. With an outside lens, and 100+ researchers on staff, they help us think about our strategic plan goals, and analyze our raw data from student test scores. With their 10,000-foot view, they can offer the following answers to our stakeholders:

Principal: 100% of our students are at or above grade level, demonstrating that they are ready to take on the challenges of school, no matter where they go. 35% of the students qualified for the Gifted and Talented programs offered by Johns Hopkins University.

Board member: The histograms show, for example, how many students in our 5th grade achieved at or above the 90th percentile on the test in language usage compared to their peers. These are national comparisons with samples randomly drawn from 5.1 million students across 13,000 schools, public and private.

Head of school: We can market the advantages of attending our school by demonstrating that we value individual student growth; our assessments are adaptive, offering an individualized experience to meet the needs of the students. In addition, we can market the design thinking/STEAM approach, with our willingness to try new assessments like the ones offered by NAEP.

Director of Jewish studies: Our test scores from the Writing Assessment Program (WrAP) by ERB suggest that we need to work as a cross-disciplinary team to develop students’ ability to cite evidence as they respond to prompts in argument, informative and narrative genres.

Director of admissions: 85% of our students in 4th-5th grade are in the top 90th percentile of students nationwide in mathematics. This is good information to share with my prospective families. (from NWEA MAP)

As you respond to your stakeholders and take stock in the system of measurement in your school, remember how important it is to determine which assessments make sense for your community. Your assessments must align with your philosophy of learning and move your school towards the vision you have set.

We are fortunate to have options, and as we explore these options, we must continue to be on the lookout for new and innovative assessments that follow the trends of learning in our school community. For years there has been a perceived lack of alignment between what we value as education and how publishers have produced tests for consumer use. The new innovations in assessment show how far the test publishers have gone to further support what we believe as meaningful learning. In fact, the assessments themselves have turned into opportunities for real instruction. This is a blessing.

Diana Wilmot PhD is the chief academic officer/principal of Yavneh Day School in Los Gatos, California; she is the former director of research, evaluation and assessment for the Palo Alto Unified School District, and the current president for the California Education Research Association (CERA). [email protected]

As debates rage on over education reform, assessments are a hot-button issue, not just for educators but also for parents and policymakers. The creation of the Common Core State Standards and the subsequent development of the two primary assessment tools that launched in the 2014–2015 school year have kicked off a new flurry of media coverage and debate about student assessments not seen since the early years of No Child Left Behind.

Spurred in part by concerns about US students’ preparedness for college and career, as well as their ability to compete with peers from around the world, educators and policymakers have reinvigorated efforts to make changes to K–12 academic standards to get the American education system back on track. Test developers, including the College Board, ACT, ERB and others, are quickly following the lead, working to reconfigure their assessment tools to keep up with the new standards.

Given the increased focus on student assessment from accreditors, data-hungry parents, and others in the education community, school leaders will be better equipped to strengthen their academic programs if they are conversant with the tools available to evaluate different aspects of their students’ learning experience.

The Increasing Need to Demonstrate Student Outcomes

Traditionally, schools have assessed students (and indirectly schools) for three primary reasons: accountability, instructional adjustment, and mission fidelity (see Figure 1).

Figure 1: Reasons for Assessing

Assessment instruments can also provide information to address the new demands from the marketplace. School choice options have expanded in the last several years, with charter schools in particular growing at an ex­ponential rate (see publiccharters.org) and being perceived as a high-quality alternative not only to public schools but also to private schools. In addition, the growth in independent school tuitions above the cost of living has left many families doubting their ability to afford them and question­ing whether independent schools are worth the financial sacrifice. This is similar to what colleges and universities are experiencing: gone are the days when the name and the reputation of a school were indication enough of the quality education it was providing. There is a stronger demand from parents, the media and the public at large for information and data that can attest to the value of an independent school education.

Another reason for collecting data on student outcomes in a more sys­tematic way is school accreditation. In February 2009, the NAIS Commis­sion on Accreditation adopted Criterion 13 as part of its standards (see naiscoa.org):

The standards require a school to provide evidence of a thoughtful process, respectful of its mission, for the collection and use in school decision-making of data (both internal and external) about student learning.

Similarly, Criterion 12 requires schools to conduct a thought­ful assessment of individual student progress consistent with the school’s mission.

Given these criteria for the accreditation of independent schools and the constituency demands for more transparency and accountability, indepen­dent schools will need to clearly articulate the difference they make in their students’ lives and how distinctive and compelling their value proposition is compared with other school options. Instruments that assess student out­comes offer schools the option to obtain this information.

The Emergence of New Types of Assessments beyond Standardized Testing

The conversation about preparing students successfully for college and ca­reer was ignited by the publication in 1983 of A Nation at Risk: The Impera­tive for Educational Reform. This report from the National Commission on Excellence in Education effectively launched the education reform move­ment that has taken us from state standards and tests to No Child Left Behind and now to the Common Core State Standards and their associated assessments. A recurring theme has been that schools have not prepared students well academically or developed the essential competencies—creativity, adaptability and global awareness—that now make up our understanding of 21st century skills. What has changed significantly along the way is that business and government leaders (and even parents) have begun to seek outcomes from graduates that are not just core academic knowledge but encompass these new competencies (see Measuring 21st Century Competencies, asiasociety.org).

The growing choice in assessment tools provides schools with a robust array of measurement options: summative, formative, or both, and cognitive, interpersonal, or intrapersonal. Deciding which tool(s) to use will depend on what needs to be measured and why. The ERB, SSAT and SAT serve a purpose (often, but not exclusively, used for admission) that is different from the Measures of Academic Progress (MAP, individual student growth), the High School Survey of Student Engagement (HSSSE, student school experience and engagement), the College and Work Readiness Assessment (CWRA+, critical thinking), or the Mission Skills Assessment (MSA, inter/ intrapersonal competencies).

What is very clear is that the era of the “bubble test” is waning (see “Beyond the Bubble Tests,” remarks by Secretary of Education Arne Duncan at ed.gov/news/speeches). Spurred on by the significant federal investment in two consortia, PARCC and Smarter Balanced, that offer assessments to evaluate students participating in the Common Core, evaluation of student accomplishment will be based not just on academic content but on what skills and competencies have been mastered to thrive in the economy and society of today and tomorrow.

Assessment and the Common Core in Independent Schools

There is a wide range of assessments schools can use, many of which incor­porate cognitive or inter/intrapersonal competencies (see Measuring 21st Century Competencies, p. 7). In May 2014, NAIS surveyed heads of school to better understand the type of assessment instru­ments they were currently using (Figure 2). The survey polled 649 member and nonmember independent schools, including elementary, middle, and up­per schools. The survey found that most schools use the SAT (63 percent), ACT (52 percent), and ERB (52 percent). To a lesser degree, independent schools use alumni surveys (47 percent) and AP exams (44 percent), while a few schools use MAP (10 percent) and HSSSE, CWRA+, and MSA (6 percent each). The same survey showed that assessments were primarily being used to assess the school’s academic program (85 percent), improve teaching prac­tices (74 percent), and demonstrate student growth (73 percent).

Figure 2: Percentage of Independent Schools Using Assessment Instruments

The year before the CCSS assessment exams were scheduled to begin, NAIS conducted a short survey to understand how independent schools were approaching the standards and, in particular, whether they were planning to adopt the new assessments. About 21 percent of survey participants were planning to fully or partially adopt the Common Core State Standards. Out of this group, 16 percent mentioned that they would be using the assessments developed by one of the CCSS consortia.

Finally, in terms of what is next in assessment, there is no question that our future will include two industry-changing trends: (1) the mainstream­ing of personalized learning strategies (PLS) and (2) student assessments (thanks to new software and learning analytics) that are specifically designed to address individual learning needs and interests (see Future Trends in K-12 Education, hanoverresearch.com). While the one-stop-shop standardized instrument will continue to be with us for some time, new tools that seek to measure learning and analytical skills will become mainstream. Expect to see a convergence between PLS and student assessments that will generate a new learning and instructional dynamic in schools as well as dif­ferent expectations from the school community: students, parents, teach­ers, and the board of trustees.

A version of this article originally appeared in the 2014-2015 NAIS Trendbook. The NAIS Trendbook compiles research about the trends that independent school leaders are likely to experience in the coming year. Available at www.nais.org.

Jefferson Burnett is senior vice president for advocacy and education innovation at NAIS. Amada Torres is vice president of studies, insights, and research at NAIS. Whitney Work was formerly the legislative director at NAIS. [email protected], [email protected]

In Jewish day schools, we are blessed to have Torah and core Jewish values as our guides for behavioral standards. At Hillel Day School of Metropolitan Detroit, we promote values that include acting with derekh eretz, performing acts of tikkun olam, and instilling our lives with kedushah. Over the course of the education from preschool through eighth grade, we constantly articulate our goal of molding our students into mensches. We work toward this goal in all aspects of the school day, whether in a Tanakh class, a social work lesson in the classroom, or in a cooperative assignment in language arts. As part of this elaborate program in menschlichkeit, in the past year we began to assess students in this area, both to measure their progress and to determine whether the school overall is meeting its goals.

So, what is a mensch? Merriam-Webster’s definition is “a person of integrity and honor.” A more Jewish definition is “literally ‘man,’ an honorable, decent, stand-up person, as in, ‘I don’t care who you marry, as long as he’s a mensch!’” (myjewishlearning.com). Because personality attributes are qualitative constructs, they cannot be measured directly. Therefore, we must identify a number of variables of which a mensch is comprised.

This year at Hillel, we have enhanced our teaching and measurement of menschlichkeit in several ways in order to maintain best practice in the social-emotional field. The first step was to precisely define and delineate the skills and behaviors that comprise being a Hillel mensch. Our MENSCH behaviors are as follows: Making a difference; Empathy for others; Nice words and actions; Sharing and collaborating; Courage to do the right thing; and Helping others in need. Now that we have a working definition, the specific actions must be described and taught explicitly in order to be executed. In this way, we can make our behavioral expectations as clear and rigorous as our academic expectations. Additionally, in order to measure the presence of these behaviors, we must know exactly what we are looking for. To this end, we have concretized the overt actions that characterize each of the behaviors.

Thus, along with the new MENSCH posters that clearly display the definition in every classroom, we also have a matrix poster of what menschlichkeit looks like in various areas of the school, such as the miznon (cafeteria), bathrooms and hallways. For example, students who are in the hallway or mercaz (heart of the school) demonstrating Making a difference in our community will keep the floor clean, smile and greet others, say slichah or excuse me, and put furniture back where it belongs. A student who is displaying Empathy for others in the hallway will help others when they need assistance.

From years of research on child development and learning (Bandura, Aggression: A Social Learning Analysis) we know that “behaviors are taught, modeled, and then approximated until they are replicated.” Because of the explicitness of the mensch expectations, we are able to teach, promote, and measure these behaviors much more effectively. We also use positive reinforcement by giving students “mensch cards,” on which teachers circle the specific behavior that they observed.

There is a consciousness about students’ intentions; they are not acting like a mensch just to earn the card. All through the week, students put their earned mensch cards into a bin in the main office, and every Friday there is a “Got ya, caught ya, being a mensch” drawing for two to four randomly chosen students in each division to win a sweet Shabbat treat. This has been a very effective behavioral reinforcer, as students are consistently vying to earn as many cards as they can each week and are thoroughly excited as a hush falls over the entire school in order to hear the lucky winners’ names over the loudspeaker. The subsequent cheers of support from the students’ classmates and the list of names outside of the office are certainly secondary reinforcers as well.

In order to ensure that students are increasingly learning and demonstrating menschlichkeit, we must consistently assess their progress on the spectrum of these defined behaviors. Although assessment was typically thought of as measuring discrete bits of knowledge or dichotomous constructs, it can also be applied to qualitative constructs, such as personal characteristics (Boyle, Saklofske and Matthews, Measures of Personality and Social Psychological Constructs). We are able to use the behavior matrix as an assessment rubric, just as one would measure progress in an academic area. We assess students formatively while monitoring their progress, as well as summatively, as the teachers use this rubric for assigning a mark in derekh eretz on students’ trimester report cards.

At this point, having just completed the first year of this program, the derekh eretz grade is the only menschlichkeit data on record. As we continue to build and improve both our behavioral and academic systems of assessment and data collection, we will be able to store and analyze this data more effectively. In January 2016, we will be upgrading our student information system and housing all of our data in one user-friendly domain. Our teachers will have access to current and past behavioral data, progress and interventions.

As with data gleaned from any assessment, we act upon the information to best meet each student’s individual needs. In this way, we can again activate the same support structure for those who are struggling in the area of menschlichkeit that we would if a student was not yet meeting academic standards. We use our system of monthly “student concerns” meetings as checkpoints, and our social workers are in touch with each teacher in order to assist students who may be developing mensch skills more slowly. They teach classroom lessons in all grades, facilitate small groups for more directed interventions, and they support individualized needs in one-to-one sessions as well. We are also able to analyze the mensch data on a broader scale, and we can make comparisons of progress between classrooms, whole grade levels, and look at the school’s functioning in its entirety. As we know, students develop skills on different timelines, and like learning any new concept, learning to be a mensch is no different.

Since the implementation of this enhanced mensch program, we have seen a significant decline in discipline referrals and a significant increase in conscious, purposeful menschlichkeit all around our school. Our parents are also on board and have been taught to reinforce these behaviors at home using the same language that we use at school. This wrap-around model is the best way to achieve our goals for our students and create a culture of menschlichkeit in the school. Staying true to our motto, “Mind and Soul, Better Together,” we will continue to foster these skills and use assessment to guide us.

Jennifer L. Friedman PhD is the dean of student learning and educational psychologist at the Hillel Day School of Metropolitan Detroit. [email protected]

Eleanor Duckworth is a professor of education at Harvard Graduate School of Education. Among the accomplishments for which she is known is a form of student self-evaluation embedded in the learning process which she describes below. She generously agreed to an interview with her student, Debra Shaffer Seeman, to share her insights with us.

To introduce our conversation, please tell us about your philosophy of student learning.

I started out as a scholar interested in children’s thinking, as a researcher talking to children and hearing them tell me how mountains actually change shape when you drive around them and how there are more steps going up a staircase than going down. I was fascinated that there was no way to change what they actually believe by telling them differently. And even more fascinated by what they did if they could not figure out how to resolve a conflict in their own thinking. My question over the years, first as a student of cognitive psychology and then as a student of teaching has been, “How do people learn things and what can anyone do to help?”

Children really want the world and their ideas to make sense. The same holds true of my adult students. Students have the habit of waiting to be told what the right answer is. If you don’t give in to that and instead give them the means for figuring it out themselves, learners find they can do most any kind of thinking with their minds and they learn deeply.

So what role does assessment play in your view?

I’m not crazy about the word assessment. I prefer the word “evaluation,” which puts values to things and doesn’t necessarily put numbers on them. There is a huge role for evaluating in education and it needs to be done “along the way,” which is to say that the way of teaching is to have the students do the talking and then a teacher shapes her work based on what she sees as the student’s understanding. The teacher’s job is to set up the class so that the teaching session and the learning materials are rich enough for the students to do the talking. Only by listening to what the students are saying and how they are making sense of their subject matter will a teacher be able to decide what ought to come next. I try to put the students in direct touch with the subject matter, rather than my view of the subject matter. It’s not a matter of mediating between the subject matter and the learners. It’s not a matter of telling them how to think about it, but keeping the learners directly in touch with the subject matter itself so that the subject matter becomes the authority.

Tell us more about “putting students directly in touch with the subject matter”?

Essentially, the process of teaching is an exploration of students’ thoughts while they are exploring the subject material. This is about getting them interested in the subject matter and not telling them what to think. The teacher is the person who gives the students something worthwhile and interesting to think about without being the authority. The center of attention becomes the subject matter, and the group’s different ideas about the materials provided. To do this, we set up the situations that provide something interesting for the students to talk about. When the teacher becomes a listener, they know what a productive next step will be.

Critical Explorers, the curriculum development project that some of my students have initiated, created a unit on the Industrial Revolution that will illustrate this well. Grade 6 students look at butter molds and ask questions about why the molds have decorations or how they open and close, and how they are worn; they read advertisements in newspapers about butter sales; they read a farmer’s journal that shows where a dairy would optimally be placed in the layout of a house to make the butter production most efficient. They move into other historical documents that debate whether a farmer ought to make her own butter or send her milk to the dairies, which were starting to do the butter-making. They investigate what that required for transportation to get to the market, including refrigeration and trains.  They read different views, published in newspapers of the time, and themselves write such letters, from different points of view.   These students just got to the whole of the Industrial Revolution through firsthand accounts of women's butter making on a farm. The kids love this process, though they do sometimes complain that they have to work hard this way. They like to say that they are solving mysteries and become completely engaged in the process.

 In what form would you like to see evaluation of learning happening?

In order to answer that I’d like to talk about evaluation for whom and at what level. The teacher wants to know what’s going on with her own teaching in order to know what to do next. Teachers also want to give a report to parents about a particular student’s learning and progress. They want to be able to say about each kid, “Here are John’s achievements, his strengths, and what he enjoys.”  Parents want to do their own evaluation of the teacher and school. The principal wants to know how the school is doing and is less concerned with an individual student, but rather needs to evaluate a particular teacher or a curriculum. And then there’s a superintendent who has to deal with many schools and the secretary of education looking at educational policies and overall systems, and so on. At the moment, the only information that we have to inform all of these stakeholders are test scores, which is terribly unfortunate. The one number, the test score, is the only thing available to this whole array of stakeholders.

I think there needs to be sampling at every level so that a principal who wants to evaluate a certain curriculum, for example, would choose a particular unit of that curriculum, then take a sampling from among the four seventh grade classes and look at 5-8 students from each of those classes. That principal would take time to learn thoroughly what they make of this part of the unit, how they’ve come to understand it. Not just using pencil and paper or test scores to evaluate the students’ learning. There could be longer, more complicated sessions where the principal explores the students’ understanding of the curricular material.

What are you looking for in that kind of evaluation?

When I evaluate the work of a class, I’m looking for collaborative work, variety of ideas, depths of those ideas, and a variety of the kids who had the ideas. I once did an evaluation of an African elementary science program that incorporated the same principles that I use for student learning. I created the evaluation tool based on a conversation that I had with Phil Morrison, a physicist who gave an exam in Cornell University’s undergraduate lab. In Morrison’s lab, each student was given the same set of materials with no questions at all. The entire exam was to find a question and answer it.

In Africa, I knew that I couldn’t give comparable tests to two sets of kids, some of whom were in the program and some who were not, because they didn’t study the same things. I had to invent a new form of evaluation based on material that neither group had studied. I took a sample of 10 students and gave them a collection of materials, some that every kid had seen in their life: things like foil from cigarette packages, rubber rings from inner tubes of tires, straw, water, various containers, flashlight batteries, and bulbs, and stuff that none had seen (various construction materials, wheels of different sizes, etc.). I told the kids they could do whatever they wanted to do with the materials and that they could talk with each other. I kept track of what they did and the complexity with which they created things. I didn’t speak with the kids since we didn’t share a common language (of course sharing a language means that a person can engage the student’s learning in many additional ways).

With an assistant, I carefully tracked each group for the same amount of time and followed how many ideas a group created, how they worked together, and how far they were able to take their ideas. Participants in the African elementary science program had a wider variety of ideas initiated by a greater number of kids in the groups. They were also just reaching high gear in what they could do with all of this “stuff.” The kids who were not participants in the program had less variety of ideas, which were shared by fewer initiators—and by the end of the hour, they had just about run out of the ideas.

What kind of advice would you give to administrators and teachers as they develop a strategy toward assessment?

I would give students documents or materials on a similar or related subject to the unit they had just completed, to see how they act as historians or scientific thinkers, like in the African elementary science evaluation.

Another way to enact this kind of evaluation is the presentation of student work. Students can be sent to talk about their own work to other teachers, school board members, principals and other stakeholders—in their own words, with their own excitement. People are always impressed with student work. They love to see their building, writing, calculating and creating—the pieces that speak directly to what the students have learned.

As an overall strategy, I would do sampling at every level other than the classroom. I would almost never do pencil and paper evaluations, which sometimes confound knowledge of subject matter with knowledge of reading and writing. There could be some pencil and paper evaluations in very specific situations following hands-on engagement with the subject matter. Though I think that it’s important to find out why kids have said what they said, which paper and pencil evaluations rarely leave room for.

A student and translator of Jean Piaget, Professor Eleanor Duckworth grounds her work in Piaget and Inhelder's insights into the nature and development of understanding and in their research method, which she has developed as a teaching-research approach, Critical Exploration in the Classroom. Duckworth has retired as professor at the Harvard Graduate School of Education; she has worked in elementary classroom teaching, curriculum development, teacher education, and program evaluation worldwide. Criticalexplorers.org

Being a “very good listener” is one of the highest forms of praise a kindergartner can receive from her teacher. “Emily is cleaning up so nicely. She is a good listener.” “Look how Daniel is doing his work. He is a very good listener.” When children are little, “a good listener” is measured by a child’s track record for sitting quietly and doing as he or she is told. Listening is often measured by obedience.

As they climb through the grade levels, “listening skills” are measured by an elementary or middle school student’s ability to hear a passage, draw basic facts and inferences and answer the test questions with accuracy.

Why is following instructions or simple summarization the measure of a good listener? This is one of many cases where our school measurements sometimes focus too narrowly on outcomes that help a classroom function (e.g., children do what the teacher wants them to do, and perform tasks at grade level in a standardized language arts exercise) rather than measuring the application of that skill in the real world. As a result, we miss the opportunity to encourage and recognize the behaviors that will enable our children to succeed as moral, upstanding human beings.

How aligned are our curriculum learning outcomes with our mission statements? In Jewish day schools, we often highlight virtues in our rhetoric, but then obscure them in our report cards. If we break down the “listening skills” checkbox on the report card into valuable life skills, we might consider these types of listening:

Listening with empathy. When we are present, available and open to the experiences of others, we can support and challenge our peers with integrity and authenticity. We can learn from them by putting ourselves in their shoes. We can make them feel heard, welcomed, valued. We can recognize and respond to a cry for help. This kind of listening is the basis for friendship, community-building, team-building at work, and the helping professions.

Listening with a discerning ear. When we take the time to internalize the arguments we hear, remove our biases and evaluate the evidence presented as well as competing claims, we can develop our own opinions rationally and thoughtfully. The public square is increasingly an echo chamber of 140-character commentaries; it takes a trained ear to find sophistication and nuance and raise the bar for intellectual debate.

Listening with conscience. When we put what we hear in dialogue with our values, we can turn listening into a springboard for moral action and protest in the face of injustice. This kind of listening is the opposite of being aloof, apathetic, or oblivious. The world is not as safe or just as kindergarten; sometimes good listening must lead to civil disobedience.

All of these habits of the ear are measures of developing EQ—the emotional quotient that is at the heart of good leadership.

In school, students can demonstrate these kinds of listening skills every day. Classroom participation can include not only one’s own contributions but the ways in which students make space for their classmates to contribute and show an interest in learning from them. Working collaboratively in chevruta pairs or groups demands generous and attentive listening. The moments when our students hear a friend’s pain and lift that person up, present a reasoned counterargument, write persuasive op-eds and launch campaigns to address problems in society. When they tell us in their own words what the call of the shofar means to them during the season of renewal and repentance. These are the measures of good listeners.

Sometimes we lose sight of measuring the virtues that are really important. Perhaps we think they are tangential to the content of the curriculum; or they are too subjective to test; or it is too resource-intensive (time and money) to write detailed narratives depicting students’ listening habits. The problem is, including only what we can easily measure only tells part of the story. To the extent that one’s report card is an artifact of one’s educational autobiography, we owe it to our children to build a more nuanced narrative of who they are, how they are growing, and how they might contribute to society.

Facing an uncertain and imperfect world on the eve of the twentieth century, John Dewey famously professed that ideally, a school is a place where students learn to build and perfect a minisociety. How they utilize their academic skills as human beings is paramount. Good listeners can change the world, not just follow instructions and summarize passages. But you wouldn’t necessarily know that from conventional report cards. In the real world, an empathetic, discerning listener who acts in good conscience can actually save lives. That’s worth measuring.

This column was inspired by the legacy of my colleague and friend Rami Wernik z”l, an exceptional educator who has been universally remembered as a very good listener, by all measures.

Dr. Miriam Heller Stern is the dean of American Jewish University's (AJU) Graduate Center for Education in Los Angeles. [email protected]

Deborah is a communication and behavior expert who helps corporations, Jewish organizations, and individuals achieve personal, interpersonal and professional success, and she serves as a lecturer of management communication at the Wharton Business School. This interview is published in partnership with the Jewish Book Council. [email protected]

Tell us about the needs you saw that inspired you to write these books.

In my role as a coach, facilitator and consultant to nonprofit Jewish organizations for more than a decade, I kept noticing common themes emerge as challenges to both the professional and volunteer workforces, ranging from dealing with difficult personalities and creating a culture of trust to empowering and motivating others to asking for money. I also kept noticing that there was no one source for the professional and volunteer leaders of these organizations to tap into to learn to get better at these skills and competencies. Of course, I also knew that any source needed to be easily accessible, highly practical, quick to read for busy people, cost-effective, and be rooted in the Jewish values and traditions that serve as the foundation for our work and our organizations.

Not so long ago, Jewish organizations often conjured the image of “not so professional” (whether fairly or not). What changes do you notice in the world of Jewish nonprofits?

I am happy to report that many of the Jewish organizations that I work with are becoming more rigorous and professionalized in certain ways: a greater commitment to identifying KPIs (Key Performance Indicators); measuring outcomes; engaging in more formal project management; elevating the role of supervisor to include coaching skills (a personal mission of mine); identifying high-potential professionals and developing them. I also notice that sometimes the assumed “niceness” of working for a Jewish organization can get in the way of Jewish professionals and volunteers making hard decisions and having difficult conversations. At the end of the day, however, you’re operating a business and need to regard it as such, which means that you can still uphold Jewish values like rachmonos and chesed while doing what’s right for the organization and for its people.

In your work consulting with a variety of organizations across the Jewish world (and beyond), what differences and similarities stand out in your mind regarding professionals and lay leaders at day schools?

I see that the commitment to mission, vision and values is the same for many professionals and lay leaders in day schools as it is for those in Federations, JCCs, Hillels, agencies, etc. What has struck me about the lay leaders in day school in particular is that they often have a personal attachment to the outcomes of their decisions that goes beyond what they think is right for the institution. It’s rooted in what they think is right for their child/children and their family. On one side, it makes them deeply passionate advocates, ambassadors, donors, stakeholders, etc. On the other side, it can make them so attached to a particular outcome that they have trouble keeping their minds and options open to other approaches.

A difference that I notice between day schools and some other Jewish organization is a hunger for and willingness to adapt to best practices from outside the Jewish world.  Some other Jewish organizations that I consult with are interested in what I’ve learned in my corporate consulting practice, but then tell me why “it won’t work here.” Jewish day school professionals and lay leaders may struggle with feelings of “it won’t work here” and then work hard to figure out what parts of it they can adapt for their school.

“Change” is a big concern for day schools, as elsewhere. Tell us some of the main difficulties you’ve observed among Jewish organizations attempting to make substantive change.

The biggest challenge I find among Jewish organizations trying to make substantive change is focusing far too much time and energy on the T and W of a SWOT model (Threats and Weaknesses) than on the S and O (Strengths and Opportunities). My favorite patriarch of modern day coaching, Tim Gallway, writes that Performance = Potential – Interference. Too many Jewish institutions try to improve their performance by focusing on how to reduce or eliminate the interferences rather than on identifying what areas of potential they can tap into and grow to make a powerful, positive change. As someone who is certified in Appreciative Inquiry, a change management model that helps people to identify what is working well, analyze the sources of the successes, and then create conditions for more of those successes to happen, I help organizations focus on possibilities more than problems.

Connecting to our issue theme, measurement, what advice do you have for Jewish professionals and lay leaders as they assess and self-assess their work?

Self-assessment is valuable in that it forces you to step back and take stock, and it’s limiting in that your self-assessment is only as strong as your blind spots permit. I do a lot of work helping professionals and volunteer engage in peer coaching and peer assessment, which adds another level and layer of distance between you and your blind spots while adding another level and layer of closeness, investment and trust between you and your peers. Rabbi David Teutsch once commented that decisions should be made by the people who will feel the pain of those decisions. Work should be evaluated by those who will feel the pain of that work—and the pleasure of that work as well.

How can school leaders assess the culture of their school?

I think that one of the challenges that school leaders have in assessing the culture of their schools is thinking that there’s one culture because they want it to be so, or they think it should be so. The fact of the matter is, there are likely multiple cultures (or subcultures) in the school: the parents, the students, the faculty, the professional staff, the board, the hands-on volunteers, the clergy, etc. Ignoring the fact that there are multiple cultures co-existing is turning a blind eye to what’s often really going on. If culture is “how we do things around here,” it must follow that there are as many cultures or subcultures as there are groups of “wes.” Part of the work of a school leader is to help the school decide where there needs to be consistency across cultures—deal breakers—and where variations in culture can peaceably co-exist.

One piece of advice that I often give to professional and volunteer leaders is this: culture change is long, hard work—and something that you may not see happen in your lifetime with this institution. What is faster and more doable for everyone is to engage in climate change. If culture is “how we do things around here,” climate is “how I do things around here,” and each person’s climate contributes to the overall culture. Figure out for yourself and your team the kind of climate you’d like to create—sunny? warm? variable? temperate?—and commit to communicating that to those around you, and to aligning your behaviors, attitudes and beliefs with the climate you’re committed to creating.

Thinking about the relationship between professional and lay leaders, what are the biggest challenges you’ve seen? What advice do you have for improvement?

The biggest challenges I see are when one or both partners doesn’t give enough focus, time or attention to both the task side and the people side of working together. Even if both partners tend to be “doers” or both partners tend to be “feelers” (or some other combination), in order to get work done and done well, you need to create the space and time for both people to contribute to conversations about what Daniel Pink calls the 4 T’s of Autonomy (Time, Task, Technique and Team) as well as to how we’re going to partner well together. Another challenge I’ve seen is a reluctance to discuss who is ultimately accountable for decisions—before a big decision needs to be made. Only one person can be accountable—have veto power or make the final choice—and you run into trouble if that hasn’t been discussed and negotiated ahead of time.

Here is my favorite game of 20 Questions, and I suggest that lay and professional partners (or anyone who works closely together) discuss these with each other as soon as possible in the relationship.

1. What do I say or have said in the past that you have appreciated the most?
2. What do I say or have said in the past that makes you uncomfortable?
3. How do you argue or disagree most effectively?
4. What disagreement approaches won’t work well between us?
5. What happens if we can’t agree on something important that involves both of us?
6. What should I never say to you, even in frustration?
7. What might I say or do to get your attention about something urgent if other approaches haven’t worked?
8. How might our relationship evolve and change over time?
9. How much room or license do we have to ask each other to change?
10. What will be the early warning signs that our work or our relationship is in trouble?
11. What can I do to make your day?
12. How do you like to receive both positive and constructive feedback?
13. What are your “hot buttons”?
14. How would you like me to remind you about my “hot buttons”?
15. What’s the biggest lesson we might be able to learn from each other?
16. Who do I remind you of?
17. What do we do if we’re both having a bad day?
18. What happens if I get discouraged about our work or our relationship?
19. What about our work together is likely to give us a recurring problem?
20. What about our work together is likely to change both of our lives for the better?

A nonprofit board, properly recruited, that institutes appropriate policies and procedures and has individual members who understand their legal duties and responsibilities is likely to be strong and effective. Combine that with accountability for vision-focused inquiry, transparency, ongoing assessment, robust discourse and mutual respect between the board and the head of school, and that board is on its way to high performance.

Assessments are the hallmark of high performing boards. Well understood by our students and parents as a report card, assessments provide insight to the board for the governance and oversight of our schools. Just as with our students, assessments involve the gathering of empirical data on defined areas of performance to make progress on expected performance and chart a course of improvement, refinement and/or further progress.

From a governance standpoint, the board of directors (sometimes called trustees and hereafter referred to simply as “the board”) of the school has significant assessment responsibilities. Assessments hold people and processes accountable. Generally, the people being assessed are the board members, the board chair and head of school (HOS), and the processes being assessed include board governance, strategic planning, committee work and the conduct of board meetings. Of course, many other aspects of governance can be assessed, such as the orientation and training processes or the development and fundraising processes. This article identifies several areas where the board must assess matters associated with their governing responsibilities but will focus on board assessments and the two critical assessments which help the board govern optimally: board chair and board member assessments. While each kind of assessment uses a set of standards tailored to what is being assessed, all assessments are designed to identify gaps between current performance and expected or hoped for performance.

Before any kind of assessment is undertaken there must be policies in place to guide the assessments. The board should agree on the process, frequency and accountability for conducting the assessments. Agreement on tools, methodology, timing, review and readiness to execute is critical to participation.

Board Assessment

In Leading with Intent, a national survey conducted by BoardSource, information was gathered from chief executives and board chairs on their experiences in the boardroom. Over 1000 boards surveyed gave themselves a B- in overall governance, with a strong desire to learn to do better. 51% of boards reported that they use a formal, written evaluation of their board. Use of the assessment process continues to gain momentum as boards seek to rise to higher levels of governance.

Assessment results identify strengths and weaknesses, create opportunities for robust dialogue around the standards to be applied, highlight priority areas of focus and also allow different views to emerge. When demographic information is also sought with the assessment (i.e... comparing responses of new and existing board members, parent and community members etc.), the assessment can also allow the board to understand the degree of consensus within the results and compare results from different demographic perspectives.

Many organizations offer assessment tools. Some resources are BoardSource, NAIS, public accounting firms and some RAVSAK members have shared their own tools on the RAVSAK Reshet site. For board assessments, a tool developed by an organization that understands governance as one of their core competencies is usually the best route. Such tools are based on standards of excellence in governance and, because they are widely used, they allow for benchmark comparisons. A comprehensive board assessment should address at a minimum the following areas:

How the board defines and charts the course for the school as embodied in its mission, vision, values and strategies, and how the board uses the assessment on an ongoing basis as a guide for deliberation, as ambassadors in the community, developing goals for the head of school and more.

Exercising oversight of the head of school, compliance matters and school finances.

Making sure there are sufficient resources to support the school through fundraising, enrollment and an effective development plan.

Ensuring the right board composition to meet the needs of the school with board members who understand their roles in the board room, in committees and in the public domain.

A solid assessment tool is a learning opportunity. In the asking of questions, board members learn the breadth of their role, and the tool helps to ensure that all board members have a shared understanding of what is expected of them. To be most effective, the assessment should be anonymous with plenty of opportunity for write-in comments.

Who completes the survey is not always obvious. Of course the board members, but query whether outgoing or incoming board members should as well. This is a question for each board to decide so that there is an accurate read on the state of the governance. Even if new members do not complete the tool, they should review it as a learning opportunity. In addition, the HOS sits on some boards as an ex officio member and thus should complete the assessment. Even where that is not the case in all but rare circumstances the HOS should participate.

Board assessments should be done every two to three years. There is no best time to start the cycle of assessment, but waiting for everything to be in place is a mistake. Bottom line: do it now.

Should you use a consultant? It is not necessary, but it can be very useful particularly as a school first gets comfortable with the assessment function or in a time of transition. The consultant will assist in the setup and customization that may be desired, work with the board to interpret the responses, analyze the data, tread through delicate issues and how best to present them, and create a broader context for the board. A consultant trained in governance provides recommendations on how to move forward with the results, knows how to present to the board, and understands how to enable the board to make the results their own. A consultant can also to work with the board on the sequencing and priorities of implementation.

Not every survey results in the creation of a governance task force to design and implement changes, but boards should not underestimate the importance of working with the results. Oftentimes, immediately following a board meeting to discuss the results, boards set up task forces, address the low hanging fruit and decide on priorities for change. The actions taken could be a defined statement of roles and responsibilities for board members, the creation of training to understand the role of the HOS and their staff and define micromanaging, training the board in development, understanding risk management practices, succession planning and more. The range is as broad as the areas addressed, and the nuances with such a study leave much to ponder. Governing is serious work, requiring a rigorous response to an assessment, using it as a guide to higher performance. Implementation cannot all be done at once, but the assessment results should not sit on the credenza with old personnel manuals.

Assessment of Individual Board Members

To improve board member performance, provide self-awareness of his or her role, determine board leadership, provide a guide for expectations of board performance, set an objective basis for renewal of board membership and more, board member assessments are critical. If a board member is not performing, a policy to evaluate all board members on a regular basis provides the opportunity to counsel, correct or not renew board service.

There are tools or simple checklists that can be developed to assess performance based upon the articulated roles and responsibilities of the board members.  Some examples are assessing committee work, board preparation, attendance, meeting conduct, and the giving policy. Depending on the use of the board member assessment, it could be a self-evaluation, an evaluation by a designated committee (in a high performing board it would be the governance committee), or a combination of both, which I consider a best practice. Some self-evaluation implies ownership of the results. It allows board members to look at their own accomplishments and judge whether they have lived by the articulated roles and responsibilities, to see if the orientation and training worked and to provide a basis for ongoing reflection and learning.

Frequency is important. It should be expected and used to enhance board engagement and participation. Often boards rate their own performance at the same time that they rate board performance. At a minimum, individual assessments should be done with new board members after the first year to help them govern and learn, and also for those being considered for another term if multiple terms are permissible in the bylaws. Individual assessments can be a rewarding experience as board members commit themselves to governance and learning; it can also be a delicate and difficult experience which may provide the basis to terminate or not renew a board member. In any event, the school deserves effective leadership, and a process to continually assess and coach performance helps achieve that.

Chair Assessment

Boards need a chair that will facilitate a productive leadership culture, ensure effective dialogue at the board table, keep the focus on the mission, work to engage each member, select committee leadership and help the board as a group feel appreciated and able to rise to high levels of leadership. The chair has a significant influence on the organization. When a person assumes the role of chair, it should be clear that there is an assessment process in place, designed to coach and guide the chair for the good of governance and the board. Just because a member is elected to chair does not mean they are equipped to lead, and like all assessments, the process should be a learning opportunity measuring, among other matters, individual member engagement, facilitation, mission focus, and appointment of board members to tasks.

Clearly, not all chairs come with a built-in desire to receive performance feedback. The process of giving the chair the coaching and feedback needed is critically important; it should be done privately, not with the full board. Ideally, the assessment would be done annually by the governance committee, with board members providing input anonymously. It may also be useful to have the assessment done external to the organization, by outside counsel, the audit partner, a consultant, coach or past chair. Because this feedback is so personal, the process should be agreed to with the designation of the chair so that the elected chair values the process and the results.

Head of School Assessment

It is the responsibility of the board, usually the full board, to participate in the assessment of the HOS. This assessment process is one of the key board responsibilities as the entity to whom the HOS reports. HOS assessments have been addressed elsewhere in HaYidion (see “Head Support & Evaluation Committee: A Win-Win-Win Strategy” by Dr. Steven Lorch, Summer 2009); it is beyond the scope of this piece to describe the process, but it must be included in any mention of critical assessments undertaken by the board.

Assessing Committees, Board Meetings, Strategic Planning

Each function of the board benefits from assessments; three others are highlighted below.

For committees the assessment should be very specific to the tasks, and address such matters as, Did the committee achieve the purpose of its portfolio? Did the committee advance the work of the board? Did the committee work effectively with staff? Did the committee chair lead effectively and engage the board members? Such assessments should be done on an annual basis and completed by board members and associated staff. The results inform the committee as the portfolio is refined and each yearly agenda developed.

Board meetings should be assessed at the conclusion of each meeting and even after some conference call meetings. Questions might include, was the meeting focused at the right level? Did the board member feel his/her time was valued? To what extent did each of the specified agenda items fulfill its purpose? Were pre-meeting materials useful? Did each board member feel engaged? Did the time for the meeting flow well, with sufficient and diverse discussion throughout? Suggestions for board follow-up and discussion should always be made. This can be done with a simple survey tool. The critical piece is to seek the feedback immediately following the meeting.

There should be tools in place for ongoing measurement of the schools’ strategic planning. Many organizations are now using dashboards for a ready look at enrollment and retention, finances and other factors which lend themselves to numeric and graphic representation. Increasingly, sophisticated metrics, surveys and logic modeling are also effectively in use and now being required by funders. All these measures should point towards the mission and goals of the school as developed by the board in strategic planning.

The idea of assessing a strategic plan is to ensure that it is not static, but rather a directional document that mandates learning, improvement and progress assessments. Schools must continue to learn and be receptive to changing needs in their community, even as they pursue specific strategies. Strategic planning assessments are a key function of the board on an annual basis. Each year, whether in a meeting or a retreat, the board should look at the data to determine progress on defined areas of performance, demographic and other environmental changes, and chart a course for improvement, redefinition and further progress. While not the classic assessment tool, any discussion of the board’s role in assessing the organization would be incomplete without this mention of strategy assessment.

In closing, just as a school’s mission and vision describe where the school wants to go, assessments tell the school how it is doing on that journey against goals set by the board and against industry standards adopted for good governance. But while “Not everything that counts can be counted, and not everything that can be counted counts” (William Bruce Cameron), a board that views assessments as tools in the learning process is prepared to embrace the rigor and  professionalism of board service that our schools deserve.

Ann Cohen is an executive business consultant who founded Ann Cohen & Associates in 1999, combining her business and nonprofit experience to enable nonprofits to identify and achieve their strategic goals. Ann is also a BoardSource Senior Governance Consultant, where she provides consulting and strategic advice to boards, their key leadership and executives seeking to rise to greater levels of high performing governance. [email protected]

Description

A teacher and an administrator sit at a table. After they briefly exchange pleasantries, the administrator opens the meeting by asking, “How’s it going?” The teacher responds at length, referring several times to a self-assessment she had completed and submitted in preparation for the meeting. She reflects on the successes and challenges in her work, areas of satisfaction and accomplishment and other areas of unease and struggle. Referring to the self-assessment as well, the administrator interjects periodically with probing questions geared to prompting the teacher to think more deeply, or in a different way, about an issue, or a success, or a challenge that she raised, or to encouraging her to reflect and give an update on an issue that they had discussed previously but she hadn’t mentioned.

Along the way, the administrator commends the teacher for her successes and offers encouragement, perspective and an occasional suggestion regarding challenges. After a while, the teacher turns to a different part of her self-assessment and presents three goals that she is proposing for her professional development over the coming weeks and months and, for each, an action plan consisting of a series of steps geared toward achieving it. The teacher and administrator discuss the goals and action steps and work on sharpening and refining them. As the meeting ends, each participant thanks the other: the administrator for the teacher’s contributions to the school, her students, her colleagues and the school community, and the teacher for the administrator’s support.

Classification

What is this? Because it is a meeting between a teacher and a supervisor, it seems like a supervisory conference. But it’s not a typical supervisory conference because it isn’t based on a lesson observation, it doesn’t feel hierarchical, and it doesn’t result in a “satisfactory” or “unsatisfactory” rating. It explores the teacher’s insights into her own practices and leads to goals and action steps, and so it seems like a professional development experience. But it also has elements in common with a supervisory process: evaluation is central to this meeting, both in the teacher’s self-assessment and in the supervisor’s guided questioning and feedback. Moreover, the meeting is memorialized in a written summary, signed by the administrator and by the teacher, just like a formal supervisory document. So what is it?

It’s both, actually: a form of supervision that revolves around professional growth, and a form of professional development that is informed by a performance evaluation. At the Solomon Schechter School of Manhattan, by limiting administrators' supervisory loads to 10-15 teachers, we ensure that a meeting of this kind takes place once or twice yearly for each teacher. We call it goal-setting and professional development.

What It Is

This model of teacher supervision and professional growth is founded on five assumptions about the optimal conditions for teacher learning.

Teacher-directed. Since the 1970s, two of the most widely replicated findings in studies of adult learning have been that adult learners prefer self-directed learning, and that they learn best when they are self-directed (Knowles). In the goal-setting and professional development process, teachers direct their own evaluation process in two key ways: they complete a self-assessment before the supervisory meeting, including setting their own goals; and they do most of the talking and thinking during the meeting, while the supervisor supports the process by offering commendations, asking clarifying and probing questions, making suggestions and taking notes.

Growth mindset. Carol Dweck popularized the insight that both adults and children learn better and accomplish more when they adopt a growth mindset, a belief that abilities can be developed and are not fixed. In this system of teacher supervision and professional development, all faculty, veteran teachers, novice teachers and seasoned administrators alike, engage in a process of continuous improvement. Each practitioner is in ongoing dialogue with a supervisor, focused on progress towards goals, additional and revised action steps, and new challenges that arise.

Reflective practice. A key to professional learning is reflective practice (Schon). This model of professional development and teacher supervision incorporates what Schon terms “reflection-on-action.” The teacher completes a self-assessment, which becomes the starting point for a conversation between the teacher and the supervisor in which they together explore the teacher’s actions and ideas, trying to understand them deeply through inquiry and probing.

Trust and honesty. Schools in which relationships among adults are characterized by trust promote improvement by lowering teachers’ vulnerability and facilitating problem-solving (Bryk and Schneider). The goal-setting and professional development process helps teachers feel safe and supported, empowering them to present a balanced perspective of their performance and to share areas of successful and improved practice, as well as those that remain in need of improvement. Supervisors celebrate successes and address challenges collaboratively with teachers, seeking shared understanding and commitment and developing plans for improvement.

• . Strong collegial relationships in schools, particularly when they are institutionalized in practices that promote teamwork among teachers, are associated with teachers’ professional development and growth and with improvements in classroom practice (McLaughlin and Talbert). In this supervisory and professional development approach, it is hard to discern hierarchy in the relationship. The supervisor and teacher work together over time on the teacher’s practice, and the supervisor supports the teacher in developing goals and action plans for improvement.

By adopting research-based strategies for promoting adult and professional learning, our school’s goal-setting and professional development process maximizes the opportunity, motivation and support for teachers to improve their practice throughout their careers.

What It Isn’t

The goal-setting and professional development process is similar in some ways to traditional teacher supervision approaches, such as formal observation and feedback, but differs in other ways.

It’s not a system of quality control. Teachers are not separated into categories, with underperforming teachers sorted out and adequate teachers sorted in. Systems of quality control have proven ineffective in providing guidance to teachers regarding their developmental needs or support for professional improvement. Moreover, they fail even on their own terms, resulting implausibly in 99% of teachers being rated satisfactory and only 1% unsatisfactory (Weisberg et al.).

It isn’t based on the observation and in-depth analysis of a single teaching episode. Observation-based supervisory systems put both teachers and supervisors in situations that are contrived, unrepresentative, unreliable, and overly narrow in the scope of skills and dispositions assessed (Tucker and Stronge). But that’s not to say that the goal-setting and professional development process ignores observable teacher behaviors. To the contrary, at Schechter Manhattan the supervisor is a frequent (at least weekly, and sometimes daily) visitor to each teacher’s classroom, usually for a few minutes, sometimes for an extended (15-minute or longer) stay. The observations and impressions gleaned from frequent sampling of teachers’ practice produce deep insights and intuitions that form the backdrop and context of the goal-setting meeting.

It’s not an infrequent professional conversation between the teacher and the supervisor. Supervisors and teachers have weekly or biweekly interaction over issues of curriculum, teaching and student progress, and the goal-setting and professional development meeting is just one of many forums in which these discussions take place. The regularity of this contact ensures that teachers are growing continuously; as well, it enhances the effectiveness of this teacher supervision and professional development model, making it more trusting, genuine and collaborative.

It’s not the sole approach to supervision and professional growth used by the school. A different system is used with beginning teachers in their first year in the school, one which is more didactic and directive, while still being growth-oriented and collegial. And yet another system, called corrective action, is used with teachers whose challenges are sufficient to raise questions about their continued employment. Because goal-setting and professional development is a very different process from corrective action, the transition from one to the other, and back to the former for teachers who successfully address their supervisors’ concerns, is unmistakable.

It’s not the sole approach to professional development or to evaluation used by the school. While the goal-setting and professional development process is the primary system set up to ensure individual teachers’ growth, many other practices and processes contribute to professional growth, including co-teaching and other forms of teacher collaboration, regularly scheduled grade-level team meetings, meetings with subject coaches and child support teams, teacher-led professional development working groups, departmental and divisional meetings, schoolwide workshops and financial support for graduate study and participation in conferences and workshops. Similarly, evaluation is not limited to goal-setting and professional development meetings. The school conducts parent surveys, student surveys, faculty surveys and alumni surveys, reviews and analyzes standardized test performance and alumni academic performance, undergoes accreditation and strategic priority reviews and invites academic research.

Conclusion

Teacher supervision presents Jewish day school leaders with a challenge and an opportunity. While avoiding the pitfalls of ineffective systems of supervision and evaluation that are in widespread use, these leaders can create systems that strengthen their schools’ culture and values, the quality of teaching and learning in their classrooms and the satisfaction and loyalty of their parent customers, teacher employees and student charges. Goal-setting and professional development is one such system that works.

To Learn More

Bryk, A. and B. Schneider. Trust in Schools: A Core Resource for Improvement.

Dweck, C. S. Mindset: The New Psychology of Success.

Knowles, M. S. Self-directed Learning: A Guide for Learners and Teachers.

McLaughlin, M. W. and J. E. Talbert. Professional Communities and the Work of High School Teaching.

Schon, D. A. The Reflective Practitioner: How Professionals Think in Action.

Tucker, P. and J. Stronge. Linking Teacher Evaluation and Student Learning.

Weisberg, D., S. Sexton, J. Mulhern, and D. Keeling. The Widget Effect: Our National Failure to Acknowledge or Act on Differences in Teacher Effectiveness.

Benjamin Mann is the head of school, and Dr. Steven Lorch was the founding head of school and is senior advisor to school leadership, of the Solomon Schechter School of Manhattan. [email protected], [email protected]

Tefillah education, including the actual davening, the range of skills, behaviors and dispositions required to do it well and with intention, and the lessons and instructions that often accompany it, is rarely assessed. This is particularly true for the affective areas. No one is dismissing the difficulty involved; it is far easier to assess the information a student has acquired in a course than it is to assess skills and even harder to measure how our students feel about or make meaning from tefillah. How can we understand a student’s relationship with tefillah, either as a personal spiritual practice or as a means to connect or strengthen one’s commitment to Jewish values and rituals? Just because something is difficult, however, is no reason not to do it.

At a recent symposium on tefillah we ran for day school educators, entitled Aleinu Leshabe’ach, assessment was one area we explored. When we started, there was little agreement as to the need for assessment in tefillah education, particularly in terms of affective goals. Our staff was not looking for consensus in terms of goals, approaches, content, or method. We were looking to build consensus in terms of the need for assessment—a glaring need for the entire field of Jewish education.

It is important to clarify that we are not equating assessment with grades. Assessment is a way of measuring the extent to which we have met our goals. Grades are one type of assessment, and they may be applicable to some aspects of tefillah education, particularly for content and skills. But even in these areas, most schools seem reluctant to give grades, justifiably worrying about the message a letter grade sends. What does it mean to give a student a B- in prayer!?

There are many reasons as to why assess. Some apply more directly to areas of tefillah education than others. We certainly want to make sure that what we choose to teach is actually having an impact. Assessing what individuals have learned allows us, as teachers, to re-teach or modify what we are doing. Assessment also signals to students what we—the school and the teachers, and by extension the parents and community—value. We measure what is important. If we don’t assess, then in the eyes of our students, we are making a strong statement that tefillah education can’t be all that important. This message undermines the centrality of tefillah education.

Tefillah is one area of Judaic studies that our students will almost all encounter throughout their lives, regardless of how they choose to identify as Jews. In spite of this, it is also the area where we see the most pushback from students, the greatest amount of indifference (sometimes even negativity), and the lack of clearly defined, age-appropriate goals and curriculum. Students learn some content and skills, but mastery is far from universal. We would not accept this in other areas of our program.

We need to start with the goals; once our goals are articulated, what we need to assess is clear. Many schools will claim they have goals for tefillah. Unfortunately, sometimes the goals that appear on a website or frame a curriculum are unrealistic, wildly beyond what the school can deliver. Who could argue with wanting to give our students the ability to navigate the siddur, an understanding of the development of prayer, the meaning of the prayers themselves, the ability to successful decode all the prayers, participate and lead services (weekday, Shabbat, etc.), a connection with our Maker, a sense of awe and wonder regarding the world and creation, a way to make prayer personally meaningful and more? The impossibly broad scope and lofty aspirations make the articulation of more realistic goals more difficult. It is no wonder that we shy away from assessment. How could we possibly succeed at all of this—unless tefillah became the sole focus of the Judaics curriculum?

Clearly, a school needs to make decisions regarding its key goals, and therefore, what it is willing to give up. As with Tanakh Standards and Benchmarks, each school needs to tailor its own set of goals based on its mission statement, the time allotted to the subject, the community, grade level, school culture, etc. Once the goals have been carefully selected and articulated, then the appropriate types of assessment can be chosen.

Cognitive and Skill Goals

We know how to assess content and skills in other areas of our curriculum. We know how to measure what students know (in this case it could be any of a number of content goals, such as vocabulary, the structure of the siddur, or the history of tefillah). We have the tools to measure tefillah skills (such as decoding, finding recurring roots, reading Torah, leading services—again depending on the goals of the school). Assessing skills may involve more time, and strong staff organization to make sure it actually gets done. But regardless, if the goals are clear, then choosing how to assess is not difficult.

If, however, we assume that because students sit in services for 30 minutes a day for 9-13 years that they can read, lead or understand what the prayers say without actually assessing, then we have a problem. In an action research project four years ago, we heard from disgruntled 9^th grade students that after years of sitting in daily services, they still didn’t know what the prayers meant. This was a case where tefillah education was limited to actual davening, with some explanations by a teacher; to achieve certain cognitive and skill-based goals, tefillah education needs to take place in the classroom as well. It is possible that we actually decide that for certain ages, tefillah education should be primarily teaching in the classroom and not involve davening. There are schools, such as Tanenbaum CHAT in Toronto, that have made that choice.

Assessing the Affective

This is the area of assessment that seems most difficult for educators. Even if we agree that we should assess the affective, how can we measure a student’s passion for prayer, her connection to God, his sense of awe and wonder? On the other hand, how can we afford not to? We want students to feel positively about prayer in the hope that it will encourage them to pray in the future. If they are not feeling positively, we want to know that as well—and do something about it.

Some question the validity of assessment unless it is objective. Why? We need to come to terms with the fact that assessing the affective is more of a subjective than objective undertaking. If we are trying to measure the extent to which our students feel and make meaning out of their learning and their experiences, then who better to judge this progress than the students themselves?

Therefore, why not ask the students? In doing so, we can assess either the group or individuals. To see the bigger picture, we can design questionnaires with statements that students can either agree with or disagree with (on a scale of 1-5) that will show where they are at the beginning, middle and end of the year, or before and after an intervention. Prompts such as “Tefillah is a meaningful time of the day for me” or “I know what most of the prayers mean” can provide important insight into our students’ thinking.

Have students write. They can express themselves through words or pictures in tefillah journals, writing freely or responding to a prompt. They can compose essays after completing a unit, such as one taught by a middle school teacher on the theme of transcendence and imminence. Students could be asked to write about their relationships to God, whether they feel close or far and when. There are thinking exercises—“I used to think/believe X and now I think/believe Y”—that can encourage students to reflect on how their thinking has changed in the area of tefillah after studying a specific prayer, disposition or theme. Students need to know that there is no one correct answer, but that there are nevertheless guidelines (determined by the teacher) to follow when it comes to expressing their own understanding of their tefillah experience.

The arts can play a critical role here. Students can do a photomontage, paint, or write poetry after studying parts of Pesukei Dezimra that focus on awe and wonder regarding creation and our world. These types of activities help students engage creatively, and express thoughts and feelings in non-verbal ways.

Reflection in small groups can be a useful tool. Discussions can help them not only to reflect on their own feelings, but expose them to the thoughts and questions of their peers. Private conversations are also important. Schools could set aside time periodically to check in with a peer (a spiritual havruta) as to where they are in terms of their own tefillah-related beliefs and feelings. Chicagoland Jewish High School has assigned each student a faculty member who engages in a check-in with each student on a monthly basis.

Asking students to express their feelings and beliefs about the experience and content of prayer allows them to grapple with core issues and meaning-making in tefillah education. For students to be willing to open up, there clearly needs to be an atmosphere of trust, with the understanding that prayer is difficult for most people, including the rabbis. Most important, making time for these assessments signals to students that this is a valued part of their school program and a critical resource in their Jewish development.

We, as teachers, can also assess by observing students in tefillah, but we need to be careful lest we make unwarranted assumptions. Certainly the students who immediately sit down after the Kedushah (as though they finished reading the rest of the Amidah silently in 3 seconds) are not so engaged. We sometimes assume incorrectly, however, that a student on the wrong page or one who is silent is not involved in prayer. Students engage the experience of prayer in a variety of ways. Conversely, the student with her finger on the right line on the right page all the time might be the most obedient, not necessarily the one most fully engaged. Also, a seemingly disinterested student provides an opportunity for a conversation. Teachers can approach students who seem disengaged and ask why. Asking and listening is a way of assessing, and signals students that you care about their experience of tefillah and their inner lives.

The question remains as to how would we give feedback and communicate to the students and parents the results of what we have assessed in the affective sphere, short of giving grades. This goal can be achieved by one-on-one discussions with students, comments on written work and in report cards, as well as sharing with the parents at as part of parent-teacher conferences.

As we have tried to illustrate here, the task of assessing all aspects of tefillah education is within our reach. It is not simple, but for those who value this important field, it is a must. We simply cannot pretend that tefillah education is an important part of our curriculum if we continue to ignore assessment.

Dr. Susan Wall is the director of the Pardes Educators Alumni Support Project. [email protected]

Schools must take seriously the need to provide early support to students who show signs of struggle. Far fewer students would lag behind their classmates academically and require remediation if our Jewish day schools were to invest in a systematic process of schoolwide prevention in the form of solid teaching practices, high quality curricula, assessment and intervention. With this proactive paradigm shift, Jewish day schools will be able to meet the challenges of the ever increasing diversity that exists in our schools and begin to identify and support those most in need.
Writing about the importance of early academic intervention, I am reminded of the words of Winston Churchill in 1935: “When the situation was manageable, it was neglected. Now that it is thoroughly out of hand, we apply too late the remedies which then might have affected a cure.” 
According to the National Institute of Child and Health Development, by the 4th grade, two hours of specialized daily instruction is required to make the same gains in student achievement that would have resulted from only 30 minutes of daily instruction if begun when the child was in kindergarten. Another daunting statistic from the NICHD is that if students are not reading at grade level by the third grade, the odds that they will ever read at grade level are 1 in 17. The reason for this is that in grades 1 and 2 students learn to read and from grades 3 and up students read to learn. In other words, reading instruction takes place during the first two years of schooling. If children have not learned to read by the end of grade 2, it is unlikely that they will have the formal reading instruction to teach them to do so.
In order to identify those students with challenges, all students need to be assessed. The goal of early intervention is not to label children who appear to need more time to develop their basic skills. The goal is to assess if young children are acquiring their basic early skills, especially in the areas of language, reading, social/behavioral acuity and numeracy. The old adage that an ounce of prevention is worth a pound of cure certainly holds true in the case of early intervention.
One model that is shown to be highly effective to address the issue of early intervention is Response to Intervention (RtI). RtI is a multileveled approach to identifying students who are lacking in some area of basic skill development and to intervene early and efficiently. There are different facets to RtI, all of which are critical to the success of this model. First, all students beginning in kindergarten are assessed using a curriculum-based measure to ascertain their reading and numeracy skills. Curriculum measures refers to assessing students on the skills that they are expected to master in any given grade. For example with regard to reading, students in first grade need to know the names and sounds of all letters, the sounds of each of the phonemes in a word and the ability to read nonsense words (plob). Assessing students’ ability to read nonsense words is an essential skill, because many students with a learning disability in the area of reading are not decoding but rather are memorizing words by their visual form. This skill, while impressive, will not enable them to become proficient readers. The only way to ensure that students are decoding is to provide them with words that they have never seen before.
With regard to math, kindergarten students need to master the skills of next-number fluency (to know number names and the count sequence), and one-to-one correspondence (number one refers to one object).
Behavioral skills are important as well. In order to gain maximum benefit from the school experience, both academic and social, students must continue to develop skills such as ability to attend for an extended period of time and impulse control. Many schools are now adopting Positive Behavior Intervention Supports (PBIS) as a method of assessing students’ behavioral acumen and intervening early when a problem is detected. The goal is not to turn teachers into alarmists, but to provide intervention early because of its critical impact on averting later more serious issues from arising.
Within the concept of early intervention, prevention is of paramount importance. Good teaching, research-based curricula and adaptive school environments that have clear but fair parameters and parent involvement allow students to develop into healthy young adults who are engaged in the learning process and leave school with a sense of accomplishment and positive academic self-regard.
This model or new paradigm empowers classroom teachers. Teachers, rather than other personnel, become proficient at identifying potential difficulties before they actually begin to impede the child’s functioning. Expert teachers are therefore expert detectives. The first step is to identify that a problem exists; the next phase is to intervene, early and effectively. Without intervention we can liken the process to taking your temperature, determining that you have a fever but doing nothing about it.
Intervention should first occur within the confines of the classroom schedule, but if the student is not showing adequate progress (through assessments referred to as progress monitoring) additional support is provided. Important to note: the physical place where the intervention occurs is not relevant. Rather, the emphasis should be on the effectiveness of the intervention. A plethora of information is available about research-based interventions. The three most useful websites are The Florida Center for Reading Research (FCRR), Best Evidence Encyclopedia (BEE) and What Works Clearinghouse.
Within the RtI model, the collection and analysis of data allow teachers to make sound decisions. No more are the days of “I feel as though the child is making progress.” These gut responses shouldn’t be used to determine something as important as whether a child is achieving critical lifelong skills. Data on student progress is examined by educators to determine if the intervention is effective or if a different course of action should be taken.
Research has repeatedly shown that effective intervention provided early (and the emphasis here is on both effective and early) can have a significant impact on later reading development. There are many effective programs that can be used as long as they develop the fundamental skills. With regard to reading, these skills must involve phonemic awareness (the ability to segment and blend the individual sounds that make up words), phonics (the ability to connect each grapheme or letter to a sound), reading fluency (the ability to read accurately and with little effort), and vocabulary development. Of course, reading comprehension, the ultimate goal of reading, must be explicitly taught to students as well.
There are many assessments that are readily available to measure students’ early skills. Dynamic Indicators of Basic Early Literacy (DIBELS) is available on-line. All materials are free of charge unless you choose to use the system that allows for more sophisticated inputting of data and outputs of student scores. Parts of the DIBELS assessment have been translated into Hebrew and is titled MaDYK. An explanation of the Hebrew assessment tool can be found here. There are other assessment and progress monitoring programs such as STAR and Aimsweb.
In Visible Learning for Teachers: Maximizing Impact on Learning, John Hattie presents 150 strategies and influences on students’ academic growth. The list of strategies is categorized into three groupings: highly effective or what he terms the winners, strategies with average impact and ineffective strategies. Of all of the 150 strategies cited, Response to Intervention is among the top five most effective. This model that entails a multipronged approach to prevention, assessment and early intervention has a significant impact on the academic growth of students.
The most significant change that needs to take place in our school system is to recognize that intervention must be based on effective practice and should not be seen as merely the physical space where students go when they are struggling in school. There are too many Individual Education Programs (IEPs) that state “resource room” under the section titled strategies for intervention. Where students receive remediation is not relevant, but the type of intervention is. Intervention must begin early, must be research-based, and the remediator must have expertise in the area of remediation.
There are educators who believe that children must progress at their own rate and not be pressured to accomplish skills until they are ready to do so. This belief is problematic. A large amount of data exists espousing the benefits of early intervention; when children are not provided with early support, they face an ever growing gap between their abilities and grade level expectations. The goal is to provide children as early as possible with every opportunity to reach grade-level milestones and develop positive feelings toward learning.

Karen Gazith PhD is the academic dean at Talmud Torah | Herzliah, faculty lecturer in the department of education and counselling psychology and adjunct faculty in the department of integrated studies in education at McGill University. She is an educational consultant who has done work in the area of assessment, intervention, understanding diverse learners and creating the brain-friendly classroom in Canada, the US, Australia, England and Israel. 

There is an unprecedented level of attention being given to the value and applicability of assessment tools, particularly in the field of education. Certainly this positive development is in part a result of the vast amounts of data seemingly at our fingertips. Practitioners, target audiences, funders, local organizations and other key stakeholders recognize that there are ways to measure the programs, initiatives, curricula, or any other intervention in question. And while not every situation lends itself to assessment, the Jim Joseph Foundation has a guiding principle that if the results of an assessment will inform that educational opportunity and others, then, yes, assess! 
In too many instances for too many institutions, however, deciding to assess is the end of the conversation. Yet, a second, equally important, issue needs to be addressed: which assessment tool (or tools) will yield the most useful results? Not every assessment is high quality, and certain assessments are more effective than others for specific classroom settings or other educational environments. Educators and education leaders often focus—on improving learning outcomes or improving the learning experience. This same mindset should be applied to assessments, as there are always ways to improve how we measure our educational efforts and interventions. 
At the Jim Joseph Foundation, we are funding the development of an assessment of teen Jewish learning and growth outcomes. This work is part of our Cross Community Jewish Teen Education and Engagement Funder Collaborative, which is a platform for shared learning and collaboration among grant making professionals at Jewish foundations and federations. All involved parties plan to invest in (and in many cases already have) community-based Jewish teen education initiatives designed to achieve the group's shared measures of success (for example, engaging Jewish teens, and achieving sustainability).
The Foundation funds this assessment because, along with our partner communities, we want to glean as many learnings as possible from the Collaborative’s efforts. Which grantmaking strategies are most effective in which communities? What program characteristics lead to better learning and growth outcomes for Jewish teens? These are complex questions that require time and resources to answer. 
Developing a set of common outcomes for the initiative itself was no small feat. But under the leadership of The Jewish Education Project, the Collaborative came to agreement on what outcomes the various local initiatives would strive to achieve (i.e., Jewish teens establish strong friendships, and Jewish teens feel a sense of pride about being Jewish, to name just two). The evaluation team then developed a teen survey to measure initiatives against those outcomes through a rigorous process of expert interviews, teen focus groups and pilot testing to ensure the survey questions are measuring the intended construct. 
The survey was piloted in three communities this summer. Now the evaluation team is analyzing the survey results, seeking input from key stakeholders and experts, and conducting another round of cognitive testing—all in order to revise the survey items to even more effectively measure the impact of Jewish teen initiatives moving forward. Undoubtedly this is a lengthy process. But by “getting it right,” we will improve our assessment ability in this space, benefitting teens and the entire field. 
From the Foundation’s perspective, equipping grantees to assess their programs represents sound use of funder assets and grantee time. We welcome the decision of many grantees to contract with independent evaluation firms to help them develop assessment tools tailored to measure their programs and desired outcomes. A truly valuable resource in these efforts is the Jewish Survey Question Bank (JSQB) (funded in part by the Foundation), which gathers survey questions used across the Jewish education field and categorizes them by topic. This vast collection intends to make it easier and more efficient for schools, organizations and individuals to develop their own surveys to assess their efforts.
As we look to further advance the quality of assessment of Jewish education initiatives, the secular education arena is a good model to reference. There, many longstanding assessment tools exist, designed to be used by a range of education programs. From my past experience in this arena, I am aware of key questions asked before deciding whether to begin an assessment and—if so—which assessment to use. Some useful questions for day schools to keep in mind include:
1) Is the administration of the assessment a burden or relatively easy? For example, some schools have unreliable technology or Internet access, so a web-based assessment tool may be too cumbersome to administer. In other schools, a paper-and-pencil version may be better.

2) Does the timing of the assessment sync with our need for information? For example, some classrooms may benefit from an initial assessment at the beginning of the school year so the results can be used for diagnostic purposes. Other classrooms might benefit more greatly from a mid-course assessment. Either way, both assessments could be informative to the entire school, or even the broader field, and should be leveraged appropriately.

3) Does the assessment measure the learning outcomes we are trying to achieve? Naturally, some assessments are more aligned with the actual curriculum being taught than others. It is well worth the time to review multiple assessment frameworks before selecting the appropriate one. 

4) Are the results easy to understand and act upon? Some assessment reports are so complicated and data heavy that it becomes impossible to wade through or to glean best practices. The best reports offer clear findings and essentially lay out a road map of small tweaks or large-scale changes to improve the education experience being measured.

5) What is the value of having comparable data from previous years? While seeking the best assessment tool is always a worthy endeavor, there are real benefits, too, to comparing current results with past results or to a wider pool of respondents. If a program has been assessed a certain way for years, or even decades, the best decision may be to stick with that framework. 
Whether in Jewish or secular education, assessment is a best practice—and high quality assessment is an even better practice. From Jewish camping initiatives, to teacher training programs and other grants, we at the Jim Joseph Foundation and its more than three dozen major grantees have used assessment to improve existing efforts and to inform new ones. Its value certainly applies to day schools as well.

Stacie Cherner is a program officer at the Jim Joseph Foundation, which seeks to foster compelling, effective Jewish learning experiences for young Jews in the United States. Established in 2006, the Jim Joseph Foundation has awarded more than $350 million in grants to engage, educate, and inspire young Jewish minds to discover the joy of living vibrant Jewish lives. [email protected]

Each year, close to one hundred Jewish day schools in North America run trips to Israel for students during the final months of eighth grade. In community, Conservative and Reform day school sectors, more than 70% of schools run such trips. While most students are expected to pay their way, few trips depart without philanthropic intervention or financial subvention directly from school budgets. With such widespread practice, it is remarkable that the ROI (return on investment) provided by these trips has, until recently, never been examined.
To make the situation more puzzling, in some quarters these trips are actually viewed with great skepticism. Educators often see the programs as little more than a means to entice students to remain enrolled in school until the end of the middle-school. Because the trips invariably occur during the last few weeks of school, others see them as a kind of graduation ritual, equivalent to an elaborate prom. And because the participants are relatively young, there is often doubt about the kind of impact these programs produce, by comparison with longer trips or experiences that serve teens or older populations.
This is where evaluation can make a significant contribution, helping school leaders understand the ways in which these trips do or do not make a difference for the participants and how best to ensure that these trips are a part of rather than apart from a school’s commitment to Israel education and engagement. When done well, program evaluation provides a chance for educators and program providers to look carefully at their programs and their practices, to test assumptions about why things appear the way they do. Most usefully, evaluation is an opportunity to facilitate informed and constructive conversation among practitioners, as well as between practitioners and those who fund their work.
Two years ago, our team at Rosov Consulting had such an opportunity. While working with Jack Wertheimer on behalf of the AVI CHAI Foundation on the Hearts and Minds: Israel in North American Jewish Day School project, we were approached by the Jewish Agency for Israel. The Agency recognized that many of the two thousand 8th grade students who were participating in the AVI CHAI study, and had already completed a student survey, would soon participate in school trips to Israel. They proposed a follow-up study with a sample of students after their return. This study would make it possible to explore a question that until then had not been researched: if and how middle-school students’ self-understanding and their connections to Israel are changed by participating in short-term educational programs in Israel.
What we learned from the study of the participating schools can be useful to all schools that run such programs. Indeed, the study models what learning can be set in motion by evaluation work in general.
The Study 
In total, 13 middle schools participated in the study: six affiliated with RAVSAK; four with the Solomon Schechter network; three schools identified themselves as modern Orthodox. In total, 227 students responded to both pre- and post-trip surveys, equivalent to 86% of all of the students who participated on trips from these schools. The students were surveyed before their departure as part of the larger study of 95 schools, and then again between two and four weeks after their return to the United States. The students were asked many of the same questions on both occasions to see whether their attitudes and understanding had shifted as a result of their time in Israel. The trips in which they participated ran for between 10 and 23 days.
What We Learned 
Taken together, our pre/post research design revealed that short, ten-day to three-week trips to Israel are related to important outcomes. In cognitive terms, the trips crystalized the ways in which young people thought about Israel and the world. They helped certain ideas about Israel fall into place. For example, there was an intensification in the students’ images of Israel as a “home away from home” and as “a place where teenagers have more freedom to do what they want.” More traditional ideas or spiritual images of Israel didn’t seem to shift. The students came to identify more strongly with ideas or attitudes they had not previously considered: for example, that Israel can be “a warm and friendly place.” And there were clearer and more consistent connections between their ideas about, for example, the meaning of Jewish peoplehood and their connection to the State of Israel.
The trips also influenced students’ affective relationships to Israel, especially for those who previously were not inclined to identify with Israel. The scale of the shift observed was moderate, but still surprisingly large given the brief length of time that participants spent in Israel.
The trips had a significant impact on some students’ understanding of contemporary Israel. Most strongly influenced were students’ thoughts about what it is like for people to live in the country, and especially what it’s like to be a teenager in Israel. In our experience, these personal themes are quite different from those, such as Israel’s place in Jewish history or in religious life, that are more heavily emphasized by day schools over the course of many years of Israel education. In this respect what students learned in Israel was quite different from what they learned about Israel in their classrooms. 
Not all students responded in the same way to their Israel experience. Those who were interested in Jewish matters before the trip were more likely to be engaged by the intellectual and historical ideas they encountered during the trip. Those less connected to Jewish life and Jewish concerns before the trip were more likely to return with a greater sense of Israel as a fun place to spend time and with a greater sense of association with those who live there. Some, as many as a quarter of the students, returned to America with a weaker sense of identification with others Jews and with Israelis.
Lastly, it seems that program time devoted to reflection and discussion were most closely correlated with the changes we observed. We should note that the participants themselves were not fully aware of these effects; they did not appreciate the impact of time spent in such discussions, especially when compared to the programs’ more dramatic components. Our pre/post analysis revealed that these elements were critical to the educational process.
From Answers to Questions
For those schools that offer Israel trips to their students, these data provide important information to help school and trip leaders reflect on their effects. These data also help answer questions school leaders have about their curriculum. From our perspective this study is no less valuable for the questions it poses than for the answers it offers. We see this study as a strong example of how evaluation can stimulate profound questions about practice. Here are just a few.
1. If, as we found, these trips provide a different view of Israel from that which is provided inside of schools, what do we make of that? On the one hand, there is cause for concern in a possible disconnect between the content of trips and the educational programming that precedes them over the course of many years. On the other hand, the programs might be playing an important complementary role, compensating for what is generally absent from regular programming in schools. At the very least, are schools even aware of this disconnect?
2. What happens if students continue on to high school, and the Israel about which they learn is once again an abstract or mythologized one that bears little relation to the one they have experienced in Israel themselves? Will the impacts of these short programs quickly dissipate without reinforcement, or will students start to feel a dissonance between the contemporary Israel they encountered and the abstract Israel they learn about in school?
3. We wonder why young people react in such different ways to their time in Israel. Why do some come home feeling more distant from Israel and from Israelis? Is it because of something they experience during their time in the program, or is it owing to a mismatch between their expectations ahead of the trip and their experiences in the country?
Evaluation, we believe, is not only about providing answers. It is also about posing questions and about stimulating a desire to think carefully and clearly. Some might see this study’s value in what it demonstrates about the ROI in middle school Israel trips; that’s something undoubtedly supported by our findings. Here we emphasize something different: how evaluation helps surface questions and insights that promise to intensify and improve the outcomes produced by trips.

Dr. Alex Pomson is the director of research and evaluation at Rosov Consulting. [email protected]

It has become the norm for companies to seek feedback constantly via email or phone surveys after we make a purchase, or to track sentiment (and take corrective actions) about them on social media. Why the obsession? Because companies have figured out that in a competitive marketplace, managing customer sentiment and loyalty are critical to profits, and a channel for honest feedback is essential to good managerial decisions.
There is a growing body of evidence in the nonprofit sector that listening to customer feedback leads to greater impact. Major foundations have initiated programs like the Fund for Shared Insight to encourage nonprofits and themselves to do a better job of listening to the voices of their customers, and creating continuous improvement cycles. 
So, too, for schools. When full-tuition families are paying $20,000-$40,000 a year for their children’s education over many years, often the greatest expense in their lives and in some cases more than their annual mortgage, the school had better be listening to them on an ongoing and systematic basis. 
Parents’ perceptions of a Jewish day school play a critical role in enrollment. In fact, we’ve tested many other factors like tuition level and advertising, and none has the systematic effect of parent perceptions. In synagogue and while shopping, at Starbucks and drop-off lines, birthday parties and bar mitzvahs, parents exchange their impressions of their children’s school and other parents listen. While Measuring Success has collected feedback from nearly 100,000 parents, students and alumni in schools, we sense that the field still has large pockets of resistance to capturing and using feedback.
So where does the resistance come from? Primarily, from board members or administrators who have strongly held anecdotes, intuition and “gut” about the changes in the school that need to be made. These anecdotes tend to be formed by the family’s own experience with their child, or a complaint from a close friend during kiddush. And as we know, it is often the loudest voice or wealthiest funder at the table who gets the grease. But the problem is knowing whether that anecdote holds for the larger population in question. We have tested this by asking school leaders to hypothesize, based on their anecdotal knowledge, the answers to questions like, “Which demographic groups in your school community are the happiest or the least happy?” 80% of those anecdotes are not supported when we examine them against representative feedback data. Which means that potentially 80% of all the time and energy your school spends is on initiatives that are not going to make a difference. So how can we get more bang for the buck in terms of our time and resources? Listening to our customers via the data.
There are many ways to collect feedback data for schools. We have outlined a number of them here in a concentric circle model, and explain when each feedback mechanism should be used, what questions can be answered, and tips for doing it.
Parent Surveys
Capturing the parent body’s objective feedback on annual surveys—or increasingly “pulse surveys” that are shorter and more frequent throughout the year—is essential. It gives a school the best leading indicator for its likelihood to grow or shrink, and a chance to demonstrate customer responsiveness by quickly addressing the issues once they’ve been identified. It’s also a great way to learn what parents are absorbing from the multitude of communications they receive about the school. At Schechter Westchester, we measured parent perception of the school’s quality relative to local private and public schools, and used the insights to direct focus to areas that the data suggested mattered to parents. It was a revelation to see that in some academic areas in which the school objectively was performing well based on test scores, the parents still perceived the school as weak. The school was not communicating effectively and needed to improve its marketing efforts in that area.
Since we have been working over the past five years with over 100 Jewish day schools on parent surveys through a program originally funded by PEJE, we have had the opportunity to see longitudinal improvement in many of the schools that have made these surveys an annual or every-other-year occurrence. For many schools, the benchmarks comparing their scores against other Jewish day schools and the local private and public competition is very useful in year 1, but over time, it’s the longitudinal progress that matters to them most. Schechter Westchester’s improvement over time, such as a nearly 10 percentage point increase in likelihood to strongly recommend the school, enabled the professional leadership and board to see that the changes they had put in place were making a difference in the minds of the customer. 
Naturally, parent sentiment is important not only for retention of current parents, but also to attract new families. Many schools wonder how they can attract families not yet in the school. The best way to do so is not advertising, but rather ensuring that families currently in the school have a high opinion of the school. Positive perception drives word of mouth, which is the most important marketing. Few new parents will spend tens of thousands of dollars without speaking several times to parents in their social networks who have already experienced the school. It’s a critical reference. In fact, Schechter Westchester’s enrollment ended up exceeding budgeted projections by 35 students due to lateral entry increasing substantially and attrition plummeting 43%.
Student Surveys
For schools with high schools, the customer and buying decision includes not only the parents but the high school students as well. As with parent surveys, it is very important that student surveys be objective and protect anonymity, since students even more than parents fear that honest critical replies will be used against them. Administrators are concerned that students may “collude” with their friends on responses in order to negatively bias the results. The Yeshivah of Flatbush solicited student input to make improvements to electives, scheduling, Judaic studies, Hebrew and school culture. It experimented with various structures until arriving at having the students all take the survey at the same time during homeroom time, under the supervision of teachers. That took care of collusion. But in order to ensure students felt they could be honest, Flatbush made it clear that only Measuring Success as the objective third party would be analyzing the individual responses, and the Flatbush administration would only see the data in the aggregate in a manner that could not be traced to any individual student.
Feedback from families who left the school or those that applied but did not attend
What we have found most effective are qualitative interviews of as many of those families as are willing to speak with the school. The reason is twofold: there are usually only a dozen or two of these families (depending on the size of the school) each year, making it difficult to interpret quantitative results for lack of sufficient sample size, and these families are also less inclined to answer surveys because they opted out. Another important time to use qualitative feedback is to generate hypotheses to test in a quantitative survey, or to use focus groups to help interpret confusing survey results. The challenge with qualitative feedback is to ensure that it is representative of the populations in question. 
Alumni Surveys
These tend to serve three different purposes: demonstrating the school’s impact on alumni to prospective families (marketing), feedback to the administration on how to improve impact, and fundraising. Alumni have a perspective different from current students and parents in that they can assess how effectively the school prepared them—academically, ethically, socio-emotionally and Jewishly—for high school, college and career relative to their peers. As living proof of your school’s impact, alumni are a critical source of feedback as to whether your school has added value. For example, a large independent school in Nashville is capturing alumni feedback via surveys for each of the next five years to generate marketing content to prospective families.
But while young alumni are critical for measuring impact, older alumni become critical sources of fundraising, as they have sufficient earning power to make significant contributions. Many schools make the mistake of waiting until an alum is over 40 and has landed in the news for business success to start cultivating a relationship. By then, it is often too late, as the alum has loyalties to several other alma maters like their college and graduate school, as well as other charitable causes which have actively cultivated a relationship with them. But it’s never too late to catch up.
Charles E. Smith Jewish Day School ran an alumni survey this year in anticipation of the school’s 50th anniversary. Asking alumni for their feedback is often the first step in rebuilding trust and a relationship. The survey results were also uploaded into the school’s database for improved knowledge of their alumni. The survey data can be analyzed to predict which alumni have the best likelihood to make a significant gift. This approach has helped federations to identify which midlevel donors have the greatest propensity to become major donors of the future. Alumni offices can receive ranked lists to use their limited time and energy on those alumni with the greatest likelihood of success. Universities also use these predictive models frequently.
Community Surveys
A survey of this final concentric circle is a tool in increasing demand because it identifies the potential size of the marketplace interested in Jewish day schools, as well as the individual prospective families that would be the best fit for the school. Charles E. Smith, for instance, sent a survey seeking to understand the educational perceptions of Jewish parents to the databases of 27 community organizations, including synagogues, JCCs, PJ Library and other early childhood programs. We were able to identify groups of parents most likely to attend the school, as well as identify a list of prospective families that would be the best fit that the admission office could use to cultivate.
While algorithms and analyses undergird much of the work identified above, the most important lessons we have learned are about trust and intention. Feedback is a powerful trust-building tool that tells your stakeholders that you are listening (especially when you take action on the feedback). Transparency about survey results, especially in areas where customers are critical of the school’s performance, also builds trust with your customers because you are acknowledging their feedback. The other lesson about intention, or kavannah, speaks to the importance of focus and accountability. Schools that are committed to listening to stakeholder feedback, setting a measurable goal, and acting on it consistently see statistically significant improvements in those areas they focused on. 
One common question that comes up is how often should a given stakeholder group be surveyed for feedback? Some schools only run surveys as a requirement in their accreditation process every seven years, clearly too infrequently to be used in a regular feedback loop. Others would argue that parent surveys run every year or every other year are still too infrequent to capture feedback quickly enough to make a difference. On the other hand, others worry that if we survey too often it will result in survey fatigue for the respondents, dropping the response rates and causing stress for the school that hasn’t had the time to implement changes as a result of the last set of feedback data. 
At one extreme is the argument that feedback must be real time and immediate. When I tried to lose weight for my reunion, for the first week I got on the scale each day, but my weight didn’t change at all despite my hoping it would! I had to track my calories during the day as I was consuming food. This monitoring of my behavior on a real-time basis was the true leading indicator, which quickly allowed me to modulate my food intake and lose the weight.
What would it mean to track data close to real time in schools?
I spent a few days this past summer with the leadership of Gann Academy in Waltham, Massachusetts, helping to think through how to track and measure the student experience. We discussed approaches to collecting data daily, such as asking students at the end of each day to complete a two-question survey on their mobile phone about how much the school had impacted or inspired them that day, to capture the sentiment at the moment and be able to chart the trends, along with a more comprehensive set of feedback every month. We also imagined asking faculty to notate on a phone app the mood in each interaction with each student. (Given that we estimated 20-25 unique faculty members having substantive interactions with a student each day, this could be quite an undertaking). Such a frequency of feedback no doubt would present a serious cultural change; could schools do this in a way that stays true to their educational mission? The quality of human interaction is so essential to outstanding teaching and learning; how might this activity be an enhancement rather than a distraction? 
Perhaps this frequency of feedback is too much for your school, but make no mistake that this is part of a larger trend. Many foundations that we have worked with are investing in public schools tracking students’ “individual learning journeys,” where the school’s job is to ensure that each student maximizes his or her potential (and thus maximizes the value-add of the school). For the past year, we’ve been working with Southern Methodist University on individualized student journeys within West Dallas public schools. The tracking system links together islands of data from schools along with data from community programs in which students and their families participate, like afterschool, summer and social service programs. The combined datasets provides an eagle-eye view of a student’s academic as well as socio-emotional trajectory, helping educators and counselors perform more effective interventions when students are not achieving their potential.
Critics might say that this is what the smaller, more intimate environments of independent schools offer innately—individualized attention from faculty who provide academic and emotional support. Increasingly, though, schools are finding that this is occurring haphazardly, or that the 20% of kids with the greatest needs are getting 80% of the faculty’s attention. 
Database Marketing through Targeted Communications
We can apply the same logic to our communications with our various stakeholders. Rather than sending out one-size-fits-all messages to our parents, students, alumni and donors, what if our schools automatically integrated a customer’s preferences, interests, prior feedback and social media activity into creating a customized set of messages aligned with what that customer’s interests? For example, Procter & Gamble takes daily data feeds from 86 brands covering 60 million customers and applies the information to customize the communications messages that each customer receives.
And that’s the takeaway message about feedback. No matter at what level of intensity, or with which stakeholders your school chooses to get feedback, the key is how your school uses the feedback to make improvements that enable each family to feel like their child is maximizing his or her potential at your school. And where you are failing to do so, the feedback lets you know so your school is empowered to fix it. Feedback loops, whether performed once every few years or every day, are critical tools for continuous improvement and allowing our schools to put the “custom” back in “customer.”

Sacha Litman is the managing director at Measuring Success, whose mission is to enable schools to harness the power of data analytics to increase enrollment, impact, and fundraising. Sacha and his team have worked with over 330 Jewish day schools and 400 independent schools, along with several dozen public and charter schools. [email protected]

How to Measure A+ Human Being Education

As the newly renamed de Toledo (formerly New Community Jewish) High School embarked in 2014-15 on its bar/t mitzvah year, the Board of Directors inquired whether we are actualizing the hopes and dreams that our mission statement holds forth, and to what extent we know whether we are concretely attaining our school vision of cultivating the A+ human being. We were charged with the task of ascertaining how best to quantify our school culture and our added value as a Jewish community school, as well as to chart the future of both. This article will describe the process we undertook.

Mission Statement

To begin, we reviewed our school’s mission.

The Mission of de Toledo High School is to raise up a new generation of Jewish leaders for whom Jewish values and tradition shape and guide their vision, and for whom knowledge creates possibilities for moral action, good character, and shalom.

In order to better reflect upon our culture and to assess our efforts in carrying out this broad mission, we divided the mission statement into three components:

1: Raise up a new generation of Jewish leaders

2: For whom Jewish values and tradition shape and guide their vision

3: For whom knowledge creates possibilities for moral action, good character and shalom

The entire de Toledo community agrees that this mission statement holds out a succinct, beautiful guiding vision which we all look to as a source of grounding for our professional work. Purposefully, it is not overly specific so that we all have the opportunity to actualize it in exciting, dynamic and creative ways. Nonetheless, it allows for multiple understandings, rendering it difficult to measure. As we understand it, each of these components raises opportunities for assessment.

The first component raises the question of what, precisely, is a Jewish leader, how do we know what the “new generation” will need to confront, and, therefore, how do we properly raise such a generation and know that we are doing it well. The second component raises different opportunities. What are the Jewish values that we want to inculcate in our students, and which Jewish traditions and from which Jewish communities and cultures is not obvious. Even if a consensus on these matters were achieved, we would still need to ask how do we actually “shape and guide” each student’s vision. The third component’s opportunities extend far beyond the confines of our school community, since our students will be sent into a larger, diverse, multi-ethnic, multicultural world that has multiple spiritual possibilities and expressions of faith. Therefore, this component calls for extended conversation regarding the kinds of knowledge our students will need in order to act morally, evince good character and to achieve inner and outer shalom.

This process of stripping a document into essential components so that we could see what challenges we might have in finding an appropriate assessment mechanism was duplicated by looking at our school’s essential values and Expected Schoolwide Learning Results (known as our ESLRs). So while the mission statement and our essential values offer a guiding vision to our community, they present numerous challenges when it comes to assessing to what degree we are effectuating that vision. This conundrum holds true for Jewish education generally—and, indeed, for all values-based institutions.

Eight Dimensions of Human Culture and Religion

Ninian Smart, a key figure in religious studies, proposed that religious cultures evince seven dimensions. He argues that one advantage of using his scheme allows us to avoid presenting a too narrow perspective of a religion (or culture). Using these dimensions thus gives a more balanced presentation of a religious culture. We chose Smart’s scheme to serve as a tool to help us do a cheshbon nefesh kehali, a communal self-reflection and assessment of our school culture. We adapted his proposal, rethinking and renaming some of the dimensions, and adding one more. The Eight Dimensions as we use them include:

1. Doctrinal/Philosophical Dimension: the beliefs, doctrines, and tenets that our school culture promotes.
2. Ethical/Legal Dimension: the ethical values of our school culture, and how our procedures and regulations try to inculcate the opportunity for all of us to live them out.
3. Mythic/Narrative Dimension: the key (“sacred”) stories that inspire and motivate everyone in our school culture, and to which we hope everyone feels connected.
4. Practical/Ritual Dimension: the practices, rituals, and customs of our school culture.
5. Experiential/Emotional: the experiences we provide for the members of our school culture so that we can together share the full range of human emotions.
6. Material/Symbolic Dimension: the artifacts and symbols of our school culture.
7. Visionary/Idealistic Dimension: the goals, hopes, and dreams that our school holds out for our students.
8. Social/Organizational/Institutional Dimension: all the organizational and institutional structures that help promote the social cohesiveness of our school community and which help us to live out our beliefs [#1 above] and our hopes and dreams [#7 above].

Just as Smart contended that to understand a religious culture in its totality one needs to separate its various components, we realized that we, too, need to separate our school culture into eight workable components. The division of our school culture into such components is, of course, impossible in real time; we did so solely for the purpose of assessment and reflection. After all, the mission statement suggests that the success of our school cannot be measured merely by assessing integration and synthesis of course content. To assess our students’ readiness to enter the world after school grounded in Jewish ethical values that would lead them to transform the world around them required us to think much more broadly. We felt that these Eight Dimensions could serve us as a good tool for clarifying the vision, values, goals and broad-based concepts that are central to the school culture, especially as we apply them to the mission statement.

The model of the Eight Dimensions has given us valuable tools: it allows reflection on the key questions we raised above concerning the mission statement; it creates a common language and understanding, and it affords the community precise evidence to see exactly where we are in terms of actualizing the mission.

The social, organizational, and institutional dimension, for example, helps us address the first challenge we raised about our mission statement: What, precisely, is a Jewish leader? Our students have established over 20 different student clubs with a vision of their own and a will to pursue actions (such as fundraise, do volunteer management, and oversee the planning through execution of schoolwide presentations) that make a difference on our campus and beyond. Tenth and eleventh grade students who are members of our T’fillah Kehillah Institute design, coordinate and execute experiential and spiritual tefillot for the student body and the larger community. Other students belong to our cohort of student ambassadors, and together with our school staff help facilitate admission and advancement programs.

The experiential and emotional dimension may help us to address the second challenge we raised about our mission statement: How do we actually “shape and guide” each student’s vision? Our Jewish Life programs create an environment in which students experience firsthand their obligation to participate in and strengthen all aspects of community life. A testament to such communal environment is our annual four day all-school shabbaton, which is held 30 miles away at a beautiful camp retreat center. The all-school retreat allows very different kinds of interactions among our students, our office staff, and our faculty and their families. Shaping and guiding our students’ vision in search of the deeper meaning in life happens through personal relationships both in the classroom and beyond.

The practical and ritual dimension may help us to address the third question we raised about our mission statement: What are the kinds of knowledge our students will need in order to act morally, evince good character and to achieve inner and outer shalom? One type of such knowledge our students acquire is through the engagement in acts of tikkun olam as part our social service program. Another example of a ritual that promotes this aspect of the mission statement is the Drishat Shalom project. Each of our students is given the opportunity to reflect on their four-year journey as they learn and search for a deeper meaning of a specific biblical text chosen especially for them. Throughout the four years, our students explore their personal connection to their text and as seniors they get to share with the community a special message that celebrates their sense of hope, joy and passion for life based on what they identify as truly important.

To use this tool effectively, we had, however, not only to teach the Eight Dimensions schema and find a way to use it for assessment, we also had to ensure that the entire faculty understood the mission statement in its various components in a similar way. We now will clarify the process we took as a school as we introduced and used the Eight Dimensions measurement.

Experience, Response and Results

Smart’s scheme and its applicability to our school culture was introduced to the faculty for the first time in May 2014. Over the summer recess, the faculty (including non-teaching faculty) read an excerpt from Dr. Smart’s work and used the model to analyze and assess a unit they teach, an activity they help facilitate or a function they regularly engage in as part of their job or profile at the school.

Thanks to this preparation, we were able to start our faculty orientation, prior to the beginning of the 2014-15 school year, with a professional development program over two days using the Eight Dimensions tool. The first day was dedicated to the introduction of the schema through a presentation of each dimension accompanied by concrete examples. On the second day, the faculty broke into small workshop groups, both within each department and in mixed groups. Each group used the model to discuss the school’s mission statement, the challenges it raises and the evidence of fulfillment in our work. The departmental groups gained the realization that they consistently gravitated toward focus on some dimensions at the expense of others that may be worth attention. The mixed faculty groups demonstrated a holistic representation of the actualized mission statement through the entire range of student experience, and enabled the realization of how each faculty member and their department made (or could make) their contribution.

This process generated a conversation not only about school success and alignment, but also about some discrepancies between school culture and school policy. As a result, the faculty themselves identified the need to assess and evaluate to what extent our curriculum, pedagogy and culture align with our mission, vision and goals. At this point, a wheel template of the Eight Dimensions measurement was introduced and used in both department level and as part of a faculty meeting. Each department came up with examples of departmental success—a program, a curricular unit, a ritual, or any other aspect they felt was done successfully—and used the template to identify which dimensions and mission statement components (and expected schoolwide learning results) are being actualized. The faculty was encouraged to continue to use the measurements when designing their lessons, creating programs and collaborating with each other.

The use of the Eight Dimensions as a tool for communal reflection and assessment led to results at all levels and scales. Teachers could now assess their own teaching in a more holistic way. Departments could review their curricula from a broader, more mission-driven perspective, and we gained perspective on our campus culture as a whole to ensure that we are aiming for the broad-based growth of our students, most especially in the area of values acquisition.

Hope for the future

This process of qualitative reflection and evaluation has given all our stakeholders a common language to discuss our campus culture and shared vision. More importantly, each of us better understands how deeply our mission statement, learning outcomes and the holistic approach towards education are interconnected. The deepened awareness enables each educator not only to self-assess, but also to design curricula and programming, both inside and outside of the classroom, which aligns with our shared vision. On a global level, this process led to a reconsideration and change of our graduation requirements so that the values and mission of our school can be better realized. Further work could be done surveying our alumni to ascertain to what degree the mission statement is actualized in their work and lives. However, our attempt to set up a practical internal system of reflection and evaluation of our mission has resulted in the recognition of the evaluation process as a core value of our community.

Practical exercise for the reader

Did you find our methodology valuable? If so, we encourage you to use the wheel below by placing your school’s mission, value or a curricular component at the center and complete it with the help of provided questions. Next, we suggest discussing and analyzing the assessment tool with a colleague. Good luck!

Yonatan Rosner and Dr. J. B. Sacks are co-directors of Jewish life at the deToledo High School in West Hills, California. [email protected], [email protected]

Many day schools recruit families and students from Jewishly diverse backgrounds and market that diversity as a competitive advantage. Diversity signals a happy family in which everyone belongs. In truth, that diversity is often arrived at pragmatically, as a necessary choice to fill the seats. I don’t mean that it’s chosen grudgingly or masks conflict; the principle that the Jewish people are one is deeply and broadly felt. But how diversity plays out in the life of the school is different when it’s arrived at pragmatically than when it’s chosen ideologically. In the former case, the fact of diversity requires accommodations to make space for everyone in the school. Typically, tefillah, kashrut, dress code, and the content and approach in Judaic studies classes are the primary contexts in which diversity is negotiated. One common version of a pragmatically diverse school is one sponsored by the Orthodox community in which Orthodox observance and outlook are the baseline in one or more respects of school life and program, while non-Orthodox practice and outlook are purposefully accommodated in a range of ways. 
In intentionally diverse, or pluralistic schools, diversity is the point: it is the organizing idea for the school. Pluralism in this context reflects and affirms the legitimacy of multiple expressions of Jewishness and Judaism. In a pluralistic school, the founding ideas may be something like this: No one expression of Judaism has a monopoly on truth or legitimacy. Taken together, the many strands of Jewish expression weave the tapestry of Judaism. This is to be celebrated. It’s important that our children learn be exposed to this diversity of Jewish expression, learn about different approaches and how to respect the people following them, and through that exposure refine and strengthen their own expression. Such schools form in the spirit of, Let’s see how broad a swath of the Jewish people we can bring together in a school to learn and form community together. Our pluralistic understanding of Judaism echoes our celebration of diversity in society in general.
It makes sense for an intentionally pluralistic school to make the celebration of diversity a central value and to establish and maintain thoughtful structures and practices for negotiating difference. Easier said than done. Jewish day schools, after all, teach toward commitment. Commitment is (typically) narrowing; pluralism is (typically) broadening. And there the challenge begins.
In 2010, when I served as rav beit hasefer and assistant head of school at the pluralistic JCDS: Boston’s Jewish Community Day School, I was tasked with implementing a board mandate to conduct what we called a pluralism audit. This was to be a thorough assessment of the state of pluralism in the school, which had pluralism as a pillar of its culture since its founding. This mandate was among the prescriptions in a strategic plan prepared the prior year, indicating the school’s recognition that if pluralism is essential to the school’s identity, it requires planning and assessment no less than the school’s formal curriculum, enrollment goals, or brick and mortar infrastructure.
Pluralism ought to permeate the operation and experience of the school. An audit, then, should seek to comprehensively examine the school’s culture of pluralism wherever possible: in formal curriculum documents, pictures and materials on the walls, printed materials like the application packet, development materials like seasonal letters to prospective donors, and the school website, of course. An audit should include live observation of classes, both secular and Judaic, to better understand the culture of inquiry in the school. And stakeholders with lived experience of the school should be surveyed, including, for example, students, faculty, alumni and parents, and parents of alumni. 
Significantly, a pluralism audit (unlike the annual audits in a CFO’s office) will not grade the school for compliance. Compliance is relevant when there is a single, clear standard against which practice is measured. Pluralism, though, is a culture, an approach to living and learning together. Its parameters are variable and contextual. Thus, the first thing to measure through an audit is whether there is even a shared understanding of what pluralism means in the school community.
At its most basic, a culture of pluralism clusters around a welcoming and caring spirit of live and let live. A comfort with and celebration of diversity. A recognition that everyone in the community has a respected place and the right to be him- or herself. This welcoming spirit, when applied in the context of a Jewish school, embraces the range of Jewish identity, expression and observance among school families and their children.
A second understanding is pluralism as a way of living and learning together in Jewish community. In this aspect, pluralism consciously mediates among constituents’ differences. The goal of pluralism, according to this understanding, is to sustain an intentionally diverse community that will learn and grow together. Such a community thrives best through students’ (and their families’) encounters with those different from themselves. Encountering difference increases one’s self-understanding at least as much as it does one’s understanding of the other. Pluralism cultivates self-confidence, a non-defensive commitment to one’s own position even as one is brought into regular contact with people who may not share one’s own commitments but are passionate about their own, alternative commitments. 
Critically, given the presumption of respect granted through the first understanding of pluralism (i.e., the ethos that I’m ok and so are you), the second understanding calls on community members to navigate the tensions between preserving the integrity of their own values and preserving the integrity of the other’s. This is the sensitive point where “mere” tolerance crosses over into pluralistic accommodation: one might be called on to compromise one’s own priorities to make room for the other and to actively seek out those compromises out of a commitment to making room. At a minimum, in a pluralistic community one must grant full legitimacy to the person holding a position with which one fundamentally disagrees and actively find ways to assure that he is free to express his position, within the bounds that allow the community to hold together. 
The compromises required to establish and sustain pluralistic community in a school highlight the inherent tensions between community and individual autonomy, value-based positions and “mere” preferences, competing cultural priorities, and sources of authority. They also highlight the shared values around which the school has coalesced and which make all the other compromises worthwhile. The discussions and negotiations around pluralism may be heady and they may be tense, but they are rarely boring. They can contribute powerfully to a culture of meaning in a school. 
Like any core value a school may hold, pluralism is hard to practice. It will be set against other competing values (like unity) and constrained by pragmatic realities (like available teaching hours and faculty competencies). Pluralism in particular is hard to realize both because it is not a single quantifiable thing (it is a culture) and because it can always be cultivated in new, more sophisticated and subtle ways throughout the life of the school. Nonetheless, or perhaps even because pluralism is an imprecise target, a mechanism like a pluralism audit offers a school the opportunity to examine a core value—a pillar of its mission—deeply. 
Pursuing an exercise like the audit can feel indulgent because it is time-intensive for staff and expends volunteer capital of parents and other lay leaders. The effort is worth it because by investigating a school through a single lens, an audit can turn up valuable data that would otherwise pass unnoticed. By examining a core value in a sophisticated way, and with the involvement of the entire school community, a school can educate its stakeholders about its values. If conducted successfully, an audit will facilitate stakeholders’ reengagement with the school’s mission and motivation to achieve as-yet unrealized potential. Lay and professional leadership may emerge with a clear mandate from parents and others about how to prioritize new and renewed efforts in cultivating and sustaining a pluralistic culture. My experience with the pluralism audit clarified for me and my school, for example, the importance of assuring that the voices of all members of the community be heard and that the pluralistic face of our tefillah program required consistent tending. 
Most importantly, the pluralism audit highlighted the importance of educating around a particular core value in our school. The fact that this value was part of the school’s vocabulary – in essence, was what the school believed and asserted about itself – did not assure that the value was understood by all community members. This is not surprising. But the exercise of conducting the audit and the documentation it generated provided great material for prioritizing, for teaching, and for programming. And this is the generalizable point of the audit I conducted of a particular value at a particular school: the implementation of a school’s core values merit periodic close examination. An audit of any value’s lived expression in a school is an exciting and motivating exercise in living the mission. And living their mission is what good schools do.

Rabbi Joel Alter is the director of admissions for the rabbinical and cantorial schools at the Jewish Theological Seminary. [email protected]

When my daughter Dina returned from the first class in managerial accounting early in her MBA program, I innocently asked how it had gone. I fully expected her to describe her boredom with the rigors of accounting, since pursuing an MBA was decidedly an afterthought for my iconoclastic daughter, who already held degrees in theatre and social work.

Imagine my surprise when Dina responded that accounting was unexpectedly interesting because, she now realized, it should be understood as a form of narrative, a kind of drama. Within the ethical and technical rules of the field, the task of the accountant is to figure out which of the stories of the company should be told through the medium of its “books.” Accounting is basically about creating the plot, characters, and setting of the story. As the instructor explained to the class, “Your task is to render an account: to tell the facts of the case, the story of the condition of a company in an accurate and yet ultimately persuasive way.”

In the world of business, an account is a story told in quantitative form. It publicly documents all the income and investments that enter the company and all the products and liabilities that emerge from it, all its assets and debits, all its profits and losses. When the books balance, the account is closed: the story has been told.

When I draw our attention, as Dina did mine, to the ghosts of narrative and storytelling that stand behind the counting, measuring, and computations that lie at the heart of modern assessment in the service of educational accountability, I do not aim to undermine the credibility of assessment. I am not referring to “mere storytelling” as if narrative is a lesser form of discourse. The connections between counting and recounting are built into the etymology of these words in many languages. Thus, in German, to count is zählen and to tell (a story) is erzählen. In Hebrew, a language with utterly different roots than English or German, the verb for counting is lispor, while the word for telling is lesaper.

I believe the lesson is clear. How and what we choose to count and the manner in which we array and display our accounts is a form of narrative—legitimately, necessarily, and inevitably.

Tools for Counting and Recounting

When my teacher Benjamin Bloom led a group of university examiners in the development of the Taxonomy of Educational Objectives in the late 1940s and early 1950s, their goal was to provide a structure within which evaluators and teachers could determine which story they wished to tell about the learning of their institution’s students. They had determined that most of the instruments then in use to assess students—and thus to render them, their teachers, and their colleges accountable—were exclusively stories of the acquisition and retention of knowledge, of the students’ success in recalling facts, events, principles, and concepts they had learned in class or read in their textbooks. Bloom and his colleagues argued that this was an impoverished story, one that missed the most important aspects of the account the examiners needed to give of students’ learning.

By elaborating the cognitive outcomes of education into a taxonomy comprised of six categories—ranging from knowledge and comprehension through application, analysis, synthesis, and evaluation—Bloom and his colleagues developed a much richer array of plots and themes for the story of academic performance. A program that appeared to be achieving great success when knowledge alone was measured might look much less impressive if the “higher-order” processes were accounted for. Bloom and his associates also were committed to extending the story from the cognitive to the affective domains in order to include the development of emotions, motivations, passions, and identity.

The story told by an assessment is thus ultimately a function of the dimensions of measurement that determine the possible directions the narrative might take. So accountability requires that we take responsibility for the story we commit ourselves to telling. We must make public the rationale for choosing that story as opposed to alternative narratives. This requires that we first deliberate with our colleagues and stakeholders about the goals we set, the missions of our schools, and the elaborated conceptions of our purposes.

Only then should we defend the adequacy of the forms of measurement and documentation we employ to warrant the narratives we offer. In the case of educational accountability, we are limited in our recountings by the instruments we use to count. As my colleague Lloyd Bond regularly reminds me, “Since we can’t normally measure everything that counts, we had better remember that what will count is what we choose to measure.” Taxonomies and indicators are critical aspects of how and with what coherence and credibility these stories can be told.

Seven Pillars of Assessment for Accountability

Most of the principles I want to offer here are familiar, even venerable. The fact that they remain pertinent suggests how persistent many of the challenges of assessment remain.

1. Become explicit about the story you need to tell and the rationale for choosing it. An account is one story among the many that could be told about the quality and character of an educational experience. No instrument can claim validity, no account can earn a warrant, without a clear explanation of why this story is being told instead of others. Indeed, it should be clear what the major alternative accounts could be and why they were rejected. Any one form of assessment, however rich, is a compromise, a choice among a set of legitimate possibilities.

2. Do not think that there is a “bottom line.” An early step in the deployment of any instrument, new or old, should be a process of locating the instrument in a larger conceptual framework that explicitly stipulates what it does measure and what it does not. Since there is no real bottom line, the first obligation of the person rendering an account is to take responsibility for locating its unavoidable insufficiencies.

Moreover, judgments of validity are never a property of measuring instruments per se. Validity can only be judged when we examine assessment results in the context of a particular argument or narrative. The cardinal principle of accountability is that counting is only meaningful and useful in the context of valid recounting. Indeed, we might make a distinction between measurement and assessment in this regard, with assessment referring to the manner in which one arrays, displays, and interprets particular measurements in the service of judgments, decisions, and actions.

3. Design multiple measures. It is dangerous to permit highly consequential decisions of policy and practice to rest on the results of a single instrument, however carefully it has been field-tested and ostensibly validated.

4. Work on combining multiple measures. A set of instruments, each with its own scores, indices, and observations, will deliver on its promise only if we take on the hard task of developing rules for deciding how to display, organize, and aggregate those indicators for making decisions. Inevitably, those decisions are functions of human judgment—which is, after all, an essential element in any such process, not something to be feared or avoided.

5. Remember that high stakes corrupt. High stakes attached to assessments have a tendency to distort the educational and evaluation processes they were intended to support. This is not only because teachers and students are sorely tempted to cheat when the stakes are high. It is also because when test designers know that high stakes are involved, they have a tendency to use items less likely to be uncertain or subject to competing judgments and arguments. As the instruments are weeded of such items or sections, they gain reliability and objectivity but often at the sacrifice of validity and nuance.

The most significant feature of high-stakes assessment is this: The higher the stakes, the greater the likelihood that teachers will teach to the test. These assessments must be designed so that the tests are worth teaching to. This is not a trivial challenge. It cries out for a strategy of embeddedness.

6. Embed assessment into ongoing instruction. Assess early and assess often. In my early days in Chicago, we used to joke, “Vote early and vote often.” High-stakes assessments are likely to be used very late in the course or program where they are employed in the service of accountability. But the later the assessment, the later the knowledge of results, and the less likely it is that the assessments will yield information that can guide instruction and learning. I call these “high-stakes/low-yield” forms of assessment. They may satisfy accountability mavens but have little educative value. Instead, we should develop low-stakes/high-yield forms of assessment, much like the “running records” used by K-12 reading teachers or the routine medical history, physical examinations, or lab tests that physicians and nurses administer.

Embedded measures will necessarily be designed with a different “grain size” from those designed exclusively for external, high-stakes assessments. They will be more particular than general; more dedicated to measuring individual student progress than institutional success; repeatedly administered, with quick turnaround, rather than being single end-of-course events. This is assessment as a regular physical exam rather than as a public autopsy.

This aspect of assessment emphasizes the need for bilateral transparency. That is, the progress students are making needs to be as accessible to them as it is to teachers. Such transparency can empower students to take greater control of their own destinies. It is, after all, ultimately the student who must own her or his understanding and progress. Systems of assessment that are opaque, secretive, and slow-responding cripple students’ sense of responsibility.

7. Become an active and collaborative site for research on new forms of assessment, new technologies to support such work, and better strategies for integration of such approaches with instruction. If the use of single-instrument, high-stakes/low-yield assessment tools will undermine the most important goals and purposes of education, then those of us who design and deploy assessments have a professional and ethical responsibility to design them to contribute more positively to the quality of teaching and learning for all students. The need now is for new assessment research and development, a project that can succeed only if institutions collaborate, experiment, and open their windows so that national work can move our fields ahead.

Taking Control of the Narrative

One of the reasons Dina was so taken with the metaphor of narrative in accounting was that the careers she had pursued before her MBA program were as an actor and as a psychotherapist. During her graduate study in social work, she had been drawn to “narrative therapy” as an approach to counseling. In narrative therapy, the central idea is that each one of us is living the life of a character in a play or a novel. Some of us feel that we have a great deal of influence over the plot of the play, while others, alas, feel that they are characters in someone else’s drama. The goal of the psychotherapy is to support one’s clients in seeing the narratives they feel they are living but have no control over, and to develop strategies for becoming the authors of their own stories, able to act responsibly in the situation and exercise real agency over their lives.

In this spirit, our responsibility is to take control of the narrative. We educators must take advantage of the deep connections between counting and recounting to define the characters, the plots, the foreground, and the background for new programs of accountability that measure the efficacy of our educational initiatives. We must summon the creative energy and ambition to take advantage of the momentum (and resources) unleashed by new policies and programs. We must exploit them to initiate the long-overdue progress in assessment needed to improve the quality of learning in education.

We are obligated to recount the narratives of most interest to our key stakeholders, but we cannot be limited to those alone. We must display the evidence of teaching and learning (and their embarrassments) through the multiple legitimate narratives we create about our work and our students’ fates. We must account for higher-order understanding and critical thinking, in addition to factual knowledge and simple skills. We must tell of the development of civic responsibility, moral courage, and identity formation even when our stakeholders have not thought to ask for those to be accounted in “the books.”

Moreover, we must make the process through which we render the accounts transparent to our stakeholders. The most important of these stakeholders are our students, who need to feel a sense of agency and responsibility in this relationship as well.

The current quest for accountability creates a precious opportunity for educators to tell the full range of stories about learning and teaching. Counting and recounting can only be pursued together. Counting without narrative is meaningless. Narrative without counting is suspect. We now have an opportunity to employ the many indicators of learning and formation that we can count in the service of the most important stories we have to tell.

Adapted from the January-February 2007 issue of Change: The Magazine of Higher Learning.

Lee S. Shulman is president emeritus of The Carnegie Foundation for the Advancement of Teaching, having served as its 8^th president (1997-2008). He is the Charles E. Ducommun Professor of Education Emeritus at Stanford University. He serves as chair of the advisory board of the Consortium for Applied Studies in Jewish Education (CASJE) www.casje.org. CASJE is dedicated to improving the quality and extensiveness of research that will be useful in understanding, guiding and evaluating the work of Jewish Education in the broadest sense. [email protected]

Dina Shulman is a qualitative market researcher in Los Angeles who engages in counting and recounting professionally. 

As a school, we have set guidelines for the hiring of teachers and other pedagogic staff, and the hiring process always includes a model lesson or some kind of interaction with students. Similarly, the evaluation process for teachers is clear, with expectations delineated in full. However, this is not the case for the non-educational staff, despite the importance of the role they play. What should we expect from those who represent the “business” of the school? How involved should they be as part of our mission?

The fact that this subject comes up so rarely is a prime indicator of how little attention many of us pay to the role of the front office and back office staffs. But they are often the first ones with whom prospective parents, community leaders and other visitors come into contact, and they represent the face of the school constantly. They often are singularly responsible for the sensitive financial areas of tuition and HR concerns. Recognizing that they are important members of the school community, what standards should be in place when hiring staff, how can we better integrate them into the school community, and what should our expectations be?
Too often, we fail to educate our non-pedagogic staff about the mission, the mores and the values of the school. Part of the pre-engagement interview should include a clear explanation of what a Jewish community day school is and who the primary stakeholders are. Employees should understand the school’s dress code, its kashrut policy (and if there are any restrictions on what they may eat or their use of common appliances such as the microwave) and the Jewish calendar. To develop a sense of belonging and commitment, they must have a clear understanding of the milieu in which they work. This is true for all incoming staff members, Jew or non-Jew, religious or secular. 
No one would dispute that being skilled in the tasks required of the job, demonstrating efficiency, and being accurate are essential. But in the school setting (perhaps in most settings) this is not enough. The way in which office staff interacts with others sets the tone for your school. They are often the first responders to distraught parents, unhappy children and distressed teachers. Are they respectful? helpful? compassionate? Or do many people dread having to go into the school office for any reason? Have you, as school leader, set the strong expectation that those who work in the office are to be treated courteously and fairly at all times? Have you ensured that no one plays favorites with teachers, students or parents? Can you rely on the staff to keep confidential all of the information to which they are privy? The head of school is in a unique position to make certain that this tone of menschlichkeit pervades the office and must take appropriate action if this is not the case.
In few areas of school life is the need for teamwork and collaboration more evident than in the office. In a multiperson office, do the employees help each other, complete tasks together, understand the job that each does, value the differences among them? Again, it is the head of school who can ensure that this happens. We understand and act on the need for professional development for teachers. School secretaries, bookkeepers and other similar staff can also benefit from opportunities to learn and grow. Have you made sure that they are up to date with the tools they need to perform well? When new technology is introduced to teachers, is it introduced to them as well? Would you consider researching programs in your community to enrich the careers of your office staff?
Often, the teaching and non-teaching staffs are viewed as two separate entities. But creating one unified group of employees, a group that shares a common understanding of the school mission and works together to achieve its goals, will significantly strengthen your institution. So while many staff meetings are not meaningful for non-pedagogic staff, many could be enriched by their presence. When you announce “full-staff” meetings, do you mean to include your office staff? Actively including them as part of the school community will build their commitment to the school, make them feel valued, and help them feel part of the larger community. And everyone will benefit as they will bring to the discussion a different and valuable perspective, and may suggest problem-solving strategies that result from their unique view of the school and its population.
Finally, don’t forget your school custodians! They, too, will do a better job if they understand not only the school rules but their underlying rationale. How have you brought them into the circle of your staff? They are a deep source of knowledge about what is happening in the school, and they have broad responsibilities for students and staff. Be sure that they get the respect due them; make certain that students and parents treat them with courtesy and dignity. Make sure they understand your school and what it stands for, and make them your partner in creating an orderly and attractive environment.
Jewish community schools must instill a true sense of community in all of its employees, regardless of their specific job descriptions. All contribute to the success of the school; all roles are interdependent. The head of school, as their leader, must create the climate in which this sense of mutual dependence and respect is fostered.

Fifteen years ago, Robert Putnam’s book Bowling Alone provided a portrait of the diminution of social, political and familial connectedness. Putnam methodically documented declining participation in and connection to our religious and civic institutions over the last several decades. The collapse of bowling leagues and other associations where relationships had been forged and nurtured had profound repercussions on our bottom line, our outreach and even our psyches. Putnam argued that ultimately we need to find ways to reconnect with one another. It’s hard to argue with that.
Social scientists have shown the damaging effects that social isolation has on people’s character. Large generational increases in self-esteem and even narcissism are clear, and each of us has likely felt this deleterious change in our everyday interactions. (Read Twenge and Campbell’s The Narcissism Epidemic: The Age of Entitlement, if you can get through it!). 
Author and New York Times columnist David Brooks explores some of these issues in his recently published The Road to Character. Brooks, who sent his children to Jewish day schools, says he wrote the book to save his soul. While he admits that he “gets paid to be a narcissistic blowhard,” he seems to want to be and be thought of differently. He may have done us a favor. He’s started a conversation. And that conversation seems very aligned with what we do.
The road to character is for every one of us who is trying to become a better version of ourselves. It is about shifting the cultural conversation to sharing stories of moral adventure and learning from each other’s examples. 
Because everyone’s road is going to be different, but that doesn’t mean we can’t take some steps together.
Check out his website: theroadtocharacter.com. It’s inspiring.
While certainly not monolithic in approach or philosophy, our schools must continually navigate the effects of community change and individual mindset while being nimble to address them. In addition to achieving academic excellence and financial sustainability, we have to aspirationally build communities that teach and deliver meaning.
We know that our schools are often sought out by families because they provide community, an oasis that offers an antidote to the crisis described in Putnam’s book. We also strive to have our curricula provide the moral framework to live a good life based on Jewish values. 
Internally, the RAVSAK board wrestles with these questions. Understanding the impact of trends on the big tent of community day schools is often part of our conversations. Our staff engages with school leaders daily on many issues, piecing together a bird’s eye view from reports on the ground. Hopefully, many of you in our network are also engaging with one another through our RAVSAK Reshets and at day school conferences.
Have we honed our message? Are we changing the conversation? 
The key to our success may very well be predicated on whether and how well we build school communities where relationships are formed and valued and meaning is accessed in a variety of ways. As we know, increasingly diverse families  seek meaning in a variety of ways. Our job is to talk about it. With a lot of people.
Consider building relationships by providing excellent family education; try speaking with those professionals who are doing outreach and engagement work and see if we can expand the JDS tent. Make sure we speak as comfortably and effectively to those who have never stepped foot in a Jewish day school as to those who always will.
Think about establishing an online storytelling space for your community like David Brooks has. We might be surprised and inspired by the stories and messages we hear.

We measure distance, we assess learning, and we evaluate results in terms of some set of criteria. Bob Kizlik

Disneyworld has measured the distance visitors will walk from a concession stand before throwing a wrapper on the ground. That distance is 27 steps. Thus, if you go to Disneyworld, which prizes cleanliness, you will find a trashcan every 27 steps along your way. This is an example of good data, valid assessment and meaningful evaluation leading to positive results.

In David and Goliath, Malcolm Gladwell posits the case of a good but not great student who is accepted into a top tier university as well as a state college. She wants to major in science. Gladwell demonstrates conclusively that good students will do better at a good college than at a high prestige institution, and will be more likely to complete their desired course of studies there. The evidence is absolutely unequivocal. Where should the student go? How many of us, faced with advising one of our students or graduates, perhaps our own child, to decide between the prestige school and the good but not renowned school will not opt for the Ivy League—even though all the evidence points to the opposite conclusion?

This illustrates a point made recently in eJewish Philanthropy by David Cygielman, founder and CEO of Moishe House. He writes that “in the Jewish community we have grown to value data and evaluation. It is crucial to be learning but only if we are using that information to make better and often times, different decisions than we made prior to obtaining this information. What I see much more … is the less healthy process of making a decision based on beliefs and then working to find or create data to show why we actually made the best decision. If the initial information or data doesn’t make our case compelling, we will find new information or data. This data and evaluation that we spend so much time, money and energy on is only beneficial if it is used to change or inform our decision making; yet, if it doesn’t, the facts remain True But Useless.”

Assessment, the focus of this issue of HaYidion, is only of value if it employs meaningful criteria. A recent New York Times article by Adam Davidson examines the fallacies that underlie the US government’s assessment of the economic data that determine social and economic policy. Davidson points out that these assessments “rely on fixed definitions—created decades ago—of the phenomena they’re charged with measuring” and thus are meaningless in an age ruled by technology. He gives the example of the impossibility, just looking at the raw data, of distinguishing “the creation of Facebook from the opening of a small deli in Dubuque.”

Another Times columnist, Phyllis Korkki, recently wrote that while “big data has made it possible to measure employee performance more thoroughly than ever,” the publication of the resulting rankings had exactly the opposite of the intended effect (motivation of employees to work harder), trumped by human nature and simple math. Most people assume they are above average but, statistically, most people are average or below average, which is demoralizing and leads employees to work less hard.

We hope you will find this issue of HaYidion helpful in making valid assessments of the measurements you take in your own setting, and in making meaningful evaluations of those assessments. As with everything we do in Jewish education, we have a moral, faith-based compass to guide us to avoid misusing true but useless facts and unanticipated results. Thus, for example, it might fly in the face of all measurement and assessment for small Jewish day schools to exist, but we know in our hearts that they are needed, that they impact students’ lives tremendously and that their communities would be weakened without them. In this case, measurements of financial resources and assessments of sustainability must be evaluated according to the criteria of the value of Jewish continuity and pluralism. As sociologist William Bruce Cameron said, “Not everything that can be counted counts, and not everything that counts can be counted.”