The National Child Welfare Resource Center for Family-Centered Practice convened its first Annual Meeting of State and Tribe Child Welfare Officials on November 29, 2000 in New Orleans. The meeting focused on the new Child and Family Services Review (CFSR) and provided an opportunity for 41 states and seven tribes to learn about it from the Children's Bureau. Peer-to-peer technical assistance was also provided in workgroup sessions. Jerry Milner, Child Welfare Specialist with the Children's Bureau, was the keynote speaker. His opening address, "The Vision of and the Opportunity for the New Review Process," focused on the development and importance of the CFSR process. Child welfare staff who participated in testing the CFSR provided many ideas, strategies, and suggestions for others preparing for their review. Below we provide some of the key points that Jerry Milner discussed, and focus on the importance of collaboration in the review process as discussed in the workgroups.

"Not everyone is as excited about this review process as I am," said Jerry Milner at the conference. "It has been very exciting to be a part of something that has such tremendous potential and promise for affecting the quality of services to children and families in our country."

Jerry Milner has been instrumental in creating the CFSR, which has benefited from and contributed to the quality assurance system in Alabama that has been operational in the state for several years. Using this experience, he has refined it further. Over the past five years, 14 states have piloted the CFSR. He says that for the most part, the 14 pilot states that helped in the development were enthusiastic about it. Some suggest the states were eager because there were no penalties attached for noncompliance; there was nothing at risk. Milner disagrees with that idea, however.

"The 14 pilot states took a tremendous risk in opening up their systems to expose, not only the things that were working well, but things that needed improvement. We appreciate their willingness to help shape the process." Based on their results, the review has been further refined. At the same time Milner says, "We were able to preserve the fundamental concepts of this overall review-a focus on the outcomes and quality." While Milner concedes that the review process is not yet a perfect process-tremendous revisions have been made along the way and will continue as we learn more-he is convinced that "it's just so darn close."

Milner stresses the importance of the review process, both for those in the field and for policy makers. "There are members of Congress who believe that this process offers us the hope of having positive effects on the way that child and family services are delivered to children and families throughout the country. Don't underestimate the importance of this."

The reviews are important because of what they are designed to accomplish and how they are done. First, the reviews give us a process to look at outcomes for children and families. While previous reviews were based on documentation and thoroughness of case records, the CFSR gives us the opportunity to see what is actually happening to children and families. It is about outcomes: whether children are safe, whether children in foster care are moving toward permanent outcomes, and whether the well-being of children and families that we serve is promoted and assured by the services we provide.

"These should be the measure of our success in child welfare, as opposed to whether we have actually documented everything in the record," explains Milner.

Second, the reviews are designed to give information that state, tribal, and other child welfare leaders, officials, and staff can use to make program improvements in their states. "The review will yield information that is important at the national level to help shape policy, to help allocate resources, and design the programs throughout the state that provide services to the children and families in your state," Milner pointed out. The opportunity to make program improvements is the primary reason that the process has been designed. "We can determine the strengths and what the needs are within programs," says Milner. "We don't just walk out of a state after a review is over; we continue to work with the states to make program improvements where we have identified the need."

Third, the reviews are important because, like services to children and families, they are part of an ongoing process. "The real heart of this process," Milner says, "is the program improvement aspect of the Child and Family Services Review. It signals a new way of doing business with states because it incorporates so many of the programs that we currently work on with states." The reviews include many different facets and activities that extend over time, as we work with states to move all states toward a higher standard than we have now. The review process also emphasizes practice principles that support improved outcomes for children and families, family-centered practice, community-based services, and individualizing services.

Next, the CFSR is comprehensive. It looks at all the programs, and all the services that affect the children and families we serve. Since many children and families receive services of many different programs, it only makes sense to look at them together, not separately.

"You will hear the word 'partnership' floating around, and that word gets used a lot, but sometimes it really doesn't add up to partnership," notes Milner. What does this mean in the CFSR context? Milner explains that these reviews are conducted in collaboration with states and with state staff and external representatives who can give us a broader perspective about the services and outcomes within that state. These staff sit along side one another, are part of the process, in meetings, in decision-making. At the end of the process, there should be very few surprises for the state, because the state staff have been involved throughout. "Partnership actually materializes in some significant and important ways." This partnership also provides tribes with opportunities to be involved in this process.

Accountability is another reason the reviews are important. "We are not sacrificing accountability," said Milner. "All of us share the responsibility to see that programs are administered as they were intended, that funds are spent on behalf of children and families in appropriate ways, and that we are moving steadily towards the outcomes that those programs were intended to achieve for children."

The review gives us an opportunity to look at factors that have a significant impact on state agencies' ability or capacity to help children and families achieve positive outcome-the systemic factors. These are the state child welfare agency's information system capacity, training of staff and providers, the case review system, recruitment of foster and adoptive families, quality assurance systems, involvement of community stakeholders in family safety, permanency, well-being, and the array and availability of services.

CFSR: Not a Checklist

"The CFSR is meant to be a process that will yield important information of depth and quality. It is meant to be difficult because the work that we are reviewing is very hard work," cautioned Milner. It requires careful planning and assessment and crucial decision making. "We decided that if we had a review process that did not reflect that level of effort, and did not review some of those same kinds of processes that we expect out of our case workers in the field, then we would not be able to collect the kind of information we need to change practice, to improve practice, and to build on the strengths that we discover in the process." The review mirrors what best practice should be.

A checklist type of review-read a few records and declare the state in or out of conformity and move on merrily to the next state-would have been much easier, concedes Milner. "There have been a few occasions when I thought that would have been a better way to go; less controversial," he chuckles, "But the reviews are strongly focused on quality-the quality of services and the quality of outcomes-and quality doesn't come cheap or easy."

The Review poses challenging questions: it cannot be reduced to a compliance-oriented checklist. The CFSR is inherently a process requiring negotiation, consultation, evaluation, and partnership between the federal, state, and regional office, and between the child welfare agency and the diverse constituency needed to produce solid outcomes for children and families. Perhaps this is what Milner means by "perfect."

CFSR: A Collaborative Process

According to those who have experienced the review process, involving other stakeholders-from the local community to the federal level-in the process is both essential and required. "We require that there be external participants in the review process," explained Milner. "Once we have given external representatives, including some fairly harsh critics, a stake in the process and an opportunity to be part of the process, their support of the agency is solidified. In some cases we have had some support from some very unexpected places because of the way we have included partners."

Who are the stakeholders? According to the planning process required by the IV-B Children and Family Service Plan there are nine categories of stakeholders involved in the consultation process:

1. All appropriate offices and agencies within the state agency: for example, child protective services, foster care and adoption, the social services block grant, emergency assistance, reunification services, independent living, and other services to youth;

2. In a state-supervised, county-administered state, county social services and/or child welfare director or representatives of the county social service/child welfare administrators' association;

3. A wide array of state, local, tribal, and community-based agencies and organizations, both public and private nonprofit, with experience in administering programs to infants, children, youth, adolescents, and families, including family preservation and support;

4. Parents, including birth and adoptive parents, foster parents, families with a member with a disability, and consumers of services;

5. For states, representatives of Indian tribes within the state;

6. Representatives of professional and advocacy organizations: for example, foundations and national resource centers, individual practitioners working with children and families, the courts, representatives of other states or Indian tribes with experience in administering family preservation and family support services, and academicians;

7. Representatives of state and local agencies administering federal funds: for example, Head Start; the local education agency; the health agency; and law enforcement;

*. Administrators, supervisors, and frontline workers of the state and child family services agency, and;

9. Other categories of organizations and individuals based on state and local circumstances.

Stakeholder involvement must be substantial. Milner outlined how stakeholders are used in the review. Stakeholder interviews are held in three different locations in the state. "Statewide" stakeholders are also interviewed. They are individuals or representatives who can give not just a local perspective on the seven systemic factors, but also a broader statewide perspective.

"We strongly encourage states, as you choose those external representative to be part of that statewide assessment, to tap the valuable resource among tribal representatives to include tribes as part of the process in evaluating outcomes and the systemic factors," said Milner. In addition, he urged states to make use of their court improvement project staff to get greater court involvement. "Many of the areas that we will be reviewing," he explained, "have direct links to the judicial system, and it's important that we have the investment of the courts as we evaluate how well many of those issues are functioning."

CFSR: Making It Work
While the conference offered many other suggestions for preparing and completing the review, involving others was a recurring theme and worth sharing again.

Working with others involved, creating partnerships and sharing ownership, brings surprising results.

The key to a successful CFSR is orchestrating a comprehensive consultation that everyone understands. One simple strategy for this, for example, is to send letters with brief descriptions of the CFSR process to all potential stakeholders who might be involved. Translate the process into terms that people can understand in their day-to-day work. They will be more prepared to begin the process.