The best partner. School effect assessment of different interorganizational collaboration using Propensity Matching Score

El mejor colaborador. Evaluación del efecto escolar de diferentes colaboraciones interorganizacionales utilizando Propensity Matching Score

Abstract: The Chilean Skill for Life (CSFL) is a school mental health program implemented by local agencies from the Education, Health, and Social Services public sectors. It represents an excellent opportunity to study inter-organizational collaboration and its advantages to public and state-subsidized schools. The propensity score matching technique was used to compare school performance in second grade (the most intensive treatment level) between schools participating in the program and those not participating and between schools with different types of sectoral collaboration to identify the best partner for the school. To select all Chilean schools’ participant in CSFL and the comparative group of schools’ non-participant, a sequential sampling was applied. The measures were obtained from government public data, considering annual school performance and other educational indicators. It was found that public schools that implement the CSFL obtain better school performance than those that do not implement it (ATT = .042; p < .05), for state-subsidized schools, the same was not found. It was also observed that when educational agencies implement the program, the gain is more significant (ATT = .046; p < .05). The importance of aligning program values and goals with local agencies and schools is discussed, analyzing the possibilities for better collaboration in school mental health.

Keywords: inter-organizational collaboration, implementer agencies, school mental health, educational outcomes, propensity scores.

Resumen: Habilidades para la Vida (HPV) es un programa chileno de salud mental escolar multinivel que llega a millones de estudiantes en miles de territorios del país, y es considerado uno de los programas de salud mental escolar más grandes del mundo. Es implementado en el país por agencias locales de los sectores públicos de educación, salud y servicios sociales, lo que representa una excelente oportunidad para estudiar la colaboración interorganizacional y sus ventajas para las escuelas públicas y subvencionadas por el Estado. Este constructo ha recibido gran atención en los últimos años en el campo de la salud mental escolar, dado que destaca la importancia de generar procesos que permitan compartir una identidad colectiva, una agenda coordinada, tener una comunicación efectiva y una colaboración mutua entre los diversos dispositivos que buscan contribuir al logro de las metas de salud mental y a los objetivos educativos de las escuelas. A pesar de los grandes beneficios que traería la colaboración interorganizacional en la ejecución de programas de salud mental escolar, los estudios al respecto son más bien escasos y centrados en intervenciones que vinculan tan solo un tipo de agencia ejecutora con las escuelas. Por este motivo, el presente estudio pretende analizar el impacto de la colaboración interorganizacional en el rendimiento académico de aquellos estudiantes que reciben el programa HPV, teniendo dos hipótesis a la base: (1) que aquellos estudiantes que forman parte de las escuelas en las que se entrega el programa tendrán un mejor rendimiento académico que aquellos que no reciben la intervención; y (2) que el sector educación será el mejor partner o colaborador, puesto que comparte las metas educativas con la escuela, lo que impactaría en el rendimiento académico de aquellos estudiantes que reciben el programa. Para medir esto, se utilizó la técnica Propensity Matching Score, la cual sirvió para comparar el rendimiento escolar de los estudiantes en segundo grado (el nivel de tratamiento más intensivo) entre las escuelas participantes y no participantes del programa, y entre las escuelas con diferentes tipos de colaboración sectorial. Para seleccionar a los participantes de las escuelas chilenas en HPV y del grupo comparativo se aplicó un muestreo secuencial. Las medidas se obtuvieron de datos públicos del gobierno de Chile, y se consideró el rendimiento escolar anual, la vulnerabilidad social de las escuelas y otros indicadores educativos. Se encontró que las escuelas públicas que implementan el HPV obtienen un mejor rendimiento escolar que las que no lo implementan (ATT = .042; p < .05), resultados que no se repitieron en aquellas escuelas subsidiadas por el Estado. También se observó que cuando las agencias educativas implementan el programa, el aumento del rendimiento académico es mayor (ATT = .046; p < .05) respecto que aquellas que lo implementan mediante agencias locales de salud o de servicios sociales. Se discute la importancia de alinear los valores y las metas del programa con las agencias locales y las escuelas, analizando las posibilidades de una mejor colaboración en la salud mental escolar. Esto implica también considerar las metas educativas de cada centro en el cual se implementan estas intervenciones, ya que es posible encontrarse en ocasiones con modelos educacionales restrictivos y tradicionales, centrados por sobre todo en lo cognitivo. Por este motivo, experimentan dificultades al alinearse con programas de habilidades socioemocionales, lo que termina obstaculizando las posibilidades de colaboración interorganizacional con agencias interventoras de dispositivos orientados a esto. En este marco, se exponen una serie de consideraciones clave vinculadas a la importancia de preparar las condiciones y los recursos para el trabajo interorganizacional entre escuelas y entidades ejecutoras de programas de salud mental escolar.

Palabras clave: colaboración interorganizacional, agencias ejecutoras, salud mental escolar, resultados educativos, puntajes de propensión.

Introduction

Schools daily face learning barriers imposed by mental health problems (Suldo, Gormley, DuPaul, & Anderson-Butcher, 2014). Students with mental illnesses have less school performance and academic motivation, and more social difficulties and absenteeism than healthy students (Vanderlind, 2017). Although schools are interested in addressing their students’ mental health problems, their aim purposes are educational, not of public health (López, Carrasco, Morales, & Ayala del Castillo, 2011), therefore, School Mental Health Programs (SMHP) must be useful both to improve the social and emotional health of students as to help the school reach its goals (Atkins, Cappella, Shernoff, Mehta, & Gustafson, 2017).

Despite its importance for the progress of the scientific and professional field of school mental health, the authors pay little attention to school performance effect of SMHP, although those who do show how the strengthening of socioemotional skills and coping strategies allows to significantly improve the learning processes (Hoagwood et al., 2007).

Becker, Brandt, Stephan and Bruce (2014) reviewed 85 articles of the SMHP, finding that 83.3 % of the time the participants surpassed the control group in academic results after controlling the type of measurement, the problem solved or the place application of the program (inside or outside the school). In another review, Durlak, Weissberg, Dymnicki, Taylor and Schellinger (2011) carried out a meta-analysis in 213 articles evaluating their effectiveness for the promotion of social-emotional skills and found that students obtain 11 percentile points more in school performance (mathematics and language) compared to those students who do not participate. They also found that the impact is more significant when the teachers implement the program instead of other professionals.

Collaboration for School Mental Health

Achieving educative successes with few resources in complex urban contexts is a challenge for school communities (Atkins, Hoagwood, Kutash, & Seidman, 2010). Because of this, some schools prefer to focus only on the pedagogic tasks, while others hire mental health professionals or collaborate with local agencies, seeking to reduce learning barriers (Fazel, Patel, Thomas, & Tol, 2014; Splett, Fowler, Weist, & McDaniel, 2013).

Collaboration with local agencies of the Health, Social Services, and Education public sectors is the most common way of implementing SMHPs (Weist et al., 2017). However, like the school effects, the collaboration is poorly studied, so its impact on the results of the SMHP is unknown (Lyon et al., 2018).

Collaboration is based on purposes and shared symbols, recognition and respect of differences, establishing clear and regular communication systems, and especially in confidence that others can help. This overlapping in collaborating organizations implies moving towards integration, that is, a shared identity, sometimes different from the initial one, which broadens the inter-organizational context, fusing the internal and external background, allowing to develop an integrated system to achieve a collaboration plan (Horwath & Morrison, 2007; Lyon et al., 2018; Stephan, Connors, Arora, & Brey, 2013).

For SMHPs, the collaborating agency should understand values and desire to contribute to the goals of the educational system, and the schools should believe, appreciate, and agree with the aims of the agency. Therefore, local agencies should not function in parallel, or instrumentally use schools for their organizational purposes, but should structurally align to begin an active and integral collaboration (Atkins et al., 2017; Bedoya-Gallego, Buitrago-Duque, & Vanegas-Arbeláez, 2019; Corcoran, Rowling, & Wise, 2015; Tooher et al., 2017).

Collaborative research in school mental health is focusing on the alignment of psychosocial variables that make up the inter-organizational context (Lyon et al., 2018), leaving aside the analysis of the structural conditions that are common to the collaborating organizations. This aspect is critical in the case of SHMPs governmental promote, where the public sectors responsible for implementation share an administrative-legislative background depending on the level of integration of public policies oriented to education, childhood and mental health.

The most common SMHP implementation barriers described by the literature are included among the structural factors that are considered key to effective collaboration. These are: (1) educational policies and the pedagogical model (Forman, Olin, Eaton, Crowe, & Saka, 2009; Thielking, Skues, & Le, 2018); (2) financial resources availability (Stiffman et al., 2010); (3) goals of the sectors involved in the collaboration (Powers, Edwards, Blackman, & Wegmann, 2013); (4) times in the institutional agenda and workspaces (Sarno et al., 2013); (5) implementation and competency supports (Blase, Van Dyke, Fixsen, & Wallace, 2012), and (6) the roles, functions and disciplinary rationalities of the professionals involved (Mellin, & Weist, 2011).

These factors can be considered in order to assess the degree of alignment of a collaboration from a structural perspective, which would complement the psychosocial model of inter-organizational context analysis. Therefore, it’s possible to check the shared factors and estimate the possibility of effective collaboration, hypothesizing that when the organizational partners are exposed to the same structural conditions, they will be more likely to collaborate effectively.

Current Study

The collaboration in school mental health is a barely explored topic (Thielking et al. 2018). Given the variety of public sectors of local agencies that implement SMHPs, it is necessary to find out if there is one sector that is more effective to work together. That is if there is one that can be call the best partner to reach the goals of the schools. In order to find out, this study analyzes the case of Chilean School for Life program (CSKL), considering the following specific objectives:

1) To compare the school performance between school participants in the CSFL and non-participants considering different analyses for the public and the State-subsidized schools, since they have different administrative and educational characteristics that prevent their homogenization.

2) To compare the school performance in public schools according to the public sector (Education, Health, and Social Services) of the local partner agency that implemented CSFL.

3) To compare the school performance in State-subsidized schools according to the public sector (Education, Health, and Social Services) of the local partner agency that implemented CSFL.

From this background, two comparative hypotheses were established. First, schools (public and private subsidized) that participated in the CSFL will have better school performance than non-participants. And second, that school performance will be better when local educational agencies implement the CSFL because they share the same administrative units and political mandates. Recognizing if collaborating education agencies are better partners for implementing mental health programs in schools will allow advocating for the inclusion of these activities within the school curriculum. Also, it will reduce the perception of mental health intervention as an external complement that uses schools instrumentally, promoting synergist school activities.

Method

Chilean Skills for Life Program

The CSFL is one of the eight SMHP with the highest coverage around the world (Murphy, Abel, Hoover, Jellinek, & Fazel, 2017) and represents an exceptional opportunity to study the execution of complex interventions in real contexts. The program’s aim is to improve school performance and mental health of students in high social risk elementary schools^[1].

In first grade, students are evaluated in School Adjustment; one year after, in second grade, students receive the highest treatment and all the strategies of the intervention model (i. e., promotion, preventions, and universal screening based assisted referral up), special psychoeducational workshops of socioemotional skills; and in third grade they are evaluated again to analyze the intervention outcomes. In other grades, the students only participate in promotional interventions and can be clinical referred to the Public Mental Health Centers.

The CSFL Theory of Change incorporates contributions from cognitive behavioral therapy, socioemotional learning, developmental epidemiology, community mental health approach, and public health systems. This theory predicts that when there is a school culture that promotes mental health, a nutritious learning climate is generated and positive interactions between adults and students develop, which allows to regulate school demands and adjust the necessary support strategies. These positive interactions would protect students from mental health risk trajectories by reducing learning barriers, promoting a better school performance (George, Guzmán, Flotts, Squicciarini, & Guzmán, 2012; Vargas & Peña, 2016).

CSFL has a promotional focus, its interventions seek to impact role and meaning interaction established by significant adults with the students (George et al., 2012;Leiva, Zavala-Villalón, Antivilo-Bruna, Torres, & Ganga-León, 2020). The consolidation of this positive relationship is through the strengthening of socioemotional competencies of the teachers and parents, which operate as moderators of school demands and the positive valence of socioemotional aspects of school life (Vargas & Peña, 2016).

Evidence shows that CSFL promotional activities can create positive learning climates, foster cohesion among school members, and stimulate the well-being of the whole community (George et al., 2012). Regarding the school effects of the program, they have only been evaluated considering the preventive intervention and without having incorporated control groups but comparing the impact at different dose-effect (low dose –none to 7 sessions– vs. high dose –8 to 12 sessions). Guzmán et al. (2015) found that students who receive high doses of socioemotional training are less likely to repeat the third grade or have an attendance below the minimum established. Likewise, Leiva et al. (2015) found that high dose generates a 13.2 % increase in cognitive achievements associated with motivation and school performance evaluated by teachers compared to low doses.

Collaboration in CSFL

The National Board of School Aid and Scholarships (JUNAEB), a national agency of the Ministry of Education (MINEDUC), designs and implements CSFL, while local agencies execute it. The local agencies are part of the local government, who are invited to execute the program in each district, providing at least 20 % of the annual financial resources.

When local agencies apply for the program, they must submit a collaboration contract signed by the schools’ principals. The rationality of the document is to highlight an agreement collaboration between CSFL partners, expressing their active interest in school mental health initiatives and assuring minimal implementation conditions. Although the engagement of the schools is a requirement for the awarding of CSFL, it is one of the most critical factors of national implementation during the initial phases (Leiva, Rojas-Andrade, & Gonzalez, 2018).

The program is executed in 50.28 % of the country's district (n = 174), and most of the implementing agencies belong to three types of public sectors of local government: Education, Health, and Social Services.

Local Educational Administration Department manages all the public schools (entirely funded by the state) and controls the local Education Development Plan, through which they prioritize resources and establish educative goals for the district. When this type of agencies implements the CSFL, the mental health team works in the same place with special education experts and curricular specialists, sharing authorities, organizational structures, and working conditions.

Local health agencies can also implement the program. These public departments supervise the administration of Primary or Secondary Care Centers, allowing more efficient access to Mental Health Centers when the students require it. The CSLF professionals generally work with or closely to psychiatrists and clinical psychologists in primary health centers, sharing the public health goals and working conditions with sanitary personal.

Finally, the program can be implemented by social services departments. These departments execute social projects of housing, quality of life and social empowerment. The CSFL professional team normally works with promotion and protection of children’s rights teams, sharing a robust political vision about childhood and participation. Of the three sectors, the most unstable is the last one, because the projects critically depend on local policies while the others depend on national policies in order to cover their resources.

CSFL Schools

The local agencies implement the CSFL in public and state-subsidized private schools. In Chile, these schools operate under different conditions because of State shared funding bonds and parental copayment. In public schools, parents do not pay fees, while in state-subsidized private schools the parents’ copay is limited to USD 125 per children. Likewise, state-subsidized schools must enroll at least 15 % of social risk students, exempting them of school fee copay (Santiago, Benavides, Danielson, Goe, & Nusche, 2013). In CSFL schools, 80 % of the students are at social risk (Junaeb, 2018).

Sampling

A sequential strategy was used to define the analytical sample. Schools from districts where the CSFL is implemented were selected, and rural schools were discarded in order to reduce geographic and cultural biases. Subsequently, schools were separated into two subsamples, public schools (n = 1 329) and state-subsidized private schools (n = 2 000). Finally, the schools were classified according to their participation in the program (yes.no) and the implementing agencies sector (see Figure 1). Both samples represent the whole population of schools with CSFL and are of adequate size for multiple regression analysis (Knofczynski, & Mundfrom, 2008).

Measures

Secondary data from MINEDUC’s official student information system and JUNAEB’s data system were used. A database was created with information related to participation in the CSFL program, annual school performance and educational indicators for each school in Chile. The operationalization of these variables is as follows.

School performance

The school performance was measured through the mean of student’s grade points obtained in all the subjects of the yearly study plan (i. e., Grade Point Average). The grading system in Chile ranges from 1.0 up to 7.0 (with one decimal place).

School resources

The school resources was measured through (1) the number of students enrolled in second grade and several courses in this same level; (2) the number of teachers; and (3) the number of education assistants, hired at the school, who had some relationship with mental health. Most of these were psychologists (35.1 %) and language therapists (27.5 %).

Quality of Education

School education quality indexes were obtained from the National Performance Evaluation System (JUNAEB, 2018). The indicators are: (1) Effectiveness (SIMCE result of the school); (2) overcoming (differences between the SIMCE results over time); (3) improvement of working conditions (compliance with administrative processes and suitability of teachers and education assistants); and (4) equal opportunities (accessibility, permanence and integration of students with multi-deficit).

Social Vulnerability

The social vulnerability of the schools was measured with JUNAEB’s School Vulnerability Index (IVE). The IVE serves as an input for the planning of public programs and the distribution of resources allocated to schools through the Law of Preferential School Subsidies (Elacqua & Santos, 2013). For its calculation, JUNAEB classifies students into three priorities estimated from poverty and school risk conditions. The first corresponds to students in extreme poverty whose families participate in state programs and subsidies destined for the most impoverished population of the country. The second and third priorities are constructed based on indicators of school failure. The total score of the IVE is a ratio between the sum of the students classified in the three priority categories and the total number of students enrolled in the school.

Collaboration

To identify the type of sectoral collaboration, the local agencies in charge of implementing the program were considered. Each school was associated with a partner in the public sector; Health, Social Services or Education. Schools with partners from other sectors (i. e. University) were discarded from the analysis, since there were so few cases.

Analysis design

The Propensity Score Matching (PSM) was used to estimate the school effects (Garrido et al., 2014). This method calculates the impact of treatment on the group receiving with Average Treatment Treated (ATT). Despite the scientific criticisms of the PSM to assess functional or pseudo-causal relationships, the authors recognized it is a useful strategy to analyze the impacts of interventions carried out in natural contexts, in which there is little control of the variables and in which the ecological validity and social significance are more important than the validity and statistical significance.

The technique consists of two phases. The first is the estimation of the probability of participation in both groups (control and treatment) and the second, the comparison of the result between the actual participants and the potential participants. The potentials are understood as “clones” that, in statistical terms, have the same characteristics as the first group, but did not participate in the treatment.

The first phase involves calculating the probability of all schools participating in the Program through propensity models. Formally the propensity score is defined through the following equation:

(1)

Where p(T) equals the probability that a school receives the program, T indicates that a school gets or does not receive the program, and S is a vector of the influence of covariates regarding whether a CSFL school. Intending to reduce moderation biases, the covariables included in the vector S were only those that had a significant relationship with the academic results. For this, regression models were carried out considering the following equation:

(2)

Where y is annual school performance; α equals the term of the constant; each x. represents the educational indicators; each β. represents the parameter for each x., and ε is the error term.

To finalize the procedures, the recommendations of Garrido et al. (2014) were followed, evaluating the balance of the Propensity Score between the participating and non-participating schools through the analysis of the common support zone (positive probability of being both participant and non-participant). Then the ATT was calculated through a pairing method with the nonparametric Kernel estimator, which compares the participant case with its respective control group, suitably weighing them by a function of the distance of each control case for the probability of participation.

In this method, observations outside the common support zone were discarded. This method was chosen because of its ability to maximize precision and reduce matching bias (Garrido et al., 2014). On the other hand, the quality of the matching was evaluated through the percentage reduction of the differences in the covariates between the group of schools that did not participate in the CSFL and those that did. As a criterion, the reduction in disagreements was considered to allow that, after matching, the differences were not significant. It’s interpreted ATT as the effect size. To calculate its significance, bootstrapping procedures (100) by unbiased calculation of standard errors are used. Its result is expressed as proportions of gain in standard deviations (of the total sample) concerning the control group. This analysis was carried out separately for public and state-subsidized private schools. PSM was carried out in each of them, differentiating the sector to which the implementer agencies. For the calculations, the pscore and psmatch 2 commands of STATA 15.0 are used.

Results

Predictors of school performance

Multiple regressions were performed to evaluate the functional relationship between the covariates and annual school performance. As shown in Table 1, some indicators were found to be significantly associated with school performance in both types of schools, and others were found to be specific only to public schools. The regression weights of the different significant indicators (p < .05 or .01) ranged from -.005 to .009. Although these values are low, their interpretation should consider both the multi-causality of school performance and the scale with which the variable was measured and its dispersion.

Estimation of the probability of participation in the CSFL program

The score function to balance propensity scores in participating and non-participating schools was used. This function divides the propensity scores into quintiles and evaluates, for each one, the differences between the participating and non-participating groups, concerning predictors (i. e., significative predictors of school performance regression model). The covariates included allowed for an optimal matched without the need for an additional procedure. In the comparisons, the common support zone included 100 % of the cases, except for public schools that collaborated with the Health sector (97 %) and Social Services sector (98.5 %).

To evaluate the matched, the percentage reduction of differences between the covariates before and after the pairing was analyzed. As shown in Table 2, the difference reduction percentage fluctuated between 99 % and -28.9 %. In all cases, the differences in the covariables between the participating and non-participating schools were not significant after the pairing.

School effect of CSFL program

ATT was estimated with the matched cases using the Kernel algorithm. Without distinguishing the agency sector, it was found that CSFL schools obtain better academic results than those where the program is not implemented (see Table 3). Although the enhancing effect of CSFL school results is small, it should be considered that the study is carried out at the national level under real conditions of implementation. The effects of the program were found to be significant only in public schools (ATT = .042; p < .05), producing a 16.24 % standard deviation gain for schools that implement it.

When distinguishing the public sector from the associated sector, it was found that, in general, all combinations produce positive school effects with different size and significance. In the case of public schools, the agencies generating the largest effect size come from the Social Services sector (ATT = .056; p > .05) and the smallest, from the Health sector (ATT = .028; p > .05). However, in both cases, the effects were not significant. In the case of private charter schools, the health sector agency produces larger, non-significant effect sizes (ATT = .038; p > .05), while the education sector partners generate the smallest results (ATT = .005; p > .05).

Finally, it was found that when the agency comes from the Education sector and implements the program in public schools, positive and significant school effects are obtained (ATT = .046; p < .05), generating a SD gain of 17.78 %.

Discussion

The mental health of students has become an essential focus of attention for the Education sector due to the critical effects over learning. For this reason, different schools have begun to implement programs that address these problems to obtain better results. However, given that schools do not have the necessary resources to carry out these interventions, it is common to develop collaborative alliances with local agencies that take care of them.

This research aimed to estimate the school effects produced by SMHP implemented by collaborating partners from different public sectors, through the Chilean case of the CSFL. Two results were expected: the first, that the schools that participate in the program will obtain better academic results than those that do not participate. The second, that the educational effects vary according to the sector of the agencies, with the Education sector being the one that would produce the best results due to the higher capacity of integration between both partners (school and local agency).

Regarding the first, the matching methodology showed that the CSFL generates positive school effects in both public and state-subsidized private schools; however, this is only significant in public schools. It is crucial to highlight that, although in statistical terms the size effect is small, the research was carried out in real-world conditions, so its interpretation and evaluation must consider that school performance is multicausal. Likewise, SMHP do not involve specific learning components, but rather a context of the school environment, through the strengthening of skills that improve positive interactions.

Previous research has not considered incorporating the variables of academic results as a product of SMHP, considering that this is a goal of the school and that, therefore, if the program wants to achieve a better collaboration, it should aim to respond also before these objectives (Becker et al., 2014; López et al., 2011). In a similar way, the results of the SMHP are measured at the individual level, since the educational establishments care about the results at the global level, at the school level (Corcoran et al., 2015; Durlak, 2016).

Regarding the second expected result, although the different sectors generate positive school effects, this is only significant when educational partners implement the CSLF in public schools. These results are explained by a higher level of collaboration and integration occurring between these two organizations. The public schools belong to the same educational sector that implements the CSFL, so there is a substantial overlap of the characteristics between the external professional teams and the members of the school. The external context merges with the internal setting, as soon as the same entity managed both schools and the implementing organization and are subject to the same political-structural conditions. In this type of collaboration, the SMHP is part of local educational policies, agencies, and school shared a financial resource, goals, and institutional agenda. All this could compliment the alignment of similar professional rationalities, which would increase the possibility of clear and regular communication.

In state-subsidized private schools, the inter-organizational alignment is more complicated. Although both the collaborating partners and the schools belong to the Education sector, one comes from the public sector and the other from the private sector, so different structural factors condition them. Also, the fact that different holders manage private schools, and each has a certain degree of curriculum autonomy, creates obstacles when reaching specific agreements on the program’s goals.

A national investigation has shown that professionals who implement the CSFL in state-subsidized private schools can face some restrictions to their regular work, sometimes due to the presence of an instructional-cognitive model that does not accommodate social-emotional dimensions or does not consider the mental health as a factor to enhance learning (Sandoval et al., 2018). Not only the implementer agencies must have an academic-oriented approach, but also the school must value the socioemotional work so that the collaboration proves effective. These requirements are possible to fulfill when schools and collaborating partners are part of the same organizational context and contribute to the same plan; however, when they belong to different organizations that understand education in a different way, this can be severely hindered.

This finding becomes especially relevant when schools interpret accountability as compliance with quality educational standards, rather than as an opportunity to improve and develop their teaching-learning processes. Because of this, it is essential that schools that collaborate with other sectors on implementing mental health programs understand the importance of these programs for their educational goals, and not only consider them as a way to contain the demands for health care required by their students (Bedoya-Gallego et al., 2019).

In addition, in private charter schools, it was found that when they collaborate with health agencies, they obtain better results than when they collaborate with educational agencies. This result could be associated with the fact that this type of schools has a higher affinity with the health approach, for the compensatory function that teachers assign to mental health professionals. An important result, although not statistically significant, is that the school effect was produced when a Social Services partner implemented the program in public schools. The schools are at social adversity urban contexts such as poverty, delinquency, and family negligence present, so that collaboration with Social Service agencies is an excellent support. In this line, it is necessary to study if the same leveling rationale present in state-subsidized private schools exists in other schools but focused on social leveling rather than leveling health.

It should also be noted that this type of research (focused on collaboration between agencies) is frequent in developed countries, where there are more resources and institutional support, so it is configured as a challenge to continue investigating the inter-organizational collaborations that arise in SMHP from low and middle-income countries, given that the schools of these nations have high levels of social vulnerability and a wide range of socioemotional development problems (Fazel et al., 2014).

Advancing the understanding of collaboration between schools and external agencies for the execution of SMHP is essential. Although this research shows that the best partner for the school comes from the Education sector, it is necessary to continue investigating how to improve integration when the partner does not come from this sector. As mentioned above, conceptualizing the demand from schools is likely to be essential to understanding the type of collaboration and the type of service that schools require and that partners can provide.

In this frame, studies interested in inter-organizational collaboration have not been interested so far in distinguishing the different types of partners with which an alliance in an SMHP can be forged (Lyon et al., 2018). Rather, they have focused on studying the dimensions of this collaboration, if most of the collaborators are community-based organizations (Lyon et al., 2018; Weist et al., 2017).

In the case of CSFL, it is a package of promotional and preventive interventions that must be implemented in its entirety to operate as an educational enhancer that favors the entire school community at all levels. The intervention model fits in with the public schools, apparently because they are all governed by the same plan and, therefore, possibly by the same demand for mental health.

It was found that an educational partner can produce positive academic results from school mental health programs, allowing to recognize the importance of the inclusion of these activities within the school curriculum. However, another type of partner maybe does too, although with other kinds of school results of interest. This implies that these programs can contribute to other fields in which schools detect needs and not only instrumentally to use to the space to carry out their own activities. For example, health partners can improve school well-being, and social service partners can improve social conditions. It could depend on school demand for mental health services, programmatic offers, and teamwork variables.

Future investigations should show how agencies from different sectors can work together to get socially significative outcomes. Mental health services, usually located outside of school, have professionals specialized in promotion, prevention, and treatment, which is useful to support students and their families, especially teachers that lead with a high-stress load.

In this sense, this research shows that the alignment of structural conditions matter, but fail to consider the variables of the agency and leadership that impact inter-organizational coordination, which should be investigated in depth later.

On the other hand, it is also necessary to investigate the implementation’s fidelity because, despite the collaboration and integration that the partners could have, the expected outcomes will not be obtained if the program is not applied correctly and completely, or if it doesn’t fit the context. Thus, each partner must take care of their part of the program’s quality of the implementation. For example, the school could guarantee a schedule of interventions and the collaborating agency could ensure the expert professionals and adequate materials.

Deepening these integration practices will provide empirical evidence to construct a collaborative model of school mental health applicable to Latin American realities (Cataldo, Liberatore, & Hermosilla, 2018), where political agendas recently incorporate mental and socioemotional health as an educational topic of interest.

Limitations

This research was conducted to estimate the importance of the different types of partners that can collaborate in CSFL using a quantitative perspective. While helpful in assessing program effects, this approach limited the understanding of the specific determinants of partner implementation. In the same vein, since collaboration was measured through a dichotomous variable information about the content and quality of collaboration was limited. Finally, since this was a national sample with a local administrative structure, the results are difficult to generalize to other latitudes and SMHPs. Even so, they indicate the differences that exist in collaborative partnerships.

Future researches should deepen it, through valid instruments that measure the different dimensions of integration and how these dimensions operate in various collaborative alliances. However, our study allowed a global understanding of school’s partnerships that contribute to local development of SMHP and allow for understanding how the programs work in real conditions, at a national level and in the middle-income countries of Latin America.

References

Atkins, M. S., Cappella, E., Shernoff, E. S., Mehta, T. G., & Gustafson, E. L. (2017). Schooling and children’s mental health: Realigning resources to reduce disparities and advance public health. Annual Review of Clinical Psychology, 13(14), 14.1-14.25. https://doi.org/10.1146/annurev-clinpsy-032816- 045234

Atkins, M. S., Hoagwood, K. E., Kutash, K., & Seidman, E. (2010). Toward the integration of education and mental health in schools. Administration and Policy in Mental Health and Mental Health Services Research, 37(1–2), 40–47. https://doi.org/10.1007/s10488-010-0299-7

Becker, K. D., Brandt, N. E., Stephan, S., & Bruce, F. (2014). A review of educational outcomes in the children ’ s mental health treatment literature. Advances in School Mental Health Promotion, 7(1), 5–23. https://doi.org/10.1080/1754730X.2013.851980

Bedoya-Gallego, D. M., Buitrago-Duque, D. C., & Vanegas-Arbeláez, A. A. (2019). Transdisciplinariedad en salud mental: Una propuesta de modelo de formacion. Interdisciplinaria, 36(1), 119-133. https://doi.org/10.16888/interd.36.1.9

Blase, K., Van Dyke, M., Fixsen, D., & Wallace, F. (2012). Implementation science. Key concepts, thems, and evidence for practitioners in Educational Psychology. In B. Kelly & D. Perkins (Eds.), Handbook of implementation science for pyschology in education (pp. 13–34). New York, United State: Cambrige University Press.

Cataldo, R., Liberatore, G., & Hermosilla, A. M. (2018). Aplicacion de la Ley 26.657 de Salud Mental: una mirada desde la formacion de psicologos en la Universidad Nacional de Mar del Plata. Interdisciplinaria, 35(2), 277-290. https://doi.org/10.16888/interd.2018.35.2.2

Corcoran, T., Rowling, L., & Wise, M. (2015). The potential contribution of Intermediary Organizations for implementation of school mental health. Advances in School Mental Health Promotion, 8(2), 57–70. https://doi.org/10.1080/1754730X.2015.1019688

Durlak, J. A. (2016). Programme implementation in social and emotional learning: basic issues and research findings. Cambridge Journal of Education, 3577(April), 1–13. https://doi.org/10.1080/0305764X.2016.1142504

Durlak, J. A., Weissberg, R. P., Dymnicki, A. B., Taylor, R. D., & Schellinger, K. B. (2011). The Impact of Enhancing Students’ Social and Emotional Learning: A Meta-Analysis of School-Based Universal Interventions. Child Development, 82(1), 405–432. https://doi.org/10.1111/j.1467-8624.2010.01564.x

Elacqua, G., & Santos, H. (2013). Preferencias reveladas de los proveedores de educación privada en Chile: el caso de la Ley de Subvención Escolar Preferencial. Gestión y política pública, 22(1), 85-129. http://www.gestionypoliticapublica.cide.edu/?page_id=390

Fazel, M., Patel, V., Thomas, S., & Tol, W. (2014). Mental health interventions in schools in low-income and middle-income countries. The Lancet Psychiatry, 1(5), 388–398. https://doi.org/10.1016/S2215-0366(14)70357-8

Forman, S. G., Olin, Æ. S. S., Eaton, K., Crowe, M., & Saka, N. (2009). Evidence-Based Interventions in Schools : Developers’ Views of Implementation Barriers and Facilitators. School Mental Health, 1, 26–36. https://doi.org/10.1007/s12310-008-9002-5

JUNAEB (2018). Balance de Gestión Integral 2018. Santiago de Chile: JUNAEB, Ministerio de Educación.

Garrido, M. M., Kelley, A. S., Paris, J., Roza, K., Meier, D. E., Morrison, R. S., & Aldridge, M. D. (2014). Methods for constructing and assessing propensity scores. Health Services Research, 49(5), 1701–1720. https://doi.org/10.1111/1475-6773.12182

George, M., Guzmán, J., Flotts, M., Squicciarini, A. M., & Guzmán, M. P. (2012). Salud mental en escuelas vulnerables : evaluación del componente promocional de un programa nacional. Revista de Psicología, 21, 55–81. https://doi.org/10.5354/0719-0581.2012.25837

Guzmán, J., Kessler, R. C., Squicciarini, A. M., George, M., Baer, L., Canenguez, K. M., … Murphy, J. M. (2015). Evidence for the effectiveness of a national school-based mental health program in Chile. Journal of the American Academy of Child and Adolescent Psychiatry, 54(10), 799–807. https://doi.org/10.1016/j.jaac.2015.07.005

Hoagwood, K., Olin, S., Kerker, B., Kratochwill, T. R., Crowe, M., & Saka, N. (2007). Empirically Based School Mental Health Interventions Targeted at Academic and Mental Health Functioning. Journal of Emotional and Behavioral Disorders, 15(2), 66–92. https://doi.org/10.1177/10634266070150020301

Horwath, J., & Morrison, T. (2007). Collaboration, integration and change in children’s services: Critical issues and key ingredients. Child Abuse and Neglect, 31(1), 55–69. https://doi.org/10.1016/j.chiabu.2006.01.007

Knofczynski, G. T., & Mundfrom, D. (2008). Sample sizes when using multiple linear regression for prediction. Educational and Psychological Measurement, 68(3), 431–442. https://doi.org/10.1177/0013164407310131

Leiva, L., George, M., Antivilo, A., Simonsohn, A., Antivilo, A., Squicciarini, A. M., … Guzman, J. (2015). Salud mental escolar: Logros de una intervención preventiva en salud mental en niños y niñas del primer ciclo de enseñanza básica. Psicoperspectivas, 14, 31–41. https://doi.org/10.5027/psicoperspectivas-Vol14-Issue1-fulltext-508

Leiva, L., Rojas-Andrade, R., & Gonzalez, L. (2018). Explorando la implementación del programa Habilidades para la Vida: ¿Qué destacan los equipos ejecutores? In F. Peña (Ed.), Apoyando el Bienestar en las Comunidades Educativas (pp. 83–94). Santiago de Chile: JUNAEB, Ministerio de Educación.

Leiva, L., Zavala‐Villalón, G., Antivilo‐Bruna, A., Torres, B., & Ganga‐León, C. (2020). Implementation of a national mental health intervention in educational communities: What do successful teams do differently?. Journal of Community Psychology, 1-19. https://doi.org/10.1002/jcop.22370

Lopéz, V., Carrasco, C., Morales, M., & Ayala del Castillo, Á. (2011). El encapsulamiento de los psicologos escolares y profesionales de apoyo psicosocial en las escuelas. A. Revista Internacional Magisterio, 53(1).

Lyon, A. R., Whitaker, K., Locke, J., Cook, C. R., King, K. M., Duong, M., … Aarons, G. A. (2018). The impact of inter-organizational alignment (IOA) on implementation outcomes: Evaluating unique and shared organizational influences in education sector mental health. Implementation Science, 13(1). https://doi.org/10.1186/s13012-018-0721-1

Mellin, E. A., & Weist, M. (2011). Exploring School Mental Health Collaboration in an Urban Community: A Social Capital Perspective. School Mental Health, 3(2), 81–92. https://doi.org/10.1007/s12310-011-9049-6

Murphy, J. M., Abel, M. R., Hoover, S., Jellinek, M., & Fazel, M. (2017). Scope, Scale, and Dose of the World’s Largest School-Based Mental Health Programs. Harvard Review of Psychiatry, 1. https://doi.org/10.1097/HRP.0000000000000149

Powers, J. D., Edwards, J. D., Blackman, K. F., & Wegmann, K. M. (2013). Key Elements of a Successful Multi-System Collaboration for School-Based Mental Health: In-Depth Interviews with District and Agency Administrators. Urban Review, 45(5), 651–670. https://doi.org/10.1007/s11256-013-0239-4

Sandoval, J., Barriga, G., Wylie, C., Cuadra, D., Palacios, D., & Storey, R. (2018). Configuraciones subjetivas de psicólogos ante la vulnerabilidad escolar : el programa HPV. Revista de Psicología, 27(1), 1–13. https://doi.org/10.5354/0719-0581.2018.50757

Santiago, P., Benavides, F., Danielson, C., Goe, L., & Nusche, D. (2013). Teacher Evaluation in Chile 2013, OECD Reviews of Evaluation and Assessment in Education. OECD Publishing. https://doi.org/10.1787/9789264172616-en

Sarno, J., Lyon, A. R., Brandt, N. E., Warner, C. M., Nadeem, E., Spiel, C., & Wagner, M. (2013). Implementation Science in School Mental Health: Key Constructs in a Developing Research Agenda. School Mental Health, 6(2), 99–111. https://doi.org/10.1007/s12310-013-9115-3

Splett, J., Fowler, J., Weist, M., & McDaniel, H. (2013). The critical role of school psychology in the school mental health movement. Psychology in the Schools, 50(3), 245–258. https://doi.org/10.1002/pits.21677

Stephan, S., Connors, E. H., Arora, P., & Brey, L. (2013). A learning collaborative approach to training school-based health providers in evidence-based mental health treatment. Children and Youth Services Review, 35(12), 1970–1978. https://doi.org/10.1016/j.childyouth.2013.09.008

Stiffman, A. R., Stelk, W., Horwitz, S. M., Evans, M. E., Outlaw, F. H., & Atkins, M. (2010). A Public Health Approach to Children’s Mental Health Services: Possible Solutions to Current Service Inadequacies. Administration and Policy in Mental Health and Mental Health Services Research, 37(1–2), 120–124. https://doi.org/10.1007/s10488-009-0259-2

Suldo, S. M., Gormley, M. J., DuPaul, G. J., & Anderson-Butcher, D. (2014). The Impact of School Mental Health on Student and School-Level Academic Outcomes: Current Status of the Research and Future Directions. School Mental Health, 6(2), 84–98. https://doi.org/10.1007/s12310-013-9116-2

Thielking, M., Skues, J., & Le, V. A. (2018). Collaborative Practices among Australian School Psychologists, Guidance Officers and School Counsellors: Important Lessons for School Psychological Practice. Educational and Developmental Psychologist, 35(1), 18–35. https://doi.org/10.1017/edp.2018.4

Tooher, R., Collins, J., Braunack-Mayer, A., Burgess, T., Skinner, S. R., O’Keefe, M., … Marshall, H. S. (2017). Intersectoral collaboration to implement schoolbased health programmes: Australian perspectives. Health Promotion International, 32(2), 312–321. https://doi.org/10.1093/heapro/dav120

Vanderlind, R. (2017). Effects of Mental Health on Student Learning. Learning Assistance Review, 22(2), 39–58.

Vargas, B., & Peña, F. (2016). Orientaciones técnico metodologícas. Talleres Preventivos Habilidades para la Vida I 2016. Santiago de Chile: JUNAEB.

Weist, M., Bruns, E. J., Whitaker, K., Wei, Y., Kutcher, S., Larsen, T., … Short, K. H. (2017). School mental health promotion and intervention: Experiences from four nations. School Psychology International, 38(4), 343–362. https://doi.org/10.1177/0143034317695379

Notes

HTML generado a partir de XML-JATS4R por