The complexity of many cases, such as public projects and programs, requires evaluation methods that acknowledge this complexity in order to facilitate learning. On the one hand, the complexity of the cases derives from their uniqueness and nested nature. On the other hand, there is a need to compare cases in such a way that lessons can be transferred to other (future) cases in a coherent and non-anecdotal way. One method that is making headway as a complexity-sensitive, comparative method is Qualitative Comparative Analysis (QCA). This contribution aims to explain to what extent QCA is complexity-informed, to show how it can be deployed as such, and to identify its strengths and weaknesses as an evaluation method. We will discuss the properties of complexity, provide an overview of evaluation literature about QCA, and present a simplified step-wise guide for utilizing QCA in evaluation studies.
While some computational models of intelligence test problems were proposed throughout the second half of the XXth century, in the first years of the XXIst century we have seen an increasing number of computer systems being able to score well on particular intelligence test tasks. However, despite this increasing trend there has been no general account of all these works in terms of how they relate to each other and what their real achievements are. Also, there is poor understanding about what intelligence tests measure in machines, whether they are useful to evaluate AI systems, whether they are really challenging problems, and whether they are useful to understand (human) intelligence. In this paper, we provide some insight on these issues, in the form of nine specific questions, by giving a comprehensive account of about thirty computer models, from the 1960s to nowadays, and their relationships, focussing on the range of intelligence test tasks they address, the purpose of the models, how general or specialised these models are, the AI techniques they use in each case, their comparison with human performance, and their evaluation of item difficulty. As a conclusion, these tests and the computer models attempting them show that AI is still lacking general techniques to deal with a variety of problems at the same time. Nonetheless, a renewed attention on these problems and a more careful understanding of what intelligence tests offer for AI may help build new bridges between psychometrics, cognitive science, and AI; and may motivate new kinds of problem repositories.
We investigate the application of grammar inference to the analysis of facial expressions to discover underlying sequential regularities characteristic for a specific mental state. The input consists of sequences of action units (AUs), which represent basic facial signals. The typical classification task for facial expression analysis is to assign a set of AUs its corresponding mental state, e.g., an emotion. To our knowledge, there is no research investigating whether there is diagnostic information in the sequence in which the AUs occur in a given time interval. Our study is based on data of facial expressions of pain obtained in a psychological experiment with 347 pain episodes of 86 subjects represented as sequences of AUs. We applied the Alignment-Based Learning (ABL) approach to infer the underlying grammar for the set of all AUs which occurred in the sequences and for a reduced alphabet of the relevant AUs only. We used 10-fold cross-validation to estimate performance and we extended ABL with a frequency-based heuristics to reduce the number of grammar rules by eliminating such rules which do not contribute significantly to performance. The resulting grammar for the reduced AU alphabet provides a first approximation for a “grammar of pain”.
As a result of public discussions regarding Germany's standing on international rankings of student achievement, increased attention was focused on enhancing cognitive stimulation in preschools. There are some concerns that preschool curricula that focus more on cognitive stimulation rather than on socio-emotional skills might neglect the socio-emotional development of children. These concerns are rooted in the tradition of German preschools focussing more on the stimulation of social than of cognitive skills. This paper examines these claims by drawing on the German preschool programme ‘KiDZ’ in order to analyse the socio-emotional development (well-being, joy of learning, and worry) of children attending a preschool with a more academically oriented curriculum. This study also investigates effects of domain-specific preschool quality and if any effects of participation in KiDZ might be explained by the quality of the preschools. Positive effects of the children's participation in KiDZ are found and will be discussed.
Peer problems are common in children with special educational needs (SEN), but the reasons are poorly understood. This study aims to identify risk factors of peer problems (e.g., SEN, school setting, pro-social behaviour) for their occurrence. A subsample of 3900 children from the National Educational Panel Study in Germany was analysed. Children and parents answered the items of the Strengths and Difficulties Questionnaire (SDQ) subscales ‘peer problems’ and ‘pro-social behaviour’. Students with SEN (attending special schools or inclusive classes) were more likely to score within the abnormal range of the SDQ subscale peer problems than students without SEN. The results further show a low level of parent–child agreement on the subscale ‘peer problems’. Logistic regression analyses showed that having SEN is always an explaining variable for ‘peer problems’ and that group differences cannot be fully explained by gender, school setting or ‘pro-social behaviour’.
An understudied aspect for the successful completion of PPP infrastructure projects is the extent to which they are satisfactorily implemented. Studying PPP implementation is important though, because well-planned projects can fail if project implementation is inadequately managed. This article aims to find out which management and public–private cooperation approaches produce satisfaction for public procurers in the implementation phase of different kinds of infrastructure projects. To this purpose, twenty-seven Dutch road construction projects are systematically analyzed with fuzzy set qualitative comparative analysis (fsQCA). The results show four configurations that produce satisfaction. It is concluded that externally-oriented management, which is characterized by a stakeholder-oriented project implementation approach, and close public–private cooperation, where public and private partners work together closely and interactively, are important for achieving satisfaction. In less complex projects with narrower scopes, however, the partners may rely on less interactive forms of cooperation, more characterized by monitoring contract compliance.
A public-private partnership (PPP) is an organizational arrangement in which knowledge and resources are pooled in order to realize outcomes. Although PPPs have become common practice in spatial planning and development, there is a continuous search for their ideal organizational form and management. This is fueled by the often poor performance in terms of e.g. time delays and budget overruns. Whilst comparative studies have been conducted into the outcomes of certain organizational forms and management strategies, fewer comparative studies evaluate their combined effects. The goal of this study is to explore what configurations of certain organizational forms and management may produce good outcomes. This is done by conducting a fuzzy set qualitative comparative analysis (fsQCA) of survey data of 50 managers involved in urban regeneration companies (URCs) in the Netherlands.