Introduction
In the past decade, there has been a surge in worldwide research, public health concerns, and media alarm regarding hikikomori, an extreme and prolonged form of social withdrawal and isolation, which was recently added to the “Culture and Psychiatric Diagnosis” chapter in Section III of the DSM-5-TR 1. This decision is in line with Teo and Gaw’s 2 original suggestion that hikikomori could be viewed as a culture-bound syndrome, with the need for further international research to evaluate if it meets the established criteria for a new psychiatric disorder. In the DSM-5-TR, hikikomori is described as a syndrome of prolonged and severe social withdrawal with no in-person social interactions and no interest or willingness to attend school or work. Nevertheless, the scientific debate on the “position” of hikikomori in mental health is ongoing, and its validity and conceptualization as a separate diagnostic entity may depend on the criteria in use 3–9. A recent review of hikikomori definitions further informed this debate, clarifying the differentiation between hikikomori and social isolation due to other conditions 4. Indeed, hikikomori has been operationalized using different definitions in empirical investigations 7.
In line with prevalence findings from Japan (1.2-1.9%) 10–13 and other Asian countries (1.9-3.2%) 14–16 with some exceptions (6.7%) 17, a recent epidemiological analysis among the working-age (15-64 years) European population showed that the weighted prevalence of severe social isolation as a proxy condition for hikikomori was 1.77% (95% CI: 1.54, 2.01) in 2018-2020 whereas the prevalence was 0.27% (95% CI: 0.00, 0.54) in adolescents 18. According to the same study, the prevalence of hikikomori risk in Italy was 2.02% (95% CI: 1.28, 2.76).
Considering questionnaires’ cut-off for hikikomori screening
The attention on hikikomori and its potential consequences for public health has also increased in Italy during the last decade 19 as hikikomori frequently co-occurs with other mental health disorders 5,6,20 and suicidal risk 21. The Italian Chamber of Deputies has recently approved a motion committing the government to prevent and support hikikomori individuals and their families (XIX Legislature, Chamber of Deputies, 2023, 1-00160 and subsequent revisions). Indeed, initial findings from epidemiological studies showed that the prevalence of lifetime hikikomori and pre-hikikomori among students is 1.7% and 2.6%, respectively 22,23. Similarly, a more recent prevalence analysis using Italian data from the European Health Interview Survey demonstrated a weighted estimate of persons at risk of hikikomori ranging from 1% to 1.7% among individuals aged 15 to 75 years and over, corresponding to a weighted national count of 523 – 867 thousand persons 24. Weighted estimates in adolescents aged 15-19 years ranged from 1.3% to 2.2% for a weighted national count of 38 – 62 thousand adolescents 24. At the same time, other findings showed lower estimates, such as 0.2% of students who received a certificate of social withdrawal from local mental health departments according to information provided by the headmasters 22 and 0.03% of students who rarely left their homes and did not attend school during the academic year 2017–2018 in Emilia-Romagna 19,25. Differences in prevalence estimates might be primarily due to different hikikomori operationalizations 4. Consequently, the existence of trustworthy thresholds in hikikomori questionnaires may aid in pinpointing individuals susceptible to the condition, thereby enhancing screening processes and contributing to studies on its frequency as well as risk and protective factors.
Reliable cut-offs of hikikomori questionnaires for Italian populations are mainly lacking. Recently, Colledani et al. 26 selected a cut-off for the short version (15 items) of the Hikikomori Risk Inventory 27. However, that cut-off measures a tendency towards isolation in a sample of students without hikikomori. Thus, additional research is needed to clarify whether the authors’ cut-off is useful for identifying hikikomori. A further methodological limitation of the Hikikomori Risk Inventory 27 - with an impact on construct validity - is that the questionnaire investigates symptoms of other conditions (e.g., depression, anxiety) considered risk factors for hikikomori rather than focusing specifically on the main symptoms of hikikomori (i.e., physical isolation, difficulty in socialization, and poor emotional support) 28. In fact, among others, definitions of hikikomori require physical isolation at home. Consequently, the investigation of hikikomori with samples of secondary school students entails a methodological consideration. In the case of hikikomori individuals, it is unlikely that they would be present at school during the questionnaire administration, and as such, would not be included in the studied sample. This methodological difficulty can be addressed by using online surveys 29,30 or studying lifetime episodes of hikikomori 22,23,31. This aspect needs to be considered when interpreting study results to avoid confusion about understanding the phenomenon.
Recently, severe social withdrawal symptoms were investigated using a different assessment tool. In 2019, a dedicated Working Group was established by the Italian Society of Childhood and Adolescent Neuropsychiatry (SINPIA) to explore the phenomenon of severe social withdrawal in children and adolescents 32–34. The Group investigated the views held by neuropsychiatrists concerning severe social isolation and gathered details about any patients they may have treated for this condition 34. The findings led to the creation of the ERRESSEGI questionnaire investigating symptoms of severe social withdrawal (ritiro sociale grave) which consists of 35 items selected from an initial pool of 55 items 35.
The objectives of this study
Hikikomori can be classified according to different research criteria proposals 2,4,36,37. Some well-designed instruments that may guide the evaluation of hikikomori have been published, such as the self-report 25-item Hikikomori Questionnaire (HQ-25) 28 and the recent structured diagnostic interview called Hikikomori Diagnostic Evaluation (HiDE) 38. The Italian versions of the HQ-25 have previously been adapted and validated, confirming the good psychometric properties of the questionnaire 29,31. However, the ability of the Italian version of the HQ-25 to differentiate between individuals with and without hikikomori (criterion validity) has not been tested, i.e. evidence on the optimal cut-off with Italian samples is lacking. Similarly, the optimal cut-off of the ERRESSEGI for detecting hikikomori has not yet been tested. Therefore, the first aim of the present study was to investigate the optimal cut-offs of the Italian version of the HQ-25 and of the ERRESSEGI with clinical and non-clinical samples of adolescents. Identifying the trustworthy thresholds of questionnaires exploring hikikomori symptoms may help to identify adolescents at high risk for the condition and advance current knowledge on its frequency and predisposing factors.
The two questionnaires originate from distinct cultural backgrounds and present differences in item development and selection. Specifically, the HQ-25 emphasizes social functioning and physical isolation with items selected by American and Japanese professionals with expertise in clinical psychology, psychiatry, and hikikomori. The ERRESSEGI also investigates additional symptoms considered characteristic of severe social withdrawal according to the opinions of a sample of Italian neuropsychiatrists, like sleep alterations, internet use, eating behaviors, somatic symptoms, and conflicts within the family. As such, we recognized the necessity to provide support on the construct validity of the ERRESSEGI, which may have implications for subsequent hikikomori research with adolescents seeking help from neuropsychiatric centers. Therefore, our second aim was to test whether the HQ-25 and the ERRESSEGI, despite different cultural origins, converged in their identification of hikikomori.
Method
Sample and procedure
The present study uses data collected to develop the ERRESSEGI questionnaire 35 described below. Participants from the two clinical samples, with and without hikikomori, were recruited from neuropsychiatric centers within the Public National Health Service (NHS). The clinical sample without hikikomori sought help from three neuropsychiatric services in the provinces of Grosseto, Lodi and Barletta-Andria-Trani, respectively. Adolescents from the clinical sample with hikikomori were referred to four neuropsychiatric services in the provinces of Roma, Torino, Grosseto and Barletta-Andria-Trani. Inclusion criteria included the sufficient ability to understand the Italian language and, for the non-clinical sample only, the absence of mental disorders and disabilities confirmed by health professionals. Exclusion criteria applied to the clinical groups’ membership included the presence of intellectual disability and autistic disorder 39.
Data collection in participating schools was preceded by the illustration of the aims of the study to the headmasters and teachers of each school and their approval to participate. Informed consent was then obtained from participants and their parents, and subsequently, questionnaires were filled out collectively in the classroom after providing brief instructions. For the clinical samples, neuropsychiatrists administered questionnaires individually and subsequently, the diagnostic evaluation. Anonymity of participants was ensured by assigning each participant a unique code.
Convenience sampling was employed to select neuropsychiatric centers within the NHS and secondary schools to propose participation in this study. Data collection occurred between October 2022 and June 2023. The initial sample size considered for the present analysis was N = 412, which consisted of 294 students attending secondary schools (provinces of Grosseto, Lodi, and Barletta-Andria-Trani) representing the non-clinical sample, 76 adolescents with mental health disorders without hikikomori, and 42 adolescents with mental health disorders and hikikomori according to the definition proposed by Kato et al. 36,37.
The study was approved by the Ethics Committee of the “Azienda USL Toscana Sud-Est” (i.e., Local Health Authority from Toscana South-East, Toscana Health Service) (Prot/Staff 2022/000120, Arezzo 1-6-2022).
Measures
Hikikomori Questionnaire – 25
The 25-item Hikikomori Questionnaire (HQ-25) 28,31 is a self-report questionnaire for the evaluation of the severity of symptoms of hikikomori during the previous 6 months. Typical psychological features and behavioral patterns of hikikomori examined include difficulties and problems in socialization, isolation, poor emotional support, and a sense of alienation from society. Participants respond on a 5-point Likert scale (from 0 = “strongly disagree” to 4 = “strongly agree”). The HQ-25 has a score range of 0 – 100, with higher values indicating higher symptomatology. Teo et al. 28 established a threshold score of 42 for identifying hikikomori among individuals aged 15 to 50 in Japan. To date, there has been no other investigation on the HQ-25 cut-off scores for versions other than the English and Japanese ones created by the original researchers.
ERRESSEGI
The term ERRESSEGI refers to the Italian pronunciation of the initial letters of “Ritiro Sociale Grave”, corresponding to the Italian translation of severe social withdrawal. Item selection for the questionnaire was based on a previous study examining Italian neuropsychiatrists’ opinions about severe social withdrawal and information from clinical cases of adolescents referred to the Italian Child and Adolescent Neuropsychiatry Services due to a condition of severe social withdrawal 33–35. The questionnaire can be used with adolescents aged 13 to 18 and consists of 35 items exploring different symptoms of severe social withdrawal – according to the opinions of a sample of Italian neuropsychiatrists - during the last 6 months such as physical isolation, school attendance, sleep alterations, depressive and anxiety symptoms, emotional support, eating behaviors, somatic symptoms, internet use, conflicts within the family, self-injury and suicidal ideation. Each item is scored on a three-point Likert scale according to the degree of symptom severity (0 = symptom absent; 1 = symptom present but mild; and 2 = moderate/severe symptomatology). Total scores can range from 0 to 70, with high scores indicating high symptoms of severe social withdrawal. It has been suggested that a T-score of 70, corresponding to a raw score of 29, may indicate a condition at increased risk for hikikomori 35.
Statistical analysis
Data from 31 respondents were excluded due to the following reasons: age lower than 12 years (n = 6), missing values on the HQ-25 or ERRESSEGI (n = 23), and inadmissible ERRESSEGI item values due to errors in item coding (n = 2). Therefore, the final sample size for subsequent analysis was N = 381, specifically, the control sample was n = 274, the clinical sample without hikikomori was n = 70, and the clinical sample with hikikomori was n = 37. Seven participants from the clinical sample without hikikomori showed psychological distress and were considered at risk for subsequent mental health disorders (ICD-10 codes = Z00-Z99 Factors influencing health status and contact with health services, and H90.5 Unspecified sensorineural hearing loss). Considering the questionnaires’ focus on the identification of high risk for and presence of hikikomori, data from the non-clinical and clinical groups without hikikomori were considered as a whole when analyzing optimal cut-offs. Therefore, to establish optimal cut-offs for the Italian version of the HQ-25 for adolescents and the ERRESSEGI, the presence of hikikomori according to the definition proposed by Kato et al. 36,37 was used as a criterion.
Initially, descriptive statistics (means or frequency, standard deviations, minimum and maximum values, skewness, and kurtosis) were calculated to examine the characteristics of the sample. A latent correlation analysis between the questionnaire scores was performed to determine whether they measured related constructs.
Information regarding the diagnosis of participants from the clinical and hikikomori groups and statistical analysis results addressing reliability, measurement invariance across sexes, and the relationship between HQ-25 subscales scores and the ERRESSEGI total score - aspects beyond the scope of this article - were included in the Supplementary Materials.
Optimal cut-offs were estimated using the “inner” bootstrap resampling procedure with 500 repetitions based on the metric of maximizing the sum of sensitivity and specificity, i.e., corresponding to the minimization of the sum of false negative and false positive misclassification likelihoods 40. The “inner” bootstrapping for cut-off estimation requires drawing several bootstrap samples from the original sample, with the final result being the average of all models that were fit to the bootstrap samples 41. The “outer” bootstrap resampling procedure with 500 repetitions was applied to evaluate optimal cut-off out-of-sample performance - to be conservative - determining optimal cut-off and estimates variability. Out-of-bag estimates (oob) for optimal cut-off confidence intervals (CI) and measures of classification accuracy (area under the receiver operating characteristic (ROC) curve (AUC), sensitivity, specificity) were considered and interpreted according to previous recommendations 42. This procedure provides information on the performance that the optimal cut-off would have in a hypothetically different sample. Moreover, positive and negative likelihood ratios (PLR and NLR, respectively) were calculated to assess the predictive properties of the questionnaires 42.
Finally, the two ROC curves obtained, resulting in optimal cut-offs for the Italian version of the HQ-25 for adolescents and the ERRESSEGI, were compared using a bootstrap method test with 1,000 repetitions for a difference in the AUC.
As part of a sensitivity analysis, optimal cut-offs and AUC were investigated by comparing non-clinical and hikikomori groups (Supplementary Materials pp. 6-9). Moreover, to provide additional information for questionnaire use in clinical settings, analyses were repeated including data from the two clinical groups only, with and without hikikomori (Supplementary Materials pp. 10-13). As suggested during the review process, we also tested the independent discriminative power of each questionnaire using multivariable logistic regression models, including age, sex, depressive and anxiety symptoms as covariates (Supplementary Materials pp. 14-16). “Cutpointr” 41 and “pROC” 43 packages were used to conduct statistical analysis in R 44.
Results
Sample characteristics and correlation analysis
Table I reports descriptive characteristics by group. Higher HQ-25 and ERRESSEGI scores were found in the hikikomori group, followed by the clinical and non-clinical groups.
Latent correlation between unique factors of the HQ-25 and ERRESSEGI showed a standardized estimate of the association of 0.858 (Chi-square (1709) = 5542.641, p < 0.001, CFI = 0.95, TLI = 0.95, RMSEA = 0.077, 90% C.I. for RMSEA: 0.075; 0.079; SRMR = 0.089).
Optimal cut-offs
Table II reports the results of the optimal cut-off (inner) bootstrap resampling analysis, differentiating the group without hikikomori (i.e., non-clinical and clinical groups) from the clinical hikikomori group, and estimates for measures of classification accuracy. The optimal cut-off for the HQ-25 was 52, while for the ERRESSEGI the optimal cut-off was 30. AUC and accuracy estimates indicated good/moderate accuracy of the questionnaires optimal cut-off in differentiating adolescents without and with hikikomori (see also Fig. 1 and Fig. 2). Likelihood ratio values suggested questionnaires’ potential in providing useful additional information to aid clinical decisions 42.
Out-of-sample performance
Table III shows the results of the (outer) bootstrap (n = 500) analysis exploring optimal cut-off and estimates variability in differentiating the group without hikikomori (i.e., non-clinical and clinical groups) from the clinical hikikomori group. Optimal cut-off intervals resulting from the (outer) bootstrap analysis included the optimal cut-off selected in the (inner) bootstrap analysis. Furthermore, estimates of classification accuracy demonstrated good stability.
ROC curves comparison
The comparison of two ROC curves, resulting in optimal cut-offs of 52 for the HQ-25 and 30 for the ERRESSEGI, found no significant difference in AUC values (D = 1.2395, n bootstrap = 1000, p-value = 0.2152).
Discussion
In this study, we analyzed the optimal cut-off of the Italian version of the HQ-25 and the new ERRESSEGI questionnaire to differentiate between individuals with and without hikikomori. The findings showed that the optimal cut-off score for the HQ-25 was 52, while for the ERRESSEGI the optimal cut-off was 30. That is, adolescents with HQ-25 scores of 52 and higher, or ERRESSEGI scores of 30 and higher, are at increased risk for hikikomori, and additional clinical evaluation (e.g., clinical interview) may be deemed necessary. Results of the sensitivity analysis further supported the results from the main analysis. Importantly, the bootstrap analysis allowed us to form expectations about the performance of unseen data 41 whose results pointed to the good stability of the in-sample results.
Moreover, to provide useful information about the use of the two questionnaires in clinical settings, we analyzed the optimal cut-off to identify hikikomori in clinical samples, i.e., discriminating between clinical adolescents without hikikomori from those with hikikomori (Supplementary Material, pp.10-13). The findings showed a reasonable increase in optimal cut-off scores, with the optimal cut-off being 58 for the HQ-25 and 33 for the ERRESSEGI.
In light of the above finding, we provide further evidence of the validity of the Italian version of the widely used HQ-25, expanding the test to criterion validity 31, and support on the construct and criterion validity of the ERRESSEGI questionnaire 35. In particular, criterion validity was confirmed by testing how well the two questionnaires predicted hikikomori according to the definition proposed by Kato et al. 36,37, here used as a criterion or gold standard measurement. As such, we filled in a gap in previous literature.
An additional aim of this study was to compare the HQ-25 and the ERRESSEGI for identifying hikikomori. We found no significant difference in AUC values when comparing the two ROC curves. This indicates that the two questionnaires do not differ in their ability “to capture” hikikomori with samples of adolescents, at least in the considered sample. Both questionnaires represent effective screening measures of hikikomori with moderate to high accuracy 42. Consequently, this supports the soundness of ERRESSEGI items selection derived from Italian neuropsychiatrists’ opinions about severe social withdrawal and questionnaire construction overall because it aligns with the widely used HQ-25 45–48, whose items were selected by American and Japanese clinical psychologists and psychiatrists with expertise in hikikomori 28.
To note, considering the previous lack of reliable questionnaire cut-offs to identify hikikomori in adolescents from Western countries, our findings encourage mental health professionals to use the HQ-25 and the ERRESSEGI in the screening for hikikomori risk. These findings represent a valuable contribution to the hikikomori scientific literature. In the context of identifying hikikomori in Western adolescents, it may be advantageous to utilize a more elevated cut-off for the HQ-25 compared to the findings reported in the initial study 28. However, when there is a lack of evidence concerning the optimal cut-off for the studied population and additional clinical evaluation is not feasible, the use of the continuous score might be preferred. In this study, both the HQ-25 and ERRESSEGI showed moderate to high accuracy in detecting hikikomori. This has implications for professionals in need of tailored measures.
Because hikikomori is relatively uncommon in the general population (1-2%), when assessing for hikikomori in the general population, selecting a questionnaire with high cut-off specificity is preferable to reduce false positives 42. Therefore, the availability of reliable optimal cut-off for the HQ-25 and ERRESSEGI may advance the study of hikikomori and minimize false positives. For example, in studies aiming at estimating prevalence and examining risk and protective factors within community samples, the use of the questionnaires’ optimal cut-off would be especially valuable because the chance of studying the condition expected to be studied would be increased. Furthermore, the likelihood ratio values suggested the questionnaires’ potential to provide useful additional information in order to make clinical decisions 42. Notably, when the aim was to differentiate non-clinical adolescents from hikikomori, the questionnaires seemed to show the potential to alter clinical decisions (Supplementary Material, pp.6-9). Conversely, the likelihood ratio values suggested that the use of the questionnaires could rarely alter clinical decisions in differentiating clinical patients with and without hikikomori (Supplementary Material, pp.10-13). Rather than indicating a limitation of the questionnaires, the latter finding may also be related to the specific hikikomori definition used in this study. The possibility that some hikikomori conceptualizations represent severe social isolation as a possible consequence of mental disorders has been recently discussed 4. However, providing useful information as part of the first screening for hikikomori, the HQ-25 and ERRESSEGI might be used to inform decisions in clinical settings. Indeed, the questionnaires’ cut-offs showed independent discriminatory power beyond the effects explained by sex, age, affective and anxiety problems (Supplementary Material, pp.14-16). Finally, the two questionnaires can be used in online surveys as the high specificity values of their optimal cut-off limit false positive results.
Study strengths and limitations
In reporting and discussing our results, we followed previous recommendations and considered different measures of test accuracy according to specific situations 42,49,50. Estimates of accuracy, sensitivity, specificity, PLR, and NLR were used to describe the technical characteristics of the HQ-25 and ERRESSEGI in correctly detecting adolescents with hikikomori, providing valuable information for screening populations for public health purposes and clinical decisions. Positive and negative predictive values and ratios are most important for clinical settings when the diagnosis is unknown or the test is used to support the diagnosis, providing estimates of the percentage of patients diagnosed correctly. However, we preferred to avoid calculating predictive values/ratios because these depend on disease prevalence in the specific sample analyzed and, as such, are difficult to interpret when the observed prevalence differs from the “true” prevalence found in representative samples or epidemiological studies 42,50. In our study, participants with hikikomori were 9.7% of the sample, and this prevalence was higher compared to the estimates of 1-2% in epidemiological studies on hikikomori 18. Therefore, future research with hikikomori in-study prevalence similar to that observed in the general population may provide information about the questionnaires’ positive and negative predictive values and ratios.
An additional limitation that needs to be considered is related to the specific sample examined, which may affect the generalizability of our findings. All participants with hikikomori received at least another diagnosis of a mental health disorder (mainly mood disorders and anxiety, dissociative, stress-related, somatoform, and other nonpsychotic mental disorders). Considering that information on the temporal association between the diagnoses in comorbidity was lacking, we cannot draw a firm conclusion on whether we included primary hikikomori whose symptoms caused other mental health disorders, or whether we analyzed a condition of extreme social isolation that emerged as a consequence of another mental health disorder. Therefore, our analysis could not distinguish between primary hikikomori and social withdrawal secondary to another disorder. Considering that a moderate proportion of ERRESSEGI items do not examine physical isolation and difficulties in social participation, it remains to be tested whether the psychometric performance of the ERRESSEGI was related to the specific sample involved in this study, i.e., psychiatric patients with hikikomori. Therefore, we can suggest the suitability of the HQ-25 and ERRESSEGI for the assessment of secondary hikikomori (i.e., if other mental disorders are suspected in comorbidity), but we encourage the use of the HQ-25 for primary hikikomori (i.e., if no other mental disorders are suspected in comorbidity). A flow-chart providing practical guidance to mental health professionals on the use of the two questionnaires is displayed in Figure 3.
Conclusions
We found that specific cut-offs for the HQ-25 and ERRESSEGI can be used to identify adolescents with a high risk of hikikomori. The optimal cut-off of the HQ-25 was 52, while the ERRESSEGI optimal cut-off was 30. We suggest using them for the identification of hikikomori with community samples and in online surveys. For use in clinical settings, the HQ-25 optimal cut-off of 58 and the ERRESSEGI optimal cut-off score of 33 may help in discriminating between patients with and without hikikomori.
Acknowledgments
This research used data previously presented in the Italian manual of the ERRESSEGI questionnaire by Camuffo and colleagues 35. We thank Dr. Vincenzina Ancona (MD, Early Intervention in Developmental Age Operational Unit, Roma 2 Local Health Authority, Rome), Dr. Elena Rainò (MD, Department of Pathology and Child Care, Regina Margherita Children’s Hospital, Turin), and Dr. Gabriella Rosso (MD, Torino 3 Local Health Authority, Turin) for their valuable work on sample recruitment and data collection.
Conflict of interest statement
SA, MC, CA, DC, OF, MS, and PV co-authored the Italian manual of the ERRESSEGI questionnaire published by Hogrefe 35. The proceeds - derived from the sale of the work - will be entirely donated to the Società Italiana di Neuropsichiatria dell’Infanzia e dell’Adolescenza (SINPIA; Italian Society of Neuropsychiatry of Childhood and Adolescence) to guarantee adequate dissemination of the work. SA, MC, CA, DC, OF, MS, and PV report no other conflicts of interest. RC and AT report no conflicts of interest. This material is the result of work partially supported with resources and the use of facilities at the VA Portland Health Care System. The contents do not represent the views of the U.S. Department of Veterans Affairs or the United States Government.
Ethical consideration
Informed consent was obtained from all participants included in the study. This study was approved by the Ethics Committee of the “Azienda USL Toscana Sud-Est” (i.e., Local Health Authority from Toscana South-East, Toscana Health Service) (Prot/Staff 2022/000120, Arezzo 1-6-2022).
Authors contribution
Amendola Simone: conceptualization, data curation, formal analysis, visualization, writing – original draft, writing – review & editing. Rita Cerutti: conceptualization, visualization, writing – original draft, writing – review & editing. Mauro Camuffo: conceptualization, methodology, data curation, investigation, visualization, writing – original draft, project administration. Cira I. Arnone: investigation, visualization, writing – review & editing. Daniela Candeloro: investigation, visualization, writing – review & editing. Oliviero Fuzzi: investigation, visualization, writing – review & editing. Maria Serra: investigation, visualization, writing – review & editing. Paola Vizziello: investigation, visualization, writing – review & editing. Alan R. Teo: conceptualization, supervision, visualization, writing – original draft, writing – review & editing.
Open Data
The authors are willing to share their data and research materials with other researchers. The material will be available upon request.
Declaration of Generative AI and AI-assisted technologies in the writing process
No generative AI technology was used to write this manuscript.
History
Published online: December 30, 2025
Figures and tables
FIGURE 1. The distribution of the predictor values per class (top left), ROC curve (top right), bootstrapped cutpoint variability (bottom left), and distribution of the out-of-bag metric values (bottom right) for the (A) HQ-25 (subfigure above) and (B) ERRESSEGI (subfigure on the following page).
FIGURE 2. Values of the metric function (i.e., maximization of the sum of sensitivity and specificity) per cutpoint with 95% bootstrap confidence interval for the HQ-25 (A) and ERRESSEGI (B).
FIGURE 3. Flow-chart providing practical guidance to mental health professionals on when to administer the two questionnaires, how to interpret their scores, and how to act on elevated scores
| Total sample | Non-clinical | Clinical | Without hikikomori (non-clinical and clinical groups) | Hikikomori | |
|---|---|---|---|---|---|
| N | 381 | 274 | 70 | 344 | 37 |
| Age, M (SD) | 15.47 (1.65) | 15.46 (1.72) | 15.63 (1.5) | 15.5 (1.68) | 15.19 (1.37) |
| Female (%) | 48.16 | 41.24 | 68.12 | 46.65 | 62.16 |
| HQ-25 | |||||
| M (SD) | 35.73 (18.2) | 31.88 (14.65) | 37.33 (19.02) | 32.99 (15.76) | 61.22 (19.69) |
| Min. - Max. | 0-86 | 0-85 | 0-76 | 0-85 | 9-86 |
| Skewness | 0.44 | 0.11 | 0.04 | 0.18 | -0.71 |
| Kurtosis | -0.06 | -0.18 | -0.95 | -0.29 | -0.43 |
| ERRESSEGI | |||||
| M (SD) | 18.88 (11.16) | 15.51 (8.49) | 22.74 (10.48) | 16.99 (9.38) | 36.46 (11.15) |
| Min. - Max. | 1-56 | 1-47 | 3-42 | 1-47 | 11-56 |
| Skewness | 0.8 | 0.85 | -0.18 | 0.65 | -0.52 |
| Kurtosis | 0.09 | 0.64 | -1.08 | -0.19 | -0.31 |
| AUC | Accuracy | Optimal cut-off | Sensitivity | Specificity | PLR | NLR | |
|---|---|---|---|---|---|---|---|
| HQ-25 | 0.86 | 0.87 | 51.76 | 0.73 | 0.88 | 6.08 | 0.31 |
| ERRESSEGI | 0.90 | 0.86 | 29.71 | 0.81 | 0.87 | 6.23 | 0.22 |
| AUC: area under the curve, PLR: positive likelihood ratio, NLR: negative likelihood ratio. | |||||||
| AUC [oob](95% CI) | Accuracy [oob](95% CI) | Optimal cut-off [ib](95% CI) | Sensitivity [oob](95% CI) | Specificity [oob](95% CI) | PLR [oob] | NLR [oob] | |
|---|---|---|---|---|---|---|---|
| HQ-25 | 0.86(0.76-0.95) | 0.86(0.79-0.93) | 52.78 (47.3-61.7) | 0.66(0.41-0.92) | 0.88(0.79-0.97) | 5.5 | 0.39 |
| ERRESSEGI | 0.90(0.82-0.97) | 0.87(0.82-0.92) | 30.16 (27.3-33.4) | 0.77(0.5-0.1) | 0.88(0.82-0.95) | 6.42 | 0.26 |
| AUC: area under the curve, oob: out-of-bag estimate, ib: in-bag estimate, PLR: positive likelihood ratio, NLR: negative likelihood ratio. | |||||||
