Southampton Mindfulness Questionnaire: Confirmatory Factor Analysis and Psychometric Properties Across Portuguese Clinical and Non-clinical Samples

Southampton Mindfulness Questionnaire (SMQ) is one of the available instruments to assess mindfulness and has been described as particularly suited for clinical populations. The original study of the SMQ conceptualized it as having four facets, but yielded a single factor structure. The aim of this study was to confirm the SMQ factor structure and examine its psychometric properties in clinical and non-clinical Portuguese samples. The community sample comprised 243 participants, ranging from 18 to 77 years. Additionally, to assess sensitivity, two distinct samples were used: individuals with meditation experience (n = 30) and a clinical sample (43 individuals with psychosis). Factor structure, reliability, validity, and sensitivity analyses were performed. Confirmatory factor analyses of the SMQ seem to support one overarching mindfulness construct containing four facets, revealing an acceptable fit through a parceling method. Significant correlations were obtained between SMQ and another mindfulness measure, and also with measures of cognitive fusion, and positive and negative affect. Results also indicated significant differences in SMQ non-judging facet between meditators and non-meditators and clinical samples. The results indicate that although the SMQ has shown adequate reliability and validity, some limitations to the measure have been found. Further studies are needed, particularly with clinical samples, to determine how the current SMQ version could be improved and whether it should be revised.

Congruent with this lack of consensus and despite the criticisms regarding the validity of self-reported mindfulness measures (e.g., Van Dam et al., 2018), several self-report scales are aiming at measuring mindfulness. These scales attempt to measure mindfulness as unidimensional or multidimensional, both as a state, such as the Toronto Mindfulness Scale (TMS; Lau et al., 2006), or as a trait, such as the Five Facet Mindfulness Questionnaire (FFMQ; Baer et al., 2006). Reviews of their conceptual frameworks, psychometric properties, advantages, and disadvantages are available (Baer, 2011;Baer et al., 2006;Bergomi et al., 2013;Sauer et al., 2013) and have described them as psychometrically sound, with common theoretical ground though assessing distinguishable aspects (Bergomi et al., 2012).
The SMQ is a 16-item questionnaire aiming at specifically assessing the relationship people tend to establish with distressing thoughts and images (a version for voices also exists, though not psychometrically studied -Southampton Mindfulness of Voices Questionnaire). Given this focus on distressing inner experience (e.g., thoughts and images), a relevant phenomenon in all mental health difficulties and therapeutic target in most cognitive-based therapeutic interventions, SMQ may be particularly suited to use with clinical populations (Baer, 2011;Bergomi et al., 2013;Chadwick et al., 2008;Sauer et al., 2013). Moreover, it has been used and evaluated using samples of people with severe mental illness (e.g., psychosis) which is not common in mindfulness questionnaires.
In terms of theoretical background, SMQ was developed within a framework rooted in a multidimensional view of trait mindfulness, comprising four complementary bipolar constructs: (1) decentered awareness of cognitions as mental events versus being lost in reaction; (2) allowing attention to remain with difficult experience versus experiential avoidance; (3) accepting difficult thoughts/images and oneself, versus judging cognitions and self; and (4) non-reacting of experience versus rumination/worry (Chadwick et al., 2008).
In its validation study, using a large combined clinical (patients with psychosis) and non-clinical (meditators and non-meditators) sample, Chadwick et al. (2008) found good internal consistencies, all for the total (α = 0.89), non-clinical (α = 0.89), and clinical (α = 0.82) samples. The average of item-total correlations was moderate (r = 0.54), and no significant effects of gender or age in SMQ scores were found. Scores were normally distributed in all three samples. Regarding the scale's factor structure, which was hypothesized as multidimensional, principal components analyses yield unifactorial structures, both in the total and in the group-divided samples. Attempts to carry out multidimensional solutions yielded uninterpretable results. Although no items were removed, very low loadings were found in the clinical sample. In terms of concurrent validity, SMQ scores were associated with mindfulness scores as assessed by the MAAS in all samples, positive subjective mood in the non-clinical sample, negative affect, positive affect, and delusional experience in the clinical sample. SMQ scores were able to discriminate between meditators, non-meditators, and people with psychosis. Böge et al. (2020) thoroughly studied the psychometric properties and factor structure of the SMQ German version, establishing internal consistency, convergent and divergent validity, and sensitivity to change over time following treatment. Concerning factor structure, results of confirmatory analyses indicated a better fit of the four-factor solution, which, however, still not revealed a good fit. In the clinical sample, exploratory factor analyses suggested a two-factor structure, which was, however, non-interpretable.
Although few other studies have explored the psychometric properties of the SMQ, the adequate internal structure has been corroborated in different clinical and non-clinical samples (Chadwick et al., 2009;Martins et al., 2019;Perona-Garcelán et al., 2014;Wong & Chen, 2015). To our knowledge, only one study assessed temporal stability (1 week), and it was considered excellent in a sample of people with residual psychotic symptoms (Wong & Chen, 2015).
Considering the need for empirical studies confirming the SMQ's factor structure in different samples (e.g., Im, 2017), the present study sought to confirm the factor structure and further explore SMQ's psychometric properties on a Portuguese community sample. In terms of concurrent validity, we hypothesized that the SMQ scores would be associated with scores from a well-known, used, and studied mindfulness scale (the FFMQ). Regarding convergent validity, mindfulness has been negatively associated with cognitive fusion (the process by which inner experiences are interpreted as an accurate description of reality; Hayes, 2004); therefore, we expected that the SMQ scores would be associated with a cognitive fusion measure. Moreover, we hypothesized that SMQ scores would also present significant associations with positive and negative affect. Similar to previous studies, we intended to explore the SMQ's sensitivity, i.e., the ability to differentiate scores from meditators and non-meditators from a community sample and people with psychosis.

Participants
A total of 243 participants from the community were recruited. The age of the participants ranged from 18 to 77 years (mean age = 32.25; SD = 12.04), and 57.5% were female. The mean of years of education was 15.21 (SD = 3.85). Concerning marital status, 66.7% were single, 28.8% were married, 4.1% were divorced, and 0.4% were widowed. To compare the scores of SMQ in three different samples, data were collected from people with psychosis and meditators, and we randomly selected 15% of participants of the community sample (in order to balance groups' sample size).
The clinical sample was composed of 43 patients with psychosis, 65.1% were male, with a mean age of 33.47 (SD = 11.47), ranging from 19 to 63. The mean of years of education was 11.53 (SD = 3.67). Regarding marital status, 83.3% of participants were single, 4.8% were married, and 11.9% were divorced.
The meditators' sample consisted of 29 participants, 82.8% were women, aged 22 to 69, with a mean age of 41.90 (SD = 11.17). The mean of years of education was 18.10 (SD = 3.23). Participants were mainly single (41.4%), 41.3% were married, and 17.2% were divorced. Regarding the time of meditation practice, participants have been practicing meditation for a mean of 84.48 months (SD = 69.48), ranging from 3 to 240 months. Concerning the type of meditation, 62.1% practice mindfulness meditation, 10.3% vipassana meditation, 17.2% contemplative meditation, and 31% another type (13.7% of participants reported two or more types of meditation). Participants practiced an average of 3.93 h (SD = 7.44) of meditation per week, and 72.4% have been in a meditation retreat.
The subsample of participants from the community sample comprised 44 participants, 54.5% women. The mean age was 30.05 (SD = 10.55), ranging from 20 to 69. The mean of years of education was 15.39 (SD = 2.28). Regarding marital status, 65.9% of participants were single, 29.6% were married, and 4.5% were divorced.

Procedures
Before data collection, the study was approved by the ethics committee of the University of Coimbra, Faculty of Psychology and Educational Sciences and National Commission for Data Protection (12,214/2015). SMQ was translated into Portuguese by two members of the research team. The initial Portuguese version was back-translated by a psychology research assistant who was unfamiliar with the original English version. The back-translated version was identical to the original SMQ.
Data from the community and meditators' samples were collected online (using the LimeSurvey tool) through social media, following a non-probabilistic sampling method (snowball effect). All participants that reported having mental health problems or being in psychotherapy were eliminated from our study. Regarding the meditators' sample, meditation associations were contacted and spread the survey among their members (and consequent snowball effect). The criteria for selecting the subjects of this sample were self-reported regular meditation practice. Participants were asked some questions about their meditation practice (e.g., How long have you been practicing meditation? What type of meditation do you practice? How many hours per week do you practice meditation? Have you attended a meditation retreat?).
The clinical sample was recruited in several Portuguese hospitals and mental health institutions. Referral to enroll in the study was performed by each participant's psychiatrist according to the following inclusion criteria: fulfilling DSM-5 diagnostic criteria for a psychosis spectrum disorder (any type, including mood disorders with psychotic features), age above 18 years old, with clinical stabilization (as assessed by their psychiatrist) and ability to understand and answer the questionnaires. Patients were informed about the study's objectives as well as the voluntary, anonymous, and confidential nature of their participation. After this brief explanation and all questions clarified, those who agreed to participate in the study gave their written informed consent (the form followed the Declaration of Helsinki).

Measures
Demographic Data Participants gave information regarding age, marital status, and educational level.
Southampton Mindfulness Questionnaire (SMQ; Chadwick et al., 2008) is a 16-item scale that assesses the relationship one establishes with distressing thoughts and images (e.g., "I am able just to notice them without reacting"). Items are scored on a 7-point Likert scale, ranging from strongly disagree (0) to strongly agree (6). In the original study, the Cronbach's alpha obtained was 0.89.
Five Facet Mindfulness Questionnaire (FFMQ; Baer et al., 2006) is a 39-item scale that assesses mindfulness. FFMQ items are rated on a 5-point Likert scale and measure five mindfulness facets, namely observing (8 items; e.g., "When I'm walking, I deliberately notice the sensations of my body moving"), describing (8 items; e.g., "I'm good at finding words to describe my feelings"), acting with awareness (8 items; e.g., "When I do things, my mind wanders off and I'm easily distracted."), non-judging (8 items; e.g., "I criticize myself for having irrational or inappropriate emotions"), and nonreactivity (7 items; e.g., "I perceive my feelings and emotions without having to react to them"). The original study obtained the following Cronbach's alpha values: nonreactivity = 0.75, observing = 0.83, acting with awareness = 0.87, describing = 0.91, and non-judging = 0.87. In the current study, the Cronbach's alpha values obtained were respectively: 0.75, 0.84, 0.92, 0.90, and 0.90.
Cognitive Fusion Questionnaire (CFQ; Gillanders et al., 2014) is a 7-item scale that assesses cognitive fusion. Each CFQ item (e.g., "My thoughts cause me distress or emotional pain") ranges from 0 (totally disagree) to 6 (totally agree), so higher scores suggest higher levels of cognitive fusion. The original validation study of CFQ showed Cronbach's alphas between 0.88 (for a mixed mental health diagnosis sample) and 0.93 (for a multiple sclerosis sample). The Cronbach's α for the current study was 0.92.
The Positive Affect and Negative Affect Scale (PANAS; Watson et al., 1988) is a 20-item scale and includes two mood scales, one measuring the positive affect (10 items; e.g., "interested") and the other measuring the negative affect (10 items; e.g., "afraid"). Respondents are asked to rate the extent to which they have experienced each particular emotion within a specified period, using a 5-point scale (1 = very slightly or not at all, 2 = a little, 3 = moderately, 4 = quite a bit, and 5 = very much). PANAS scores ranging between 10 and 30 for each subscale and higher scores indicate higher levels of positive and negative affect, respectively. In the original study, the reliability of the positive affect scale ranged from 0.86 to 0.90, and the negative affect scale from 0.84 to 0.87. In the current study, the internal reliability was 0.88 and 0.87 for positive and negative affect subscales.

Data Analyses
Preliminary data analyses were executed to examine the adequacy of the data using IBM SPSS Statistics 25 and MVN: an R package for assessing multivariate normality software (Korkmaz et al., 2014). Univariate and multivariate skewness (Sk) and kurtosis (Ku) values were verified to examine if there was a severe bias to normal distribution. Mahalanobis distance statistic (D 2 ) was used to inspect the presence of multivariate outliers.
Confirmatory factor analysis (CFA) was performed, using the MPlus 8.1 software, to test the factor structure of the SMQ. A robust maximum likelihood parameter (MLR) estimation was chosen. We performed an item-level CFA and a parcel-level CFA. Regarding item-level CFA, we tested a unifactorial structure of the SMQ, as obtained by Chadwick et al. (2008), with 16 items being the observed indicators. In the parcel-level CFA, to test the multidimensionality of the item sets, four parcels were created as observed indicators for the latent factor. Each parcel corresponded to the facets of SMQ theoretically assumed by Chadwick et al. (2008). One method suggested by Kishton and Widaman (1994), for dealing with multidimensional item sets, is the internal consistency approach, in which each parcel is created using each facet as the grouping criteria. Therefore, each parcel would reflect one facet of the SMQ and would be the sum or average of the items that composed that facet. This parceling technique has several advantages, such as it reduces the residual error variances of individual items and avoids violations in the normal distribution (Little et al., 2002). To evaluate the model fit several goodness-of-fit indices were used: chi-square (χ 2 ), comparative fit index (CFI), the standardized root mean square residual (SRMR), and the root mean square error of approximation (RMSEA). Chisquare is very sensitive to sample size and tends to produce significant results with large samples (Kline, 2005). The criteria for acceptable fit were RMSEA < 0.08 (Diamantopoulos & Siguaw, 2000); CFI > 0.90 (Hair et al., 2010); and SRMR < 0.08 (Hair et al., 2010). Regarding local adjustment, all standardized factor loadings (λ) and individual reliability (R 2 ) should present statistical significance (p < 0.05; Schumacker & Lomax, 2010). Cronbach's alpha was used to assess internal consistency.
Pearson correlation coefficients were performed to assess concurrent and convergent validities. Correlations between 0.30 and 0.50 are considered low, between 0.50 and 0.70 moderate, and above 0.70 high (Tabachnick & Fidell, 2007). A one-way independent ANOVA was performed to compare mean scores of mindful awareness of distressing thoughts and images between groups (non-meditators vs. meditators vs. clinical sample). The subsample of participants from the community sample was randomly selected (15%) to perform a difference test with sample sizes. The homogeneity assumption was tested through Levene's test. The post hoc Tukey HSD procedure was performed because it is considered the most powerful test for controlling type I errors (Field, 2013).

Descriptive Statistics and Reliability Analysis
Means, standard deviations, corrected item-total correlations, Cronbach's alpha if item deleted, and Cronbach's alpha for item-level and parcel-level of the SMQ are displayed in Tables 1 and 2, respectively. For the item-level CFA, item-total correlations were above 0.30, ranging from 0.35 (item 8) to 0.57 (items 5 and 11), and all items significantly contributed to the instrument's internal consistency. Cronbach's α of the SMQ total score (without the 3 items) was 0.84. For the meditators' sample, the SMQ presented very good internal reliability (α = 0.92), whereas, for the clinical sample, the reliability was adequate (α = 0.70).
For the parcel-level CFA, all item-total correlations were between 0.50 (parcel 4) and 0.66 (parcel 2), and all four parcels contributed to the internal consistency. SMQ total score (with the 4 parcels) presented an adequate internal consistency (α = 0.77). SMQ total score presented very good internal reliability for the meditators' sample (α = 0.91), whereas for the clinical sample reliability was adequate (α = 0.63). Considering that the CFA results indicate a better fit for the multidimensional structure, all the following analyses were performed accordingly. Table 3 displays Pearson's correlation coefficients between variables in the community sample. As expected, all SMQ facets were positively and low to moderately related to each other and were highly correlated to SMQ's total score. Also, in general, SMQ and its facets showed positive and low to moderate associations with FFMQ subscales, except for the FFMQ observe subscale and SMQ letting go facet, which presented a negative and low correlation. Significant positive but low associations were found between SMQ total score and SMQ's observing facet and positive affect (PANAS). Additionally, significant negative and low to moderate associations were found between SMQ and CFQ and negative affect (PANAS).

Sensitivity
In this study, one-way independent ANOVA was performed to test mean differences in SMQ total score and subscale scores between three groups (non-meditators vs. meditators vs. clinical sample). Results showed significant differences in mindful awareness of distressing thoughts and images between groups, F (2, 113) = 16.37, p < 0.001. This significant difference has a large effect size, η 2 = 0.23, according to Cohen's (1988) recommendations. Post hoc comparisons using Tukey's HSD test indicated that meditators have higher levels of mindful awareness of distressing thoughts and images (n = 29; M = 4.22, SD = 0.89) than both the community (non-meditators) sample (p < 0.001; n = 44; M = 3.52, SD = 0.72) and the clinical sample with psychotic spectrum disorders (p < 0.001; n = 43; M = 3.13, SD = 0.82). Surprisingly, no significant differences were found between the community and clinical samples in the SMQ levels (p = 0.063).
Regarding results for SMQ's facets, there were significant differences between groups in observing dimension, F (2, 113) = 4.06, p = 0.020, with a moderate effect size, η 2 = 0.07. Post hoc comparisons showed that only meditators had higher levels of observing facet than clinical sample

Discussion
When assessing clinical samples, the Southampton Mindfulness Questionnaire has been pointed out as particularly adequate, especially for people struggling with distressing internal experiences. Thus, and further developing the work of Chadwick et al. (2008), this study aimed to analyze both the unidimensional structure and the theoretically driven facet structure of SMQ in our sample through CFA. The model that tested the unifactorial structure with 13 items presented a similar but yet alternative factor structure than the one proposed by the original study. Two of the items removed were mainly associated with "internal experiences"-judgment (item 3: "I judge the thought/ image as good or bad"; item 15: "I try just to experience the thoughts or images without judging them"). We hypothesize that these items might have been more difficult for people from the community sample to understand since they might struggle with being aware of thoughts (since in non-clinical samples, negative automatic thoughts might be less distressing and thus salient) and metacognitive processes involving these thoughts (e.g., judgment). Since these items focused on the judgment of internal experience did not contribute to explaining the latent factor of SMQ, it justifies its removal from the final CFA model. It is worth noticing that the other item removed, though was not related to judgment (item 12: "In my mind I try and push them away"), also had poor psychometric properties in the original study. Even with the removal of these items, the unidimensional model failed to adequately fit the data. This suggests that a multidimensional hypothesis could be tested.
We tested the multidimensionality of the SMQ through a parceling method. The model fit obtained was acceptable, except for the RMSEA value. According to Kenny et al. (2015), a falsely high RMSEA could be obtained for models with small degrees of freedom and a small sample size. Given this, we suggest testing the multidimensionality of the scale using a larger sample size and other statistical methods to further study the factor structure of the SMQ.
Taking into consideration the results from the original study (in which attempts to explore the theoretically driven multidimensional factor structure yield uninterpretable solutions), conclusions from Böge et al. (2020) (stating that a multidimensional structure might be more appropriate, although the model fit was still not good), and the present study's results, it seems that SMQ still raises crucial concerns regarding its definitive factor structure that warrant further research. It could be possible that the factor structure needs structural adaptation according to the addressed population, leading to distinct versions. Future studies are needed to address this issue exploring measurement invariance. Nonetheless, the present study makes a significant contribution to the ongoing discussion around the items comprising SMQ.
Overall, the Portuguese version of the SMQ presented good internal reliability for the community and meditators' samples, which resembles the results found in the original version (Chadwick et al., 2008). Still, for the clinical sample, internal reliability was only adequate. Furthermore, also similar to the original version, all item-total correlations were moderate, indicating that the individual indicators are representative of the latent construct.
Concerning concurrent validity and as predicted, SMQ showed positive associations with mindfulness assessed by FFMQ. Interestingly, non-significant associations were found between the observe subscale of FFMQ and SMQ facets and total score, except for the letting go facet of SMQ, in which we obtained a negative association. However, many items from the FFMQ observe subscale specifically assess the tendency to observe external stimuli (sounds, smells, etc.) and bodily sensations (Baer et al., 2006). In turn, SMQ does not include such items but rather is focused on one's ability to notice distressing thoughts and images.
Moreover, higher SMQ scores were also associated with more positive affect and less negative affect and cognitive fusion. This is in line with the vast literature regarding the importance of mindfulness that points out that having mindful skills allows people to be more aware and experience their internal events in a more open and less biased manner and is related to increased positive emotion (e.g., Teasdale et al., 2003).
As expected, significant differences were found regarding levels of mindfulness between meditators and non-meditators (clinical and community samples). The majority of our meditators' sample reported mindfulness meditation practice (62.1%) which might explain these results, which are in accordance with Böge et al. (2020). Additionally and surprisingly, no significant differences between the community sample and clinical sample were found. Although differences between the community sample and clinical sample did not reach significance, lower levels of mindfulness abilities were found in the clinical sample, which is in accordance with previous research (Chadwick et al., 2008). However, this result limits the sensitivity of SMQ to distinguish between samples. The potential distinction between groups seems to be only observed in the non-judging facet of SMQ. Our results indicated that meditators tend to report higher levels of non-judgment than non-meditators and the clinical sample, respectively. The same pattern was found between community and clinical samples, with the latter being more judgmental to own distressing thoughts and images. Overall, these results should be interpreted with caution due to the low ability of the SMQ to differentiate samples. Thus, further research is needed particularly regarding differences with total score and other facets of mindfulness. Still, some caution in this interpretation is needed due to the modest internal reliability of the SMQ total scale in the clinical sample.

Limitations and Directions for Future Research
There are some limitations to the present study that should be considered and tackled in future studies. A major limitation of this study is its cross-sectional design that does not allow for further conclusions to be drawn, namely in what regard temporal stability and sensitivity to treatment (clinical sample) or sensitivity to increases in meditation practice throughout time (meditators' sample). The small sample sizes of the clinical and meditators' samples did not allow us to test and compare the factorial structure in these specific samples through more complex and robust analysis (e.g., multigroup analysis testing for factor structure invariance). An additional limitation is that in this study, except for the meditators' sample, the previous contact and experience with mindfulness practices were neither assessed nor controlled in the analysis.
Nevertheless, this study contributes to broadening the available measures for both non-clinical and clinical samples, especially self-report measures to capture possible changes over time (including the course of intervention or treatment) in the way individuals deal with their distressing thoughts and images. Although the measurement of mindfulness processes has proven to be a difficult task, the replication studies on psychometric properties of the same psychological questionnaires are valuable. Moreover, this study is a contribution to the Portuguese research in a way that provides psychometric data for this well-known and widely used mindfulness measure.
Author Contribution MJM designed and executed the study, assisted with the data analyses, and wrote the paper. AX and LP collaborated with the data collection, data analysis, and writing of the study. CCM analyzed the data and assisted with the writing of the paper. RQ collaborated with the data collection. PC collaborated with the design and critically reviewed the paper.

Data Availability
The data that support the findings of this study are available from the corresponding author upon reasonable request.

Declarations
Ethical Approval The study was approved by the ethics committee of the University of Coimbra, Faculty of Psychology and Educational Sciences and National Commission for Data Protection (12214/2015). All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/ or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.