Assessing Shame and Guilt in Adolescents: Translation and Adaptation of Test of Selfconscious Affect for Adolescents (TOSCA-A)

Received 07 May, 2015

Accepted 20 Sep, 2016

Published 30 Jun, 2017

The study was aimed to translate, adapt, and validate a universally recognized measure of shame and guilt that is Test of Selfconscious Affect for Adolescents (TOSCA-A) and to test generalization across gender. Sample of the study comprised of 459 adolescents from public and private educational institutions of Rawalpindi and Islamabad. Confirmatory factor analysis was carried out to estimate the factor structure of TOSCA-A. Model fit indices confirmed the factor structure of TOSCA-A. Multi-group invariance test was conducted to estimate the gender sensitivity of the scale. Results revealed that Scale is a valid measure of shame and guilt for boys and girls. The range of factor loadings on shame was .36 to .55 (except for item 9a and 5b) and factor loadings on guilt were ranging from .31 to .56 (except for item 6b). The three items were retained as they appeared to show good loadings in multi-group analysis across gender. Reliabilities (Cronbach’s alpha .80 and .81 for shame and guilt, respectively) showed that the scale is a reliable measure of shame and guilt. Structural models showed nonsignificant gender differences suggesting generalization of the factor structure for both boys and girls. Translated version of TOSCA-A appeared to be a reliable and valid measure of shame and guilt. It was also found to be equally generalizable and valid measure for both boys and girls as a measure of shame and guilt.

An emotion is a distinctive, episodic, relatively short-term mental state that arises when an individual is evaluating an event relevant to a personal goal that is important, and that prompts a readiness to act (Frijda, 1986). An emotion can be positive or negative depending on whether the event is signaling an advancement of a personal goal or impeding the goal. Though there are different categories of emotions, these can be broadly categorized into basic emotions and complex emotions. Self-conscious emotions which include shame, guilt, embarrassment, and pride are considered complex emotions. These are called so because these are believed to be conjured by self-reflection and self-evaluation (Frijda, 1986).

The term “self-conscious emotions” is originally coined by M. Lewis (as cited in Motan, 2007) who used this term to mention guilt, shame, pride, and their many possible variations. Self-conscious emotions engage appraisal of one regarding some contextual or comprehensive criteria (M. Lewis, Sullivan, Stanger, & Weiss, 1989). Owing to their complexity, these are called ‘social’, ‘higher-order’, or ‘moral’ emotions, etc. The ability to structure stable self representations (me) and to focus on these illustrations (I) as well as putting all these together for generation of self-evaluation is the primary distinguishing feature of self-conscious emotions (Tracy & Robins, 2004). Though non self-conscious emotions may involve self-evaluations, but the appraisal process is not necessary in non self-conscious emotions.

Shame and guilt are two self-conscious emotions which have been largely focused in research as compared to other self-conscious emotions (Kim, Thibodeau, & Jorgensen, 2011; Tangney, 1991; Tangney & Dearing, 2002, 2003; Tangney, Stuewig, & Mashek, 2007; Tracy & Robins, 2004, 2006; Tracy, Robins, & Tangney, 2007). Lay people as well as clinicians and researchers use the words “shame” and “guilt” interchangeably. However, one finds the attempts focusing on differentiation of shame from guilt, but still these are considered negative emotions in that both involve feeling bad about oneself. Simple dictionary definitions explain shame to be a painful emotion resulting from an awareness of inadequacy, has synonyms like disgrace and dishonor and related words like self-disgust and self-hatred. Guilt on the other hand is described as remorseful awareness of an offense or that something is done wrong (Mayor, 2009; Stevenson, 2010).

Over the years, efforts to distinguish between shame and guilt can be classified into three broader categories: (a) Differentiation based on eliciting events; (b) differentiation based on the transgressions that is, either public or private, (c) differentiation on the basis of focus that is, self or behavior (Tangney et al., 2007). Research has indicated that nature and characteristics of events amazingly do not define the either shame will be experienced or guilt. Studies on children and adults on personal shame and guilt experiences have revealed that there are few, if any, “classic” shame-inducing or guilt-inducing situations (Tangney, 1992; Tracy & Robins, 2006).It was reported by Tangney and Dearing (2003) that some people report feeling guilt, while others feeling shame even in the similar situations (e.g., lying stealing or cheating).

Another often cited differentiation of shame and guilt comes from anthropological perspective. This distinction conceived shame as a public emotion and guilt as a private emotion. Shame arises when there is a public exposure, and it is assumed that the transgression or wrongdoing will be disapproved by the public; while, guilt is considered to arise as a result of conscience (e.g., Benedict, 1967). This public/private distinction is also disapproved by empirical research (Tangney, Miller, Flicker, & Barlow, 1996).

Third perspective for distinguishing between shame and guilt focuses on “self versus behavior” (Tangney et al., 2007, p. 349). This distinction is currently most dominant basis and was first proposed by H. B. Lewis (1971). Tracy and Robins (2004)recently elaborated this distinction in appraisal-based model. In accordance withH. B. Lewis (1971), shame reflects a negative appraisal of the one’s self, whereas guilt reflects a negative appraisal of one’s particular behavior. Guilt is experienced when we focus on what we did, the “bad” act. Shame focuses “self”, the bad self or who we are. For example, someone feeling shame might say, “Iam a horrible thing”, but someone feeling guilt might say, “Ididthat horriblething”. This distinctive focus on self and behavior albeit may appear elusive, it gained enormous empirical support.

It has been revealed that this differential focus leads to very different emotional experiences and patterns of motivations. Also shame and guilt feelings set stage for and different subsequent behaviors (Tangney, 1991; Tangney & Dearing, 2003; Tangney et al., 2007). Even though both have inner attributions, guilt attributions are specific, controllable, malleable, and have potential to motivate an individual to change behavior or reparative actions, whereas the shame attributions are global uncontrollable, much more persistent and lowers a person’s self-esteem (Tracy & Robins, 2006). Orth, Robins, and Soto (2010) found that when mutually controlled for; depression was positively linked to trait shame, but negatively associated to guilt. They concluded, “Shame was consistently maladaptive and shame-free guilt is consistently adaptive across all stages of the life span” (p. 1067).

Some phases of growth and development are more critical and have lifelong impact on individual’s life. One of such phases of life is adolescence. It is a time of rapid and tremendous physical (pubertal), cognitive, emotional, and social maturity (Lowe & Gibson, 2005).Several of the factors that may influence development of shame and guilt in children and adolescents include pubertal changes, defiance and oppositional behaviors, peer and social relationships, parenting, technology use, etc. Adolescence, literally means “to grow into maturity”, is the stage of life when individuals attain sexual maturity. Bowlbys’ (1973) attachment theory articulates that those children and adolescents who are insecurely attached to their primary caregivers are susceptible to entrap in identity relevant issues, thus, also more prone to experience rejection and the self-conscious emotions, particularly, shame and guilt. It has been found that negative parenting behaviors, which may involve excessive conflict between parents and children, are associated with children’s proneness to experience self-conscious emotions. This is, especially, true for shame-proneness, and is proved by retrospective reports of adults as well as cross-sectional and prospective studies with children and adolescents. These studies found that increased shame-proneness is associated with indifference, rejection, and abandonment of parents (Claesson & Sohlberg, 2002; Gilbert, Miller, Berk, Ho, & Castle, 2003; Han & Kim, 2012). This further leads adolescents towards externalizing and internalizing problems including depression, anxiety, and stress (Zahn-Waxler, Klimes-Dougan, & Slattery, 2000). The indigenous empirical research has confirmed the new developments in role of shame and guilt. The predictive role of shame, but not guilt in development of psychopathology is supported in collectivist cultures (Shahnawaz & Malik, 2016; Taihara & Malik, 2016).

Different scales to measure shame and guilt have been developed; some measure dispositions to shame, while others measure state based on shame and guilt experiences (Robins, Noftle, & Tracy, 2007). Some of them are scenario based questionnaires, while others are adjective checklists. One of these scales with well-established psychometric properties and most frequently used in research settings across cultures is called Test of Self-Conscious Affect for Adolescents (TOSCA-A; Tangney & Dearing, 2003). Other frequently used scales of shame and guilt include Personal Factor Questionnaire (PFQ-2; Harder & Zalma, 1990), Differential Emotions Scale-IV (DES-IV; Izard, Libero, Putnam, & Haynes, 1993), etc.

Ferguson and Crowley (1997) concluded that both TOSCA and PFQ-2 are valid measures of self-conscious emotions, shame, and guilt, through multitrait-multimethod analysis with nonclinical sample. They further added that that guilt measures are qualitatively different. TOSCA measures non ruminative guilt that is, it is free from rumination. PFQ-2 is a measure of unhealthy or pathological guilt. Also Shame scale of TOSCA has support of construct validity, but validation of PFQ-2 is found to be problematic (Robins et al., 2007). PFQ-2 is supposed to have two factors, as proposed by the authors (Harder & Zalma, 1990), the discriminant validity of these two factors is found to be poor in different studies (e.g., Harder, 1990; Harder, Rockart, & Cutler, 1993; Harder & Zalma, 1990).

Robins et al. (2007) reported that the items of DES - IV were based on Izard’s differential emotions theory (Izard, 1991). Item pool of this scale was generated from cross cultural emotional expression labels. These labels were expanded to short statements to be used easily across various groups. However, it is found that Shame scale appears to be measuring embarrassment, while Hostility-Inward scale is in fact a measure of clinical notion of shame. Items assessing guilt seem conceptually closer to self-blame, misconduct, and regret (Robins et al., 2007).

Given the empirical support on psychometrics of the TOSCA-A, the present study was designed to translate and adapt TOSCA-A in Urdu to be available to assess self-conscious emotions in Pakistani adolescent. We further aimed to assess validation of the instrument by examining construct validity and its effectiveness by testing generalization across gender. Finally, convergent and discriminant validity was assessed using correlated and uncorrelated constructs to shame and guilt.

METHOD

Translation and Adaptation
Test of Self-Conscious Affect for Adolescents (TOSCA-A). It is a self-report measure of self-conscious emotions (Tangney & Dearing, 2003). It comprised of 15 scenarios out of which 5 scenarios are positive and 10 are negative ones that adolescents would possibly encounter in everyday life. Each scenario is followed by four response items that assess guilt-proneness, shame proneness, detachment, and externalization. For example a scenario: “You trip in the cafeteria and spill your friend’s drink”, has responses such as, “I would be thinking that everyone is watching me and laughing” (shame-proneness). Responses are rated on a 5-point Likert scale (1=Not at all likely, 2 = Unlikely, 3 = May be (half & half), 4 = Likely, 5 = Very likely). Shame and Guilt subscales has 15 items each and the score range for each scale is from 15 to 75. Subscale scores are the sum of responses to relevant items (e.g., the score of Shame subscale equals the respondent’s answer to 1a, plus the answer to 2c, etc.). Earlier literature suggests the scale to be highly reliable and internally consistent, as measured by Chronbach’s alpha, for adolescents (.77 for the Shame subscale and .81 for the Guilt subscale). Also test-retest reliability and predictive as well as convergent validity of TOSCA-A have been established (Tangney, 1996; Tangney, Wagner, Hill-Barlow, Marschall, & Gramzow, 1996).

Permission of translation and adaptation. The permission for translation, adaptation, and cross-language validation was obtained from author of the scale. Adaptations were also made to make the items more appropriate to the indigenous cultural context. This complex process required excessive care just to make sure that the instrument remains consistent with its original version and at the same time appropriate for new culture. It was taken care that the instrument should practically perform in the same way as original versions. The focus was on cross-cultural and conceptual along with on linguistic/literal equivalence. A well established method to achieve this goal is to use backward translations.

Translation and content validation. Only shame and guilt items were translated from English language to Urdu and adapted according to the indigenous culture. The instrument was given to 4 independent bilingual experts for forward translation from English to Urdu. These independent translations were reviewed by a committee of three members to select and finalize the most appropriate translation. This finalized Urdu version was further handed to three translators for back translation into English. Back translations were reviewed by the committee to finalize most appropriate translation.

Review from the authors of the scales. The finalized back translation of TOSCA-A was forwarded to the authors of the for their review. The author identified two significant changes in back translation. First in the scenario one and other in the item 12b. The meaning of “trip in cafeteria” was changed to “thokar khaty hn” and it was previously “dakhil hoty hen”, in scenario one. The statement 12b was “mein koshish karon ga kay jald az jald apnay dost tak pohanch sakoon”. This was changed to “mein koshish karon ga kay jald az jald iss ki talafi kar sakoon”.

Tryout
In second phase, a tryout of the translated instrumen was conducted to test the psychometric properties of the scale. The sample comprised of 60 adolescents with age ranging from 15 years to 19 years (M = 16.60, SD = 1.61). Data were collected through convenient sampling from boys and girls of various public and private institutions (schools, colleges, and universities) of Rawalpindi and Islamabad. Among the respondents 16 (26.7%) were boys and 44 (73.3%) were girls. Pilot study results indicated that Cronbach’s alpha are in desirable range. The alpha coefficient reliability of Shame was .72 and that of Guilt was .79. This suggested that the translated versions were reliable enough to be used on the large sample. Finally, in the third phase data was collected for main study. The alpha coefficient reliability of main study sample showed improvement in internal consistency with alpha for Shame = .80 and alpha for Guilt = .81.

Validation Study
Sample and procedure. Permission was sought by the Federal Directorate of Education (FDE) to collect data from Islamabad model schools and colleges. Data were collected through convenient sampling in classroom settings and informed consent was obtained from respondents. The sample comprised of 459 adolescents from public and private institutions of Rawalpindi and Islamabad. The age range of the sample was from 15 years to 19 years (M = 16.47, SD = 1.3). Among the respondents, 222 (48.4%) were boys and 237 (51.6%) were girls. A demographic sheet was also administered to collect information on the demographic variables including age, gender, education, family structure, parent’s education, and family income.

Instrument. Following measures were used in validation study.

Urdu Translated Version of (TOSCA-A). See details as mentioned above (see Appendix).

Depression Anxiety Stress Scale (DASS-21). It is a quantitative measure of distress along the 3 axes; depression, anxiety, and stress. It is used to measure mood symptoms over the past week (Lovibond & Lovibond, 1995). Scoring on DASS-21 is summing up the score of each item to get a total score. Higher scores indicate greater levels of psychological distress (Henry & Crawford, 2005). DASS was translated by Aslam (2007) in Urdu language. For the present study, overall score of DASS was used as an outcome measure of psychopathology. The scale showed high internal consistency with Chronbach’s alpha .84.

RESULTS

The objective of Validation Study was to establish psychometrics of Urdu translated Shame and Guilt subscales of TOSCA-A.

For the validation of Urdu version of the scale, Confirmatory Factor Analysis (CFA) was carried out using AMOS statistical package version 21. First of all chi-square, one of the frequently usedstatistics of model fit, demonstrates the inconsistency between the hypothesized model and model entailed by the data. In addition to chi square, different model fit indices, as suggested byByrne (2001) and Keith (2005) were examined. CFA was carried out in the following different steps.

Table 1:	Model Fit Indices of Urdu Version of TOSCA-A (N= 459)
Note. M1 = Default Model; M2 = M1 with correlated errors variances; M3 = M2 across gender; M4 = M2 with equality constraints on item loadings

The factor loadings of almost all items is above .30, which means that the items are valid indicators of their respective factors, except for item 5 (λ= .20), and item 9 (λ=.08) of the Shame subscale and item 6 (λ= .16) of the Guilt subscale. The items were retained in the model for further analysis to estimate gender sensitivity and item equivalence (equal importance) for both boys and girls. To check the assumption that the scale is generalizable across gender, and that the underlying constructs have the same theoretical structure for both girls and boys, multi-group invariance testing through AMOS was carried out. Generalizability was tested by estimating equivalence of item loadings on the respective constructs (i.e., Shame and Guilt) across gender. For the purpose third model was estimated firstly with open estimation of item loadings across gender (Model 3) and then by applying equality constraints on items loadings for boys and girls (Model 4). Several measures have been suggested to be appropriate when comparing nested models, as opposed to evaluating the absolute fit of one particular model. One of these is Akaike's Information Criterion(AIC). For this index, smaller values indicate relatively better fit of alternative models. AIC value for model with gender equivalence (M4, AIC = 1483.37) is low compare to freely estimated model (M3, AIC = 1496.76), suggesting model with equality constraints as preferred model compare to freely estimated model. To estimate the internal consistency of translated version of TOSCA-A, Cronbach’s alpha were computed. Reliability of Urdu version of TOSCA-A for overall sample was .80 and .81 for Shame and Guilt, respectively.

Table 2:	Group-wise Mean and Standard Deviation of Urdu Version of TOSCA-A items across Gender and Age Groups (N=459)

Descriptive statistics including mean and standard deviations across age and gender for Shame and Guilt subscales are given in Table 2. Mean and standard deviation of all items is reported to get an overview of the sample distribution for the items of the instrument.

To assess convergant and discriminate validity, correlations were estimated for the relationship of Shame and Guilt subscales of TOSCA-A with psychopathology as measured by DASS (see Table 3).

Table 3:	Mean, Standard Deviation, and Correlations of Demographic Variables, Shame, and Guilt in Adolescents (N=459)
Note. FY = Formal Years Education; DASS = Depression Anxiety Stress Scale. Mean and standard deviations are not reported for categorical variables *p < .05. **p < .01

The result in Table 3 shows that Shame but not Guilt appears to significantly positively correlate with psychopathology. Results also show that shame-proneness and guilt-proneness are significantly and positively correlated. It is also evidenced that there is a significant positive relationship between age of the adolescents and shame-proneness, though the magnitude of correlations is low. Nonsignificant relationship was found between age and guilt-proneness. Guilt-proneness and the years of formal education of adolescents showed a negative relationship (r = -.10, p < .05). As presented in Table 3, no significant relationship of family system, number of siblings, and family income are evidenced with either shame-proneness or guilt-proneness.

DISCUSSION

Over the past 2 decades, interest of developmental psychologist has grown dramatically in the self-conscious emotions especially shame and guilt (Tracy et al., 2007). Shame-proneness is considered as “bedrock of psychopathology”; on the other hand, guilt is considered to be adaptive emotion. While these emotions are very important an individual’s psychological health, there is a dire need for sound instruments to measure these constructs across cultures. One of the frequently used instrument to measure shame and guilt experiences across different situations, shame- and guilt-proneness in adolescents is TOSCA-A (Robins et al., 2007). Given the utility and effectiveness of TOSCA-A in measuring shame and guilt in adolescent across various cultures, the present study was aimed to translate, adapt, and validate Urdu translated version TOSCA-A. Translation and adaptation was carried out following the standard guidelines agreed by most psychometricians (Beaton, Bombardier, Guillemin, & Ferraz, 2000; Gudmundsson, 2009; Hambleton, 2005; International Test Commission [ITC], 2010; Simonsen & Mortensen, 1990; Sireci, Yang, Harter, & Ehrlich, 2006). The process of translation and adaptation comprised of different steps including forward translations by multiple translators, selection of the most appropriate translations by a committee, backward translations by independent translators, and review of back translation from the author of TOSCA-A.

After completion of the process of translation and adaptation of the scale, pilot study was carried out to further check comprehension and appropriateness of items regarding their meaning and difficulty in reference to target population of the instrument. Translated version of TOSCA-A appeared to be internally consistent and reliable in the pilot study, the scale was further used to collect data without any changes for testing construct validation and generalizability across hydrogenous groups of adolescents (N = 459). CFA confirmed the factor structure of TOSCA-A for Pakistani adolescents, as proposed by the authors of the scale. In generalization testing across boys and girls, the model with equality constraints on item loadings supported generalization and utility of the instrument across boys and girls. Results indicated that none of the items is sensitive towards a particular gender. It may be concluded that TOSCA-A is equally effective for both boys and girls in measuring shame and guilt and any differences appearing on the constructs (i.e., shame and guilt) between boys and girls shall be attributed to gender.

Our results indicated non significant gender differences between boys and girls on the factor structure of shame- and guilt-proneness. Although, these findings contradict a general argument in the literature suggesting that girls are more prone to experiences of shame and guilt; our results are in line with empirical literature evidencing non-significant differences in early adolescents by De Rubeis and Hollenstein (2009). The suggested line of argument is that may be boys are more shame-prone than previously thought. This rationalization is supported by Ferguson and Eyre (2000) who argued that male participants may be more prone to shame and guilt under certain circumstances.

The results showed that shame, but not guilt appear to significantly correlate with psychopathology (Shahnawaz & Malik, 2016; Taihara & Malik, 2016). These results evidenced convergent and discriminate validity of the measure with two independent constructs that is shame and guilt. The positive relationship between shame-proneness and age of the adolescents in the current study may be explained with the developmental complications. The similarity in pattern of relationship of age and education might be a function of age as the data is collected from adolescent with an obviously high correlation between age and education. The literature suggesting a decrease in shame-proneness with increase in age (Crystal, Parrott, Okazaki, & Watanabe, 2001) reports results from diverse age group. As others (e.g., De Rubeis & Hollenstein, 2009) reported no relationship between age and shame-proneness during early adolescence in a longitudinal study. A decrease in guilt-proneness with increase in formal years of education may be attributed to the fact that with increase in education, individuals learn to justify their transgressions by rationalization.

LIMITATIONS AND SUGGESTIONS

The sample of the study was collected from educational instructions of two cities only. The psychometrics of the instrument shall be confirmed in a more hydrogenous sample.

IMPLICATIONS

The Urdu version of the Test of Self-Conscious Effect-Adolescent (TOSCA-A) is available to be used by researcher for the population who can read and understand Urdu.

CONCLUSION

Urdu translated version of TOSCA-A appeared to be a reliable and valid measure of shame and guilt. It is also found to be equally affective and generalizable for both boys and girls as a measure of shame and guilt.

REFERENCES

1. Aslam, N. (2007). Psychological disorders and resilience in earthquake affected individuals. (Unpublished M. Phil dissertation). National Institute of Psychology, Quaid-i-Azam University, Islamabad.
2. Beaton, D. E., Bombardier, C., Guillemin, F., & Ferraz, M. B. (2000). Guidelines for the process of cross-cultural adaptation of self-report measures. Spine, 25(24), 3186-3191.
3. Benedict, R. (1967). The chrysanthemum and the sword: Patterns of Japanese culture. Boston, MA: Houghton Mifflin Harcourt.
4. Byrne, B. M. (2001). Structural equation modeling with AMOS, EQS, and LISREL: Comparative approaches to testing for the factorial validity of a measuring instrument. International Journal of Testing, 1(1), 55-86.
5. Bowlby, J. (1973). Attachment and loss: Separation, anxiety and anger. New York, NY: Basic Books.
6. International Test Commission (ITC; 2010). International Test Commission guidelines for translating and adapting tests.
7. Claesson, K., & Sohlberg, S. (2002). Internalized shame and early interactions characterized by indifference, abandonment and rejection: Replicated findings. Clinical Psychology & Psychotherapy, 9(4), 277-284
8. Crystal, D. S., Parrott, W. G., Okazaki, Y., & Watanabe, H. (2001). Examining relations between shame and personality among university students in the United States and Japan: A developmental perspective. International Journal of Behavioral Development, 25(2), 113-123.
9. De Rubeis, S., & Hollenstein, T. (2009). Individual differences in shame and depressive symptoms during early adolescence. Personality and Individual Differences, 46(4), 477-482.
10. Ferguson, T. J., & Crowley, S. L. (1997). Measure for measure: A multitrait-multimethod analysis of guilt and shame. Journal of Personality Assessment, 69(2), 425-441.
11. Ferguson, T. J., & Eyre, H. L. (2000). Engendering gender differences in shame and guilt: Stereotypes, socialization, and situational pressures. In A. H. Fischer (Ed.), Gender and emotion: Social psychological perspectives (pp. 254-275). Cambridge, UK: Cambridge University Press.
12. Frijda, N. H. (1986). The emotions: Studies in emotion and social interaction. Cambridge University Press.
13. Gilbert, M., Miller, K., Berk, L., Ho, V., & Castle, D. (2003). Scope for psychosocial treatments in psychosis: An overview of collaborative therapy. Australasian Psychiatry, 11(2), 220-224.
14. Gudmundsson, E. (2009). Guidelines for translating and adapting psychological instruments. Nordic Psychology, 61(2), 29-45.
15. Hambleton, R. K. (2005). Issues, designs, and technical guidelines for adapting tests into multiple languages and cultures. In R. K. Hambleton, P. F. Merenda & C. D. Spielberger (Eds.), Adapting educational and psychological tests for cross-cultural assessment (pp. 3-38). Mahwah, NJ: Lawrence Erlbaum Associates.
16. Han, S. Y., & Kim, Y. H. (2012). Interpersonal rejection experiences and shame as predictors of susceptibility to peer pressure among Korean children. Social Behavior and Personality: An International Journal, 40(7), 1213-1231.
17. Harder, D. (1990). Additional construct validity evidence for the Harder Personal Feelings Questionnaire measure of shame and guilt proneness. Psychological Reports, 67(1), 288-290.
18. Harder, D., Rockart, L., & Cutler, L. (1993). Additional validity evidence for the Harder Personal Feelings Questionnaire‐2 (PFQ2): A measure of shame and guilt proneness. Journal of Clinical Psychology, 49(3), 345-348.
19. Harder, D., & Zalma, A. (1990). Two promising shame and guilt scales: A construct validity comparison. Journal of Personality Assessment, 55(3-4), 729-745.
20. Henry, J. D., & Crawford, J. R. (2005). The short-form version of the Depression Anxiety Stress Scales (DASS-21): Construct validity and normative data in a large non-clinical sample. British Journal of Clinical Psychology, 44(2), 227-239.
21. Izard, C. E. (1991). The psychology of emotions. New York, NY: Plenum Press.
22. Izard, C. E., Libero, D. Z., Putnam, P., & Haynes, O. M. (1993). Stability of emotion experiences and their relations to traits of personality. Journal of Personality and Social Psychology, 64(5), 847-860.
23. Keith, T. Z. (2005). Using confirmatory factor analysis to aid in under-standing the constructs measured by intelligence tests. In D. P. Flanagan & P. L. Harrison (Eds.), Contemporary intellectual assessment: Theories, tests, and issues (2nd ed., pp. 581-614). New York, NY: Guilford Press.
24. Kim, S., Thibodeau, R., & Jorgensen, R. S. (2011). Shame, guilt, and depressive symptoms: A meta-analytic review. Psychological Bulletin, 137(1), 68-96.
25. Lewis, H. B. (1971). Shame and guilt in neurosis. Psychoanalytic Review. 58(3), 419-438.
26. Lewis, M., Sullivan, M. W., Stanger, C., & Weiss, M. (1989). Self development and self-conscious emotions. Child Development, 60(1), 146-156.
27. Lovibond, P. F., & Lovibond, S. H. (1995). The structure of negative emotional states: Comparison of the Depression Anxiety Stress Scales (DASS) with the Beck Depression and Anxiety Inventories. Behaviour Research and Therapy, 33(3), 335-343.
28. Lowe, G., & Gibson, R. (2005). Depression in adolescence: New developments. The West Indian Medical Journal, 54(6), 387-391.
29. Mayor, M. (2009). Longman dictionary of contemporary English (5th ed.). Harlow, UK: Pearson Education.
30. Motan, I. (2007). Recognition of self-conscious emotions in relation to psychopathology (Unpublished Doctoral Dissertation). Middle East Technical University, Turkey. Retrieved from https://etd. lib. metu. edu. tr/ upload/12609222/index. pdf
31. Orth, U., Robins, R. W., & Soto, C. J. (2010). Tracking the trajectory of shame, guilt, and pride across the life span. Journal of Personality and Social Psychology, 99(6), 1061-1071.
32. Robins, R. W., Noftle, E. E., & Tracy, J. L. (2007). Assessing self‐conscious emotions: A review of self‐report and nonverbal measures. In J. L. Tracy, R. W. Robins, & J. P. Tangney (Eds.), The self‐conscious emotions: Theory and research (pp. 443-467). New York, NY: Guilford Press.
33. Shahnawaz, S, & Malik, J. A. (2016). Effect of shame and guilt in development of psychological distress among adolescents. FWU Journal of Social Sciences, 2(Special Issue-1), 54-63.
34. Simonsen, E., & Mortensen, E. L. (1990). Difficulties in translation of personality scales. Journal of Personality Disorders, 4(3), 290-296.
35. Sireci, S. G., Yang, Y., Harter, J., & Ehrlich, E. J. (2006). Evaluating guidelines for test adaptations a methodological analysis of translation quality. Journal of Cross-Cultural Psychology, 37(5), 557-567.
36. Stevenson, A. (2010). Oxford dictionary of English. UK: Oxford University Press.
37. Taihara, Q, & Malik, J. A. (2016). Is it adaptive or maladaptive? elaborating conditional role of shame and guilt in development of psychopathologies. Psychological Studies, 61(4), 331-339.
38. Tangney, J. P. (1991). Moral affect: The good, the bad, and the ugly. Journal of Personality and Social Psychology, 61(4), 598-607.
39. Tangney, J. P. (1992). Situational detenninants of shame and guilt in young adulthood. Personality and Social Psychology Bulletin, 18(2), 199-206.
40. Tangney, J. P. (1996). Conceptual and methodological issues in the assessment of shame and guilt. Behaviour Research and Therapy, 34(9), 741-754.
41. Tangney, J. P., & Dearing, R. L. (2002). Shame and guilt (emotions and social behavior). New York, NY: Guilford Press.
42. Tangney, J. P., & Dearing, R. L. (2003). Shame and guilt. New York, NY: Guilford Press.
43. Tangney, J. P., Miller, R. S., Flicker, L., & Barlow, D. H. (1996). Are shame, guilt, and embarrassment distinct emotions? Journal of Personality and Social Psychology, 70(6), 1256-1269.
44. Tangney, J. P., Stuewig, J., & Mashek, D. J. (2007). Moral emotions and moral behavior. Annual Review of Psychology, 58, 345-372. doi: 10. 1146/annurev. psych. 56. 091103. 070145
45. Tangney, J. P., Wagner, P. E., Hill-Barlow, D., Marschall, D. E., & Gramzow, R. (1996). Relation of shame and guilt to constructive versus destructive responses to anger across the lifespan. Journal of Personality and Social Psychology, 70(4), 797-809.
46. Tracy, J. L., & Robins, R. W. (2004). Putting the self into self-conscious emotions: A theoretical model. Psychological Inquiry, 15(2), 103-125.
47. Tracy, J. L., & Robins, R. W. (2006). Appraisal antecedents of shame and guilt: Support for a theoretical model. Personality and Social Psychology Bulletin, 32(10), 1339-1351.
48. Tracy, J. L., Robins, R. W., & Tangney, J. P. (Eds.). (2007). The self-conscious emotions: Theory and research. New York, NY: Guilford Press.