Test and item information of oral health literacy adult questionnaire: An item response theory study in Himachal Pradesh

Deepak Gurung; Vinay Kumar Bhardwaj; Shailee Fotedar

doi:10.4103/cjhr.cjhr_94_22

ORIGINAL ARTICLE

Year : 2022 | Volume : 9 | Issue : 4 | Page : 246-251

Test and item information of oral health literacy adult questionnaire: An item response theory study in Himachal Pradesh

Deepak Gurung, Vinay Kumar Bhardwaj, Shailee Fotedar
Department of Public Health Dentistry, HP Government Dental College and Hospital, Shimla, Himachal Pradesh, India

Date of Submission	10-Sep-2022
Date of Decision	01-Nov-2022
Date of Acceptance	06-Nov-2022
Date of Web Publication	17-Mar-2023

Correspondence Address:
Deepak Gurung
Department of Public Health Dentistry, HP Government Dental College and Hospital, Shimla, Himachal Pradesh - 171 001
India

Source of Support: None, Conflict of Interest: None

Check

DOI: 10.4103/cjhr.cjhr_94_22

Abstract

Context: Item response theory (IRT), is a psychometric measure of trait considering each response positioned on a continuum. Aim: Assessment of item and test information Oral Health Literacy Adults Questionnaire (OHL-AQ) using IRT in the patient visiting the tertiary institution of Himachal Pradesh. Settings and Design: A descriptive cross-sectional study conducted on patients visiting the outpatient department. Methods: Data were obtained from the participant and recorded on a structured schedule using OHL-AQ. Statistical Analysis: The two-assumption essential for IRT are unidimensionality and local independence of items. Unidimensional dichotomous IRT consists of three models and three parameters of difficulty discrimination and guessing. Item response interpretation is based on item characteristic curves, test characteristic curve (TCC), and test information function, and analyses were conducted using the statistical software package STATA version 14. Results: The selection of 3PL model is based on the likelihood ratio test which is higher (χ² = 133.62, P < 0.0001) than other models (χ² = 130.41, P < 0.0001). The parameter estimate of pseudo-guessing is 0.02 indicating mild degree of guessing and represent the smallest probability of correct response. The TCC of the study shows that 95% of the randomly selected respondent scored between 3.23 and 14. The median probability of correct response is 8.16 when both difficulty and discrimination parameter is equal. Conclusion: IRT is an estimated probability of a response to a given item. Our study showed that OHL-AQ measures higher levels of OHL more precisely compared to lower levels. The OHL-AQ scale is the precise measure of both high and low levels of OHL.

Keywords: Item characteristic curves, item response theory, oral health literacy, oral health literacy-adults questionnaire, parameter logistic model, test information function

How to cite this article:
Gurung D, Bhardwaj VK, Fotedar S. Test and item information of oral health literacy adult questionnaire: An item response theory study in Himachal Pradesh. CHRISMED J Health Res 2022;9:246-51

How to cite this URL:
Gurung D, Bhardwaj VK, Fotedar S. Test and item information of oral health literacy adult questionnaire: An item response theory study in Himachal Pradesh. CHRISMED J Health Res [serial online] 2022 [cited 2023 Apr 1];9:246-51. Available from: https://www.cjhr.org/text.asp?2022/9/4/246/371949

Introduction

Oral health literacy (OHL) is an integral part of health literacy. OHL is the predictor of oral health status for the given determinants, especially in developing countries where the burden of oral disease is very high.^[1] Assessment of OHL holds paramount importance in oral health promotion programs. Various OHL scales have been extensively used in the assessment of OHL.^[2] Oral Health Literacy-Adults Questionnaire (OHL-AQ) is one such psychometric measurement tool, measuring the OHL developed by Naghibi Sistani et al.^[3] in 2013. The use of OHL-AQ is limited and the unique construction which aligns itself to the cognitive development of literacy. OHL-AQ consists of 17 items with six items in reading, four items of numeracy, two items of listening, and five items of decision-making. Hence, each item in the OHL-AQ scale provides information about the item based on the difficulty and response based on the different levels of the trait. This scale and item information is well reported by item response theory (IRT) than classical test theory (CTT).^[4]

IRT is a psychometric measure of trait considering each response positioned on a continuum.^[5] This psychometric measurement is dependent on ability assessment. The response to each item shows variability and standard error of measurement (SEM) is based on these characteristics of the item. The smaller the SEM for item closer is the observed value to the expected values of the estimated parameter, indicating better precision. The assumption of constant SEM in CTT, provides less information on the variability of item in the scale.^[6] This makes IRT adequate and appropriate method to assess the variability of the item parameter in given continuum of measurement and is completely group invariant. This lack of group variance in IRT is an added advantage over CTT. Hence, IRT provides information about both item and respondent.

Pattanaik et al.^[4] have reported item and scale properties of the OHL-AQ using IRT but do report its lack of generalizability in other population. With this background, the need arises to further validate the test/scale and item information provided by OHL-AQ in other population using IRT. Hence, we aim to assess the item and test information OHL-AQ using IRT in the patient visiting the tertiary institution of Himachal Pradesh.

Methods

The source of data for this cross-sectional study was patients visiting the outpatient department of tertiary institution of Himachal Pradesh. Inclusion criteria included adults visiting the outpatient department of the tertiary institute aged between 18 and 65 years who are willing to participate in the study. Exclusion criteria: (1) participants not willing to participate in the study, (2) participants with any physical or medical condition that affects their participation. The sampling technique used was convenience sampling. As there is no consensus on a specific number of sample size, a better conservative approach was considered and a total sample of 350 participants was taken as advocated by Comrey and Lee.^[7]

The information obtained from the participant was recorded on a self-administered schedule, except for the listening part which was read out to the patient. Data collection was done for 3 months. Correct answer for item was scored as “1” and incorrect answer for item was scored as “0” with the OHL total scores ranging from 0 to 17. OHL-AQ scores were categorized into three groups ‒ Inadequate (0–9), marginal (10–11), and adequate (12–17) OHL.^[3],[4] The permission was obtained from the competent higher authority. Written informed consent was obtained from all participants for the present study which was voluntary and anonymous.

Statistical analysis

The two-assumption essentials for IRT are unidimensionality and local independence of items. The selection of best model is dependent on model coefficients, statistical testing, and model fit criteria.^[4],[8] Unidimensional dichotomous IRT consists of three models. The first model called 1 parameter logistic (1PL) model measures a single parameter of difficulty and constant in the probability equation is fixed for all the item. This was introduced by Georg Rasch in 1960. The second model called 2 parameter logistic (2PL) model measures two parameters of difficulty and discrimination. The third model called 3 parameter logistic (3PL) model measures three parameters of difficulty, discrimination, and guessing/pseudo-guessing. This was introduced by Allan Birnbaum in 1968. The constant in the probability equation is variable for items.^{[9],[10],[11]} In 1PL, discrimination and guessing are constant whereas in 2PL guessing is constant.^[12] The selection of the best model was based on comparison of various models. The item difficulty value ranges from −4 representing easy item and +4 representing difficult item and deviation from this indicates improper functioning of item.^[13] Similarly, for the discrimination the value ranges from 0.5 to 2 and higher values means higher discrimination.

IRT is not based on the ability of the respondent but on the probability of correct response at different levels of ability represented in the form of S-curve called item characteristic curve (ICC)/item response function.^[13],[14] The shape of the curve is a logistic function and ICC is the fundamental unit of IRT. The test characteristic curve (TCC) shows the relationship between the expected scores with observed OHL scores. The test information function (TIF) determines the reliability and provides both scale/test and item information. Hence, item response interpretation is based on ICCs, TCC, and TIF. IRT analyses were conducted using the statistical software package STATA version 14 (College Station, TX, USA: Stata Corp LP).

Results

The Horn's parallel analysis shows that it considers three factors based on criteria of adjusted component/factor >1 as shown in [Figure 1]. The adjusted Eigenvalue of component or factor 1 (3.09) was greater than factor 2 (1.21). Further, the assumption of unidimensionality is established and ensures that single latent construct influences item responses and local independence are obtained. This is also supported by the slope of all items which was not >4 in the discriminatory parameter of the 3PL model. The selection of 3PL model is based on likelihood ratio test which is higher (χ² = 133.62, P < 0.0001) than other models (χ² = 130.41, P < 0.0001). The parameter estimate of pseudo-guessing is 0.02 indicating mild degree of guessing and representing the smallest probability of correct response. Hence, even with the low level of given trait, a minimum of 2% chance due to probable guessing is possible in responding on any given item of OHL-AQ. For the 17 items of OHL-AQ, the total score ranges from 0 to 17, and the minimum expected score due to pseudo-guessing is 0.34 (17 × 0.02).^[15]

Figure 1: Horns parallel analysis for principal components. Three factors considered based on criteria of adjusted component/factor >1

Click here to view

The threshold or difficulty parameter of OHL-AQ items ranges from −1.60 to 3.86 which was lowest for item 4 and highest for item 6 as shown in [Table 1] and [Figure 2]. The slope or discriminatory parameter of OHL-AQ items ranges from 0.27 to 3.65 which was lowest for item 6 and highest for item 10 as shown in [Table 1] and [Figure 2]. Discriminatory parameter is like item‒total correlation and the values of all the 17 items in OHL-AQ are above the set criteria of >0.5 value. Hence, all items were within the moderate, high, and very high discriminatory level except for items 1 and 6 which had low discriminatory levels as per Baker^[13] criteria. TCC represents the expected score at different levels of trait. The TCC of the study shows that 95% of the randomly selected respondent to score between 3.23 and 14 as shown in [Figure 3]. Further, the median probability of correct response is 8.16 when both difficulty and discrimination parameters are equal. The TIF shows higher information when difficulty of item is closer to given trait, discriminatory parameter is high and guessing parameter is smaller. TIF of OHL-AQ represented precise higher information of item at a slightly high level of OHL as shown in [Figure 4].

Table 1: Parameter estimate of 1, 2, and 3 parameter logistic model

Click here to view

Figure 2: Item characteristic curve of 3 parameter logistic model with difficulty and discriminatory parameter. Item 6 shows highest difficulty and lowest discriminatory levels and item 10 has the highest discriminatory/slope parameter

Click here to view

Figure 3: Test characteristic curve of expected score for the given level of oral health literacy

Click here to view

Figure 4: Test information function of scale information for the given oral health literacy

Click here to view

Maximum information is provided by items 2, 8, 13, and 17, whereas minimum information is provided by items 7 and 16 as shown in [Table 2]. The proportion of correct response was highest for item 4 and lowest for item 7. The standard error of all items varies from 2.04 to 2.67. The internal consistency or homogeneity of reliability with Cronbach's alpha was 0.71, significant at P < 0.001.^[16]

Table 2: Information provided by each item of Oral Health Literacy Adults Questionnaire within the study sample

Click here to view

Discussion

IRT is a probability model and the information is cumulative. IRT provides item-level information in comparison to CTT that provides test/scale-level information. Scale information is the sum of all the individual item information that varies with the trait level, subjected to variability in precision for some, not for others. Further, the scale/test information depends on the number of items and quality of items.^[8] The item information is the sum of all responses to all the given items.

The best-fitting model for this study was 3PL model in contrast to the 2PL model in the Pattanaik et al.'s^[4] study. This selection was based on difficulty and discriminatory parameters improved from 2PL model to 3PL model in the study. The proportion of correct responses was lower for items of OHL-AQ in our study compared to Pattanaik et al.^[4] except for item 10 and item 11. Item 4 of OHL-AQ had the highest correct response with lowest difficulty parameter in our study compared to item 12 of Pattanaik et al.^[4] Item 7 of OHL-AQ had the lowest correct response in our study, but the difficulty parameter of item 6 was lowest in the 3PL model of our study. Such variability of proportion in correct responses to difficulty parameter has also been reported in other study.^[4]

Item 4 with the highest correct response reported moderate discriminatory parameter and item 7 with lowest correct response reported very high discriminatory parameter as per Baker^[13] criteria in our study. This variability of proportion in correct responses to discriminatory parameter has also been reported in other study.^[4] Item 10 in the study, is the highest discriminating in contrast to item 16, reported by Pattanaik et al.^[4] Similarly, item 6 is the lowest discriminating in the study compared to item 1, reported by Pattanaik et al.^[4] The above-observed variations could be understood by the fact that the information calculation is low when the likelihood of endorsing is high.

The OHL-AQ scale in this study provided greater information more precisely at higher level of OHL in contrast to the Pattanaik et al.'s^[4] study. The probable reason for this difference is attributed to the different study setting, where developed countries have higher oral health promotion programs compared to the developing country as in our study. When likelihood response is changing a lot, little information is provided with high level of trait and a much of information at low level of trait evident in Pattanaik et al.'s^[4] study in developed countries. This also underscores the fact that OHL as an outcome of oral health promotion.^[17],[18] The OHL-AQ scale more precisely measures low level of OHL in developed countries and high level of OHL in developing countries although further studies are still needed.

The guessing parameter is represented on Y-intercept of ICC indicating individual with zero ability has nonzero likelihood of endorsing a response to any item by random guessing.^[15] This guessing parameter in the 3PL was reported minimally (2%) in our study.

The strength of the study was, it is one of its kind in our region using IRT in OHL-AQ scale as far as our knowledge. There were no missing data which were ensured after the completion of the questionnaire by the participants, during the data collection. The sample size of our study was adequate for the given range of 250–500 when item response study for the OHL-AQ was done and reported by Pattanaik et al.^[4],[19]

The limitation is that our study being cross-sectional fails to provide necessary evidence on OHL and oral health outcome and further prospective studies with enhanced methodology are recommended. Second, over and underreporting could not ignored although the anonymity of the participants was considered.

Conclusion

IRT is an estimated probability of a response to a given item. This probability to endorse item depends on item difficulty and level of trait. Our study showed that OHL-AQ measures higher levels of OHL more precisely compared to lower levels. OHL is an important measure of oral health promotion and IRT provides an adequate test and item information about the OHL-AQ scale. Thus, OHL-AQ scale is the precise measure of both high and low levels of OHL.

Acknowledgment

The author would like to thank all the participants and Mr. Savitesh Khuswaha, Ph. D. Research Scholar, Department of Community Medicine, PGIMER, Chandigarh, for doing statistical analysis for this study.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.

References

1.	Naghibi Sistani MM, Yazdani R, Virtanen J, Pakdaman A, Murtomaa H. Determinants of oral health: Does oral health literacy matter? ISRN Dent 2013;2013:249591.
2.	Dickson-Swift V, Kenny A, Farmer J, Gussy M, Larkins S. Measuring oral health literacy: A scoping review of existing tools. BMC Oral Health 2014;14:148.
3.	Naghibi Sistani MM, Montazeri A, Yazdani R, Murtomaa H. New oral health literacy instrument for public health: Development and pilot testing. J Investig Clin Dent 2014;5:313-21.
4.	Pattanaik S, John MT, Kohli N, Davison ML, Chung S, Self K, et al. Item and scale properties of the oral health literacy adults questionnaire assessed by item response theory. J Public Health Dent 2021;81:214-23.
5.	Yang FM, Kao ST. Item response theory for measurement validity. Shanghai Arch Psychiatry 2014;26:171-7.
6.	Kolen MJ, Wang T, Lee W. Conditional standard errors of measurement for composite scores using IRT. Int J Test 2012;12:1-20.
7.	Comrey AL, Lee HB. A First Course in Factor Analysis. Hillsdale, NJ: Erlbaum; 1992.
8.	Hays RD, Morales LS, Reise SP. Item response theory and health outcomes measurement in the 21^st century. Med Care 2000;38:I28-42.
9.	Edelen MO, Reeve BB. Applying Item Response Theory (IRT) modeling to questionnaire development, evaluation, and refinement. Qual Life Res 2007;16 Suppl 1:5-18.
10.	Reise SP, Waller NG. Item response theory and clinical measurement. Annu Rev Clin Psychol 2009;5:27-48.
11.	Streiner DL, Norman GR, Cairney J. Health Measurement Scales a Practical Guide to Their Development and Use. 5^th ed. Oxford: Oxford Press University; 2014.
12.	Harris D. Comparison of 1, 2, and 3 parameter IRT models. Educ Meas 1989;8:35-41.
13.	Baker F. The basics of item response theory. In: ERIC Clearinghouse on Assessment and Evaluation. College Park, MD: University of Maryland; 2001.
14.	Flynn PM, John MT, Sistani MM. Confirmation of the unidimensional structure of the oral health literacy Adults questionnaire. Int Dent J 2019;69:207-13.
15.	STATA Item Response Theory Reference Manual Release 17. Available from: https://www.stata.com/manuals/irt.pdf. [Last accessed on 2022 Sep 02].
16.	Nunnally JC, Bernstein IH. Psychometric Theory. 3^rd ed. New York: McGraw-Hill; 1994.
17.	Nutbeam D. Health literacy as a public health goal: A challenge for contemporary health education and communication strategies into the 21^st century. Health Promot Int 2000;15:259-67.
18.	Sørensen K, Van den Broucke S, Fullam J, Doyle G, Pelikan J, Slonska Z, et al. Health literacy and public health: A systematic review and integration of definitions and models. BMC Public Health 2012;12:80.
19.	Holmes F, Brian FF. A comparison of estimation techniques for IRT models with small samples. Appl Meas Educ 2019;32:77-96.