alexa Structural Validity and Reliability of the Dutch Older Patient in Acute Care Survey (OPACS), Measuring Nurses Attitude towards Older Patients | Open Access Journals
ISSN: 2167-7182
Journal of Gerontology & Geriatric Research
Make the best use of Scientific Research and information from our 700+ peer reviewed, Open Access Journals that operates with the help of 50,000+ Editorial Board Members and esteemed reviewers and 1000+ Scientific associations in Medical, Clinical, Pharmaceutical, Engineering, Technology and Management Fields.
Meet Inspiring Speakers and Experts at our 3000+ Global Conferenceseries Events with over 600+ Conferences, 1200+ Symposiums and 1200+ Workshops on
Medical, Pharma, Engineering, Science, Technology and Business

Structural Validity and Reliability of the Dutch Older Patient in Acute Care Survey (OPACS), Measuring Nurses Attitude towards Older Patients

Jeroen Dikken1*, Jita G Hoogerduijn1, Sharon Klaassen2 and Marieke J Schuurmans1,3

1Research Group Care for the Chronically Ill, Faculty of Health Care, University of Applied Sciences Utrecht, Utrecht, The Netherlands

2Independent Statistician, The Netherlands

3Department of Rehabilitation, Nursing Sciences and Sports, Utrecht University, Utrecht, The Netherlands

*Corresponding Author:
Jeroen Dikken
Research Group Care for the Chronically Ill
Faculty of Health Care
University of Applied Sciences Utrecht
Heidelberglaan 7, P.O. Box 85182 - 3508 AD
3584CS Utrecht, The Netherlands
Tel: +31 (0)6 18 842 336

Received date: January 27, 2017; Accepted date: February 06, 2017; Published date: February 08, 2017

Citation: Dikken J, Hoogerduijn JG, Klaassen S, Schuurmans MJ (2017) Construct Validity and Reliability of the Dutch Older Patient in Acute Care Survey (OPACS), Measuring Nurses’ Attitude towards Older Patients. J Gerontol Geriatr Res 6:393. doi: 10.4172/2167-7182.1000393

Copyright: © 2017 Dikken J, et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Visit for more related articles at Journal of Gerontology & Geriatric Research


Background: In clinical practice, nurses’ attitudes regarding older patients are important in relation to quality of care. The Older People in Acute Care Survey (OPACS) is an instrument measuring hospital nurses attitudes regarding older patients and validated in Australia and the USA. The OPACS is translated in Dutch language and content validity of this translation is previously assessed, presenting questionable results. Measurement instruments, however, cannot be “validated” based on content validity evidence alone. Judgmental evidence and statistical analysis should be combined to fully evaluate content domain definition and representation and guide further development.

Objective: Assess structural validity and reliability to fully evaluate the OPACS for use in the Netherlands, complementing previous conducted content validity results.

Design: Cross-sectional.

Setting: Three general hospitals in the Netherlands.

Participants: 201 registered nurses.

Methods: Confirmatory factor analysis was used to assess the structural validity. Reliability was assessed with Cronbach’s alpha.

Results: OPACS Section A (measuring practice experiences) demonstrated to have acceptable structural validity- and good reliability outcomes after exclusion of two items (model fit: x² [df=537]=8475.40, p <0.001, CFI=0.96, TLI=0.96, RMSEA=0.21; Cronbach’s alpha=0.82). Section B (measuring general opinion) demonstrated to have inadequate structural validity outcomes (model fit: x² [df=1127]=9200.29, p<0.001, CFI=0.68, TLI=0.67, RMSEA=0.15). None of the items contributed significant to the factor and therefore no further analysis could be performed (range p(>|z|)= 0.551 -0 .788).

Conclusion: Even though structural validity for section A was acceptable, content validity scores of a majority of items in this subscale were low, resulting in questionable use of this subscale for the Dutch context. The findings of this study, in relation to the earlier findings regarding content validity, justify the conclusion that use of the Dutch OPACS in clinical practice and research is not recommended. Given these findings, future research should pursue the development or (cross-cultural) validation of other instruments measuring hospital nurses attitudes towards older patients for the Dutch cultural context. Furthermore, this study demonstrated the influence of cultural differences on measurement instruments and the need for rigorous research before using a measurement instrument in a new culture or context.


OPACS; Attitude; Experience; Opinion; Netherlands; Nurses; Hospital; Cross-cultural


Worldwide, people are aging [1]. This demographic change results in an increase of older people admitted in hospitals. A growing number of registered nurses will encounter older patients in their daily work and a positive attitude is often promoted [2,3]. Healthcare professionals need to understand current attitudes regarding older patients when workforce strategies for promoting positive attitudes are to be implemented [4,5].

The Older Patient in Acute Care Survey (OPACS), developed in Australia, measures hospital nurses practice experiences and general opinion regarding older patients which are considered aspects of attitude [6,7]. The OPACS was developed using focus groups with 16 nurses discussing their experience of caring for older patients in the acute care setting. The final OPACS consisted of 86 items related to 13 different aspects influencing the nursing care of older patients. Verbal statements regarding these 13 aspects are scored on a 5 point Likert scale ranging from strongly disagree – strongly agree [6]. The OPACS has been translated and content validity is assessed in the USA. Results demonstrated excellent content validity scores [8]. Furthermore, structural validity and reliability outcomes for the American OPACS proved to be good [7]. After translation towards the Dutch language, however, a majority of items were considered “not to be relevant” by experts, resulting in low content validity scores even though translation was considered good [9]. Why the content was not considered relevant for the Netherlands, and whether or not this is reflected in the construct is unknown, making it difficult to adjust the OPACS to the Dutch context.

In literature, the concept of content validity has been controversial since its inception and it is described that although content validity is a fundamental requirement of all assessment instruments, measures cannot be considered “valid” based on content validity evidence alone [10]. Both judgmental and statistical analysis of test content provide important information regarding content- and construct validity and both approaches have their limitations. Therefore it is recommended to use both types of analysis to fully evaluate content domain definition and representation [11-13]. With assessment of the structural validity and reliability of the Dutch OPACS, content validity results of a previous study will be complemented, resulting in a full evaluation of the OPACS content and use for the Dutch cultural context.

The aim of this study is therefore evaluating the construct validity and reliability of the Dutch OPACS, complementing previous study results.



This study followed a multicenter cross-sectional design.

Setting and subjects

Data of nurses from the Netherlands were derived over a six-month period. Registered nurses working in three general hospitals located in the middle of the Netherlands were recruited and included after informed consent was obtained. Nurses were invited to participate through e-mail from their ward manager, flyers, and a message on the online hospital communication boards. Nurses completed the Dutch OPACS and several questions regarding their socio- demographic characteristics online. The study was approved by the medical review board of the University Medical Center Utrecht, the Netherlands (METC protocol number: 14-345/C).


The OPACS is originally developed in Australia [6]. It consists of two scales; section A measuring practical experience (36 items) and section B measuring general opinions towards older patient’s needs (50 items) on a 5 point Likert scale. The Australian OPACS demonstrated good face validity and high reliability scores (Kappa 0.76). The United States version of the OPACS showed a high Scale-Content Validity Index/universal agreement (S-CVI/ua) score (S-CVI/ua=0.92) [8] and good structural validity and excellent reliability scores (Cronbach’s alpha=0.93) [7]. The American OPACS was translated and validated on content in the Netherlands, demonstrating positive translation but alarming content validation results (S-CVI/average 0.62) with major differences in rating of relevance between experts (S-CVI/ua=0.13). The same Dutch OPACS was used for data collection in the present study to assess the construct validity.


Confirmatory factor analysis (CFA) was used to assess the construct validity of the Dutch OPACS. The aim of CFA is to test a hypothesized factor structure or model and assess its fit to the data. Relations of indicators (observed variables) to factors (latent variables) as well as the correlations among the latter are tested in the measurement model [14].

First, missing values were assessed to determine whether list-wise deletion could be used. Then CFA was performed for Dutch OPACS section A and section B by testing several models using Lavaan: an R package for structural equation modeling [15]. Evaluation of each model was based on considering a variety of fit measures: the X² minimum fit function test; the Comparative Fit Index (CFI); the Tucker Lewis Index (TLI) and the Root Mean Square Error of Approximation (RMSEA). Values of >0.95 for the CFI/TLI indicate a good fitting model. The RMSEA should be <0.06 indicating a good fitting model [14,16]. All analysis are performed using R [17].


Of the participating sample, only complete cases were included in this study (73.6%). The socio-demographic characteristics of respondents with missing values were not significantly different from complete cases (all p>0.05). Socio-demographic characteristics are presented in (Table 1).

  NL respondents (n= 201)
Gender, female n (%) 185 (92.0)
Missing, n 1
Age, mean (SD) 38.7 (12.3)
Highest education, n (%)  
AAS, 113 (56.2)
BSN 80 (39.8)
Masters/PhD 6 (3)
Missing, n 2 (1.0)
Years of experience, mean (SD) 16.0 (12.0)
Missing, n 5
Hours a week working, mean (SD) 26.7 (8.8)
Missing, n 1

Table 1: Sample characteristics.

Validity and reliability of the Dutch OPACS section A (Practice experiences)

In Table 2, the different CFA models assessing section A (practice experiences) are presented. The unidimensional model for the Dutch OPACS section A (Model 1) did fit the data (x2 [df=594]=9088.53, p<0.001, CFI=0.96, TLI=0.96, RMSEA=0.21). There were no items with a negative loading on the factor “practice experiences”. However, 2 items (items 20, 22) did not significantly contribute to the factor and were therefore excluded. As expected, exclusion of these two items did not worsen the model fit to the data in Model 2 (x2 [df=537]=8475.40, p<0.001, CFI= 0.96, TLI= 0.96, RMSEA= 0.21). Internal consistency was considered good (Cronbach’s alpha=0.82 [0.79 – 0.84]) (Table 2).

Model Model fit statistics Items deleted
  X2 df p CFI TLI RMSEA (95% CI)  
Model 1. 36 items, only factor structure constrained 9088.53 594 <0.001 0.96 0.96 0.21 (0.20 - 0.21) [20,22]
Model 2. 34 items, exclusion of non-significant loading of items on construct 8475.40 527 <0.001 0.96 0.96 0.21 (0.21 - 0.22)  

Table 2: Confirmatory Factor Analysis model fit statistics for OPACS-NL section A (practice experiences) 36 items.

Validity and reliability of the Dutch OPACS section B (General opinion)

The same CFA model was used to assess the Dutch OPACS section B: general opinion. The unidimensional model for the Dutch OPACS section B (Model 1) did not fit the data well (x2 [df=1127]=9200.29, p<0.001, CFI=0.68, TLI=.67, RMSEA=0.15). When looking at the items separately to assess which items should be excluded to improve the model fit to the data, it appeared that none of them contributed significant to the factor (Table 3) and therefore no further analysis could be performed (range p(>|z|)=0.551 - 0.788).

  Estimate Std Error Z-value P(>|z|)
Q1 1 - - -
Q2 -20.06 33.891 -0.592 0.554
Q3 -9.958 16.882 -0.59 0.555
Q4 0.573 2.131 0.269 0.788
Q5 26.751 45.199 0.592 0.554
Q6 30.953 52.279 0.592 0.554
Q7 8.453 14.256 0.593 0.553
Q8 10.749 18.103 0.594 0.553
Q9 -13.898 23.583 -0.589 0.556
Q10 23.934 40.431 0.592 0.554
Q11 23.611 39.897 0.592 0.554
Q12 4.598 7.967 0.577 0.564
Q13 31.201 52.612 0.593 0.553
Q14 0.655 1.934 0.339 0.735
Q15 -8.616 14.639 -0.589 0.556
Q16 -2.405 4.284 -0.561 0.574
Q17 -1.866 3.507 -0.532 0.595
Q18 -3.493 6.235 -0.56 0.575
Q19 -1.196 2.848 -0.42 0.675
Q20 -5.06 8.888 -0.569 0.569
Q21 -1.557 3.547 -0.439 0.661
Q22 -1.657 3.383 -0.49 0.624
Q23 -10.25 17.253 -0.594 0.552
Q24 -13.517 22.697 -0.596 0.551
Q25 -15.165 25.516 -0.594 0.552
Q26 -13.69 23.159 -0.591 0.554
Q27 2.459 4.563 0.539 0.590
Q28 10.073 17.208 0.585 0.558
Q29 -10.526 18.017 -0.584 0.559
Q30 16.081 27.096 0.593 0.553
Q31 -7.984 13.58 -0.588 0.557
Q32 -18.842 31.723 -0.594 0.553
Q33 -14.303 24.088 -0.594 0.553
Q34 -1.889 3.997 -0.473 0.637
Q35 -13.25 22.46 -0.590 0.555
Q36 3.412 5.8 0.588 0.556
Q37 -9.591 16.294 -0.589 0.556
Q38 -13.721 23.176 -0.592 0.554
Q39 -6.706 11.330 -0.592 0.554
Q40 -13.078 22.037 -0.593 0.553
Q41 -14.787 24.927 -0.593 0.553
Q42 22.193 37.456 0.593 0.554
Q43 -5.629 9.474 -0.594 0.552
Q44 -9.484 15.958 -0.594 0.552
Q45 -10.363 17.511 -0.592 0.554
Q46 -5.939 10.171 -0.584 0.559
Q47 -17.327 29.16 -0.594 0.552
Q48 -18.535 31.355 -0.591 0.554
Q49 -17.146 29.049 -0.59 0.555
Q50 -16.246 27.387 -0.593 0.553

Table 3: Final item loadings and test statistics for the Dutch OPACS section B (generalopinion) 50 items.


This study assessed the construct validity and reliability of the Dutch OPACS measuring practice experiences and general opinion of hospital nurses regarding older patients.

The items of section A: practice experiences, demonstrated to measure one construct. Only two items did not contribute to the construct and were therefore excluded. None of the items in OPACS section B, contributed significant to the factor general opinion, meaning that none of the items measured the construct solely making it impossible to include good items and exclude bad items using statistics.

In a previous study by van Schelven et al. [9], low content validity scores for 20 (58.8%) of the 34 items were presented for subscale A: practice experiences. By assessment of items with low content validity, several cultural reasons were found explaining the low rating by experts. First, 26 items (72.2%) mentioned a difference in care giving between old and young patients with only 5 (19.2%) of these items considered relevant. Focusing on the difference in care giving between old and young patients undermines the Dutch vision that care should be adjusted to the need of the individual patient (the same basic principle for old and young) which is taught in education and in clinical practice in the Netherlands. For example the item: “I ask older patients if they have pain more often than I ask younger patients”, with total agreement reflecting a positive attitude, is considered not to be relevant because nurses should assess pain three times a day in every hospitalized patient regardless their age according to Dutch quality systems. Second, the relation between several items and “positive or negative attitude” was unclear for experts resulting in a questionable scoring system. For example, the question: “I am more likely to speak in simple language to an older patient than to a younger patient” with total agreement reflecting a positive attitude. However, language used by nurses should always be adjusted to the individual patient, not be based on age alone because this can lead to a feeling of stereotyping by the older patient influencing the perceived quality of care [18]. Therefore, not agreeing on this item can also be explained as “good attitude” by nurses respecting the older patient and approach him/her as an adult. This makes it disputable what “good attitude” is in relation to the item as presented by the OPACS. Before this subscale can be used in clinical practice in the Netherlands, items should be reexamined, discussed and adjusted by experts on content. The number and form of adjustments needed is so rigorous that this will result in a new instrument which means that 1) it will not be comparable with the OPACS-US or any other existing instrument making cross-national comparisons impossible and 2) might not be worth the effort with other existing instruments possibly more suited to the Dutch culture in its origin. The study by van Schelven et al. [9] also presented low content validity scores for 60% of the items in section B: general opinion. Results from the content validity study by van Schelven [9], combined with this study suggest that the translated version of the OPACS in the Netherlands should not be used.

This study demonstrated that assessment of both content- and structural validity are necessary to fully comprehend the validity of an instrument in a particular culture [10]. Content validity is considered a fundamental requirement [10]. Our results support this, demonstrated by the Dutch OPACS section A which would be assess valid if only structural validity and reliability scores would have been conducted. Too often, instruments are tested only using quantitative tests to assess validity and reliability when used in different cultures, settings and groups. Our studies demonstrate that validity and reliability of instruments can differ substantial between countries emphasizing the importance for rigorous cross-cultural validation before an instrument should be used in clinical practice in different cultures and in research. Researchers should therefore always assess content validity and describe possible (cultural) differences on item and scale level, as this influences the results of the study conducted.

Some considerations regarding this study should be discussed. Missing data were not imputed and cases were excluded (even though missing values were completely at random) to maximize the validity of the item selection during the item reduction process. This is considered acceptable as no differences were found in characteristics between full cases and cases having missing values and performance of analysis was not affected by sample size. Furthermore, considering the response rate and sample size of nurses from the different centers, the representativeness (having an convenience sample) can be questioned and selection bias could have led to an overestimation of effect as nurses with interest in older patients are more likely to participate. However, this is not considered a problem as the primary focus was on structural validation of the Dutch OPACS and not an exploration of attitudes of Dutch hospital nurses in the Netherlands. Furthermore, no sample size problems were indicated in analysis of the data. Third, OPACS section A proved unidimensional. However, whether the same construct is measured in the United States as in the Netherlands is not assessed in this study. Measurement invariance between items should always be assessed before comparisons between countries can be performed [19]. With regards to the Dutch OPACS, it is likely that the Dutch subscale measures a different construct taking content validity results into account.


In conclusion, in clinical practice, identifying attitude problems is an important step to improve the quality of care for older patients [4]. However, it is important that valid and reliable instruments are used to do so. The results from this study cannot justify the use of the Dutch OPACS in clinical practice and/or research. Even though section A (measuring practice experiences) demonstrated to have good structural validity results, items measuring practice experiences are considered unclear in interpretation and scoring and therefore not ready for use in clinical practice and research in the Netherlands. Section B (measuring general opinion) also demonstrated not to be applicable for use in the Netherlands as a result of low structural validity and reliability. Although section A might have some pointers for developing a new instrument, it might not worth the effort having other instruments potentially more suited to the Dutch culture in its origin.


This work was supported by the Foundation of Innovation Alliance, Regional Attention and Action for Knowledge circulation (SIA RAAK) and by an internal grant of the University of Applied Sciences Utrecht, the Netherlands.


Select your language of interest to view the total content in your interested language
Post your comment

Share This Article

Relevant Topics

Recommended Conferences

Article Usage

  • Total views: 248
  • [From(publication date):
    February-2017 - Aug 18, 2017]
  • Breakdown by view type
  • HTML page views : 218
  • PDF downloads :30

Post your comment

captcha   Reload  Can't read the image? click here to refresh

Peer Reviewed Journals
Make the best use of Scientific Research and information from our 700 + peer reviewed, Open Access Journals
International Conferences 2017-18
Meet Inspiring Speakers and Experts at our 3000+ Global Annual Meetings

Contact Us

Agri, Food, Aqua and Veterinary Science Journals

Dr. Krish

1-702-714-7001 Extn: 9040

Clinical and Biochemistry Journals

Datta A

1-702-714-7001Extn: 9037

Business & Management Journals


1-702-714-7001Extn: 9042

Chemical Engineering and Chemistry Journals

Gabriel Shaw

1-702-714-7001 Extn: 9040

Earth & Environmental Sciences

Katie Wilson

1-702-714-7001Extn: 9042

Engineering Journals

James Franklin

1-702-714-7001Extn: 9042

General Science and Health care Journals

Andrea Jason

1-702-714-7001Extn: 9043

Genetics and Molecular Biology Journals

Anna Melissa

1-702-714-7001 Extn: 9006

Immunology & Microbiology Journals

David Gorantl

1-702-714-7001Extn: 9014

Informatics Journals

Stephanie Skinner

1-702-714-7001Extn: 9039

Material Sciences Journals

Rachle Green

1-702-714-7001Extn: 9039

Mathematics and Physics Journals

Jim Willison

1-702-714-7001 Extn: 9042

Medical Journals

Nimmi Anna

1-702-714-7001 Extn: 9038

Neuroscience & Psychology Journals

Nathan T

1-702-714-7001Extn: 9041

Pharmaceutical Sciences Journals

John Behannon

1-702-714-7001Extn: 9007

Social & Political Science Journals

Steve Harry

1-702-714-7001 Extn: 9042

© 2008-2017 OMICS International - Open Access Publisher. Best viewed in Mozilla Firefox | Google Chrome | Above IE 7.0 version