Evaluation of the WinROP system for identifying retinopathy of prematurity in Czech preterm infants

Aims. Retinopathy of Prematurity (ROP) is a potentially serious condition that can afflict preterm infants. Timely and correct identification of individuals at risk of developing a serious form of ROP is therefore of paramount importance. WinROP is an online system for predicting ROP based on birth weight and weight increments. However, the results vary significantly for various populations. It has not been evaluated in the Czech population. This study evaluates the test characteristics (specificity, sensitivity, positive and negative predictive values) of the WinROP system in Czech preterm infants. Methods. Data on 445 prematurely born infants included in the ROP screening program at the University Hospital Ostrava, Czech Republic, were retrospectively entered into the WinROP system and the outcomes of the WinROP and regular screening were compared. Results. All 24 infants who developed high-risk (Type 1 or Type 2) ROP were correctly identified by the system. The sensitivity and negative predictive values for this group were 100%. However, the specificity and positive predictive values were substantially lower, resulting in a large number of false positives. Extending the analysis to low risk ROP, the system did not provide such reliable results. Conclusions. The system is a valuable tool for identifying infants who are not likely to develop high-risk ROP and this could help to substantially reduce the number of preterm infants in need of regular ROP screening. It is not suitable for predicting the development of less serious forms of ROP which is however in accordance with the declared aims of the WinROP system.

IGF-1 plays a key role in the development of multiple tissues and affects processes including vascularization.During gestation, transplacental transfer of maternal IGF-1 into the foetus is responsible for maintaining the optimum levels of IGF-1.In prematurely born infants, this maternal supplement is missing, which leads to the pathologically low levels of IGF-1 in new-borns 17,21,22 .In the first stage of ROP, low levels of IGF-1 lead to low activation of VEGF and thus to obliteration of retinal capillary vessels.Later, in the second stage, the levels of IGF-1 are conversely too high and contribute towards the uncontrolled neoangiogenesis 15,17,[19][20][21] .
Over the last decade, numerous studies have confirmed the relationship between low IGF-1 levels in prematurely born infants, extremely low birth weight, low weight increments in the postnatal period and development of ROP requiring treatment [23][24][25][26][27][28][29][30][31][32] .Based on the results of weekly weight increments in Swedish infants, an online system WinROP was created aiming at timely prediction of a high risk of serious ROP development 25,32,33 .
Several studies that evaluated ROP in different populations have been published with varying results 23,25,26,[32][33][34][35][36][37][38][39] .In this paper, we would like to contribute to this discussion by presenting an evaluation of the WinROP system based on data acquired at our department.

METHODS
We performed a retrospective analysis of a group of 445 infants who were born before gestational week 32 (GW 32) at the University Hospital Ostrava, Czech Republic, between 1. 11. 2011 and 31.10. 2014 and automatically included in the ROP screening program.The screening was carried out by an experienced ophthalmologist using a RetCam3™ ophthalmic imaging system (Clarity Medical Systems Inc., Pleasanton, CA, USA).
Retrospectively, the anonymised data of new-borns (date of birth, birth weight and gestational week at birth) were entered into the WinROP system.A unique ID number was assigned to each child by the system, the number was recorded in the new-born's documentation and used as the only identifier thereafter.Subsequently, weekly weight increments were entered into the system up to week 40 of postconceptional age or until alarm was triggered by the system.All data were exported from the system into MS Excel (Microsoft Corporation, Redmond, Washington, USA) after the screening completion, compared with the results of screening and statistically evaluated.Sensitivity, specificity, positive and negative predictive values were calculated where possible, along with 95% Clopper-Pearson confidence intervals.
For evaluation purposes, all preterm infants were divided into four groups according to the maximum ROP stage reached: Type 1 and Type 2 in accordance with the criteria outlined in the ETROP study (Type 1 ROP -infants requiring treatment within 48-72 h after the diagnose, Type 2 ROP -not requiring immediate intervention but close and frequent monitoring) (ref. 40).All new-borns who developed ROP that was not serious enough to be classified as the Type 1 or Type 2 were included in a group denominated as Type 3. The last group then consisted of prematurely born infants without ROP.For a more detailed assessment, the infants were also divided according to their gestational age at birth and birth weight into the following subgroups: extremely preterm (born at less than 29 weeks of pregnancy) and very preterm (29-32 weeks) new-borns, infants with low birth weight (1500 g and more), very low (1000 -1499 g) and extremely low birth weight (under 1000 g).

RESULTS
Fig. 1 shows the distribution of patients in our group.Only 3.6% of our patients developed ROP Type 1 and 1.8% developed ROP Type 2, i.e., altogether 5.4% developed a high risk ROP requiring or potentially requiring treatment.25.6% of infants were diagnosed with ROP not fulfilling criteria for Type 1 or 2 ROP and 69% developed no ROP.No ROP Type 1 or Type 2 was found among the infants with birth weight over 1500 g and only 1 patient with ROP Type 1 was present in the group with very low birth weight.In contrast, 14% of infants with extremely low birth weight developed ROP Type 1 requiring treatment and 8% ROP Type 2 potentially requiring treatment.A similar situation was found when the infants were divided according to their gestation age at birth -only two (1%) very preterm infants (gestation age at birth 29-32 weeks) developed ROP Type 1 and one developed ROP Type 2 while 11% of infants born before gestational week 29 developed ROP Type 1 and 6% ROP Type 2.
The mean age of the new-borns at WinROP alarm was 29.7 weeks (standard deviation 1.8 weeks), the mean age of ROP diagnosis by the ophthalmologist was 34.3 weeks (SD 2.1 weeks).On average, the WinROP system identified the patients who developed ROP as high-risk 4.7 weeks (SD 2.3 weeks) before the diagnosis was established via standard means.The intervention, if necessary, followed the WinROP alarm on average by 10.2 weeks (SD 4.2 weeks).
An overview of the groups including the data on the number of WinROP alarms and actual presence or absence of ROP of a certain Type is presented in Table 1.Notably, the results for ROP Type 1 and ROP Type 2 are very similar in that WinROP system triggered alarm in all cases with these types of ROP and, accordingly, no infants with these ROP types passed undetected by the system.test characteristics were therefore calculated for that group.Sensitivity for both types of ROP with high risk of permanent damage to the retina (Type 1 and Type 2) was 100% in all weight and maturity groups where it could be calculated.Similarly, the negative predictive value was 100% in all instances.Due to small numbers of patients with Type 1 or Type 2 ROP in the "Very preterm" and "Very low birth weight" subgroups, however, the sensitivity values are not reliable as indicated by the wide 95% Clopper-Pearson confidence intervals.Specificity and positive predictive values in the entire group and subgroups for Type 1 and Type 2 ROP were however relatively lower, particularly in the "Extremely low birth weight" and "Extremely preterm" subgroups.
Including the patients with Type 3 ROP in the analysis significantly reduced the sensitivity and negative predictive value of the "Extremely low birth weight" group apparently dropped to 0, which is however an unreliable value caused by small number of patients as indicated by the wide confidence interval.

DISCUSSION
As expected, the incidence of both ROP and serious ROP (Type 1 or 2) increased with lower birth weight and lower gestation age at birth.Altogether, 31% of infants born before GW 32 developed ROP in our study, only 14% of whom (4% of the total number) required treatment.Despite the fact that an exact comparison with Cryo-ROP and ETROP studies claiming that 65 -68% of children with birth weight below 1250 g developed ROP (ref. 41,42) is not possible due to different entry criteria, we can say that the incidence in our group was relatively low in comparison.Most likely, the difference can be related to the improvement in care for prematurely born infants over the last decade.Our results also correspond with results of more recent studies reporting an incidence in infants with birth weight below 1500 g ranging from 15% to 45%.According to those studies, intervention was only necessary in 2 -15% of infants 2,4,7-9,14,43 .
The WinROP system was originally developed for detection of preterm infants needing or potentially needing treatment.In this respect, WinROP results were very reliable in our study group.All neonates who eventually developed Type 1 or Type 2 ROP were identified as "high risk" by the WinROP system before ROP was diagnosed by standard means.On average, the alarm was triggered 10.2 weeks before ROP progressed into a stage when treatment was necessary (minimum 2, maximum 21 weeks).On the other hand, the specificity and positive predictive values were lower, in particular in the groups of extremely preterm neonates and those with extremely low birth weight, and we recorded relatively high numbers of false positive alarms.
In our opinion, the negative predictive value could be considered the most valuable characteristic of the test.We can safely say that in our group, no infant undetected as "high risk" by WinROP developed Type 1 or Type 2 ROP.In effect, this could mean that if we reduced the number of ophthalmologic examinations in infants who are not identified by the system (or even removed them from the ROP screening programme), no harm would be caused to any of the infants.Even if they had developed ROP, it would regress spontaneously.This could significantly reduce the number of examinations, which would be a great benefit to the prematurely born infants (who would avoid the stress of examinations) as well as to the healthcare system (effecting costs savings).In our group, 299 out of 445 patients (67.1%) could be safely removed from the ROP screening programme in this way.
Such use of WinROP would however have to be confirmed by further studies, on a larger number of patients.9] ) while studies from China, Mexico, Brazil or Korea [34][35][36][37] reported sensitivity values ranging from 85% to 95%.The WinROP system correctly identified all 24 infants requiring or potentially requiring treatment (Type 1 or Type 2).Two factors may be responsible for such differences.One of these is the factor of population -the WinROP algorithm was based on the results from Sweden.There are likely similarities between the Swedish and Czech populations as far as general stature and growth are concerned.It is possible that with Asian or Latin American populations being generally of smaller stature, the WinROP system might need adjustment to the particular populations to provide valid results 34 .
The other factor contributing to the different WinROP outcomes may lie in differences in the quality of care for prematurely born neonates in Europe and the USA compared to countries where such care may not be so readily available.The importance of this factor is supported by the fact that in our study (as well as in the studies from Sweden or USA), practically no neonates born after GW 29 developed serious ROP that would require treatment 26,32,33,[38][39] while multiple cases requiring treatment were reported in the studies from Brazil, Mexico or China [35][36][37] .
In attempts to use the WinROP system for predicting development of ROP (type 1 -3), all test characteristics decreased significantly.This is in accordance with the principal focus of WinROP system, which was developed to identify prematurely born infants with Type 1 or Type 2 ROP.It is possible that including other variables in the system (such as oxygen therapy, respiration distress syndrome, etc.) could improve the detection rates even for lower risk ROP.On the other hand, the effect of such improvement is questionable since these low-risk patients need no treatment and as long as WinROP is capable of reliable identification of neonates who are not in risk of developing Type 1 or Type 2 ROP, it could still be a valuable tool for reducing the number of prematurely born infants needing regular ROP screening.

CONCLUSION
WinROP system reliably identified all prematurely born infants who developed Type 1 or Type 2 ROP, i.e., all neonates who needed treatment or who were in high risk of developing treatment-requiring ROP.Although the specificity and positive predictive values were not particularly high, the 100% sensitivity and 100% negative predictive value showed that WinROP is a valuable tool for identification of preterm new-borns who are not in risk of developing serious ROP.Providing this is confirmed in larger studies, this could lead to a significant reduction of the number of preterm infants needing ROP screening, obviating stressful examinations and with economic benefit to the healthcare systems.

Table 2
presents the test characteristics of WinROP for the entire group and subgroups.No infants with ROP were present in the group with low birth weight and no Fig. 1.Distribution of prematurely born infants throughout the entire study group and in subgroups according to the presence and type of ROP.

Table 1 .
Results of WinROP alarm compared to diagnosis established by standard means.