Introduction

In December 2019, the first cases of pneumonia of unknown origin were reported in Wuhan, China, later attributed to a novel coronavirus named SARS-CoV-2 [1]. Its rapid transmission and high mortality caused an unprecedented disruption in healthcare systems, as well as in social and economic structures worldwide [2-4]. Spain was among the most affected European countries during the first wave (March–May 2020), with high incidence and mortality rates [5-6].

In this context, emergency departments (EDs) experienced extreme healthcare pressure [7-9]. The initial lack of knowledge about the clinical course of the disease led to generalized hospital admissions and delayed recognition of severe complications, contributing to increased morbidity and mortality [10-11].

The biphasic and unpredictable evolution of COVID-19, with sudden deterioration in patients who initially appeared stable, further complicated clinical decision-making, resulting in unnecessary hospitalizations and inappropriate discharges [10-13].

Traditional prognostic scores (PSI, CURB-65) proved to be of limited use in this novel disease, while early predictive models often lacked robust validation [14-20].

In a setting of limited resources, simple, reproducible, and easily applicable tools were urgently needed in the emergency department to stratify risk and guide critical decisions such as hospital admission, ICU referral, or safe discharge [21-23].

This study was designed in two phases: (1) to derive and validate a clinical model for predicting 30-day mortality in high-risk patients with COVID-19 evaluated in the ED, and (2) to assess clinical outcomes in adult patients with COVID-19 pneumonia classified as low-risk by the model and discharged home from the ED.

Material and methods

First phase: model derivation in high-risk patients with COVID-19

Study design, setting and participants

A secondary analysis was performed using the multipurpose registry COVID-19_URG-HCSC. The registry has been previously described in detail in other publications [24].

Patients aged ≥18 years with probable or confirmed COVID-19 who presented to the ED of Hospital Clínico San Carlos (HCSC) between 4 September 2020 and 6 May 2021 and were classified as high-risk at triage were included for model derivation. High-risk status at triage was defined as age ≥55 years, baseline oxygen saturation <96% on arrival, or pulmonary infiltrates on chest radiograph.

Diagnosis was made by detection of SARS-CoV-2 RNA by reverse transcription polymerase chain reaction (RT-PCR) in nasopharyngeal and oropharyngeal samples. In those cases, in which the RT-PCR was repeated because the result was non-conclusive or a false-negative result was suspected, the latest result of the diagnostic test performed during the study period was selected. 

The COVID-19_URG_HCSC registry was developed according to the principles of the declaration of Helsinki for medical research in humans. The protocol was approved by the HCSC Ethics and Clinical Research Committee (Cl 20/338-E). Written informed consent was obtained from all patients. Data is reported in accordance to the TRIPOD (Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis) statement.

Clinical assessment, data collection and follow-up

Clinical assessment and treatment were left to the discretion of the attending physician. Sociodemographic, clinical, laboratory, radiological, treatment and outcome data were extracted from electronic medical records. Patient follow-up was performed 30 days after the index episode to know the vital status. Information regarding death during follow-up was obtained from the patient’s hospital records, the family physician’s records, and the national registry on mortality. Additionally, patients were contacted 30 days after discharge by telephone calls in those cases where information was missing. 

Laboratory measurements

Venous blood samples were drawn via a peripheral intravenous line on ED arrival. EDTA tubes were used for whole blood and in silica and gel tubes for serum samples. The first blood sample before initiation of any therapeutic intervention was considered for analysis. Routine haematological parameters were measured using a Beckman Coulter Launches DxH 900 Hematology Analyzer. 

Candidate predictors 

A total of 16 candidate predictor variables were used for constructing the full model. Six variables were continuous variables: age, oxygen saturation on admission, glomerular filtration rate (GFR), C-reactive

protein (CRP), lactate dehydrogenase (LDH) and platelets, while 10 were binary variables: sex, hypertension, diabetes mellitus, asthma, chronic obstructive pulmonary disease (COPD), coronary artery disease (CAD), heart failure, cerebrovascular disease, cancer and dementia. All candidate predictors were selected based on matter-based knowledge, previous literature, clinical meaningfulness and availability at the setting of application (ED).

Outcome measure

The primary outcome was 30-day all-cause mortality. Outcome was adjudicated by two senior investigators accessing the regional administrative database and hospital and primary care electronic medical records or contacting through as necessary.

Sample size

Before developing the multivariable prediction model, we calculated whether our sample size would allow us to include all candidate variables with their respective modelling strategies. We followed the recommendations by Riley R et al [25]. In brief, we used the pmsampsize R package to calculate the minimum sample size required to develop a multivariable prediction model for a binary outcome, considering 35 candidate predictor parameters, an outcome prevalence of 0.14 (14%), and a lower bound of 0.218 for the new model’s R-squared value. The model’s R-squared value was taken from the adjusted Cox-Snell R-squared value from an existing prediction model (4 C mortality score). The minimum sample size required for the new model development was 1,194 patients with 180 events. 

Derivation of the prediction model

All patients were used for deriving the model. We first constructed a multivariable logistic regression model that included all 16 candidate variables. To avoid dichotomizing continuous variables into a few discrete, ordered levels and to avoid imposing linearity, we modelled all 6 continuous variables using restricted cubic splines to allow for non-linearity in their relationship with the outcome [26]. Age and oxygen saturation were modelled with five spline knots while GFR, LDH, CRP and platelets were modelled with 3 spline knots. Knots were placed per default at the percentiles of each variable marginal distribution, following Harrell’s recommendations [27]. Model reduction using backward elimination based on the Akaike information criterion was performed. The apparent performance of the model was calculated.

Internal validation of the prediction model

To account for model overfitting (optimism), internal validation with a bootstrapping procedure was performed [28]. Briefly, in each bootstrap sample, the entire model derivation process was repeated. We determined the performance of the model developed from each bootstrap sample in the bootstrap sample (training data) and in the original full data-set sample (test data). The difference between training and test data was calculated, thereby accounting for optimism. This optimism is averaged among the 500 repetitions and subtracted from the apparent performance obtained in the derivation phase, thus obtaining the optimism-corrected estimate performance (bias-corrected AUC, bias-corrected slope and bootstrap calibration plot). Furthermore, to protect ourselves from overfitting, a shrinkage coefficient (the optimism-corrected slope) was uniformly applied to the coefficients of the predictors in the multivariable model, obtaining a final multivariable shrinkage model. 

The function of the final reduced optimism-corrected

model was extracted and used to calculate predicted probabilities based on the shrunk coefficients.

A web-based risk calculator for 30-day all-cause death in COVID-19 patients was developed using R Shiny.

Statistical analysis

Continuous variables were presented as mean (standard deviation [SD]) or median (interquartile range [IQR]), as appropriate. Categorical variables were expressed as numbers and percentages. T-test or Mann-Whitney-U test was applied for comparison of continuous variables and Pearson’s chi-square test or Fisher’s exact test for comparison of categorical variables, as appropriate. Median biomarker concentrations were compared among groups by using the Mann-Whitney-U test. Confidence intervals of proportions were computed using Wilson’s methods.

Performance of the model was determined with model discrimination using bias-corrected AUC and model calibration with a calibration plot comparing observed with predicted probabilities [29]. Confidence intervals (CI) of AUCs were calculated according to DeLong. 

All hypothesis testing was two-tailed and p values

< 0.05 were considered statistically significant. Data analysis was performed using Stata^® version 15.1 (Stata Corp, College Station, TX, USA) and R, version 3.6.3 (R foundation for Statistical Computing, Vienna, Austria).

Phase 2. Validation of the model in low-risk patients with COVID-19 pneumonia

Study design, setting and participants

A secondary analysis was conducted using the COVID-19_URG-HCSC registry. Patients aged ≥18 years with confirmed SARS-CoV-2 pneumonia, classified as low-risk according to a previously developed model, and discharged from the ED of HCSC between 4 September 2020 and 6 May 2021 were included.

Diagnosis was based on clinical, laboratory and radiological criteria, and confirmed by RT-PCR for SARS-CoV-2 in naso- or oropharyngeal samples. In cases of inconclusive results, the last available test was considered.

Clinical assessment, data collection and follow-up

All patients underwent electrocardiography and full blood tests (complete blood count, coagulation, renal and liver profile, electrolytes, and acute phase reactants). Discharge decisions were made by the attending emergency physician, considering the risk model and clinical criteria: baseline oxygen saturation ≥95%, adequate oral tolerance, caregiver availability and feasibility of telephone follow-up.

Structured follow-up was carried out by an emergency physician through telephone calls for 6–14 days, recording adverse events, need for antibiotic prescription, and clinical progress. In addition, 30-day outcomes after discharge were assessed.

Sociodemographic, clinical, laboratory, radiological and therapeutic data were collected from electronic health records. Primary outcome variables were all-cause mortality, re-attendance at the emergency department, and hospitalisation within 30 days. Outcome adjudication was performed by two senior investigators through review of administrative databases and electronic medical records.

Statistical analysis

Descriptive statistics were applied (mean ± SD or median [IQR], depending on distribution). Comparisons between groups were performed using Student’s t test or Mann–Whitney U test for quantitative variables, and chi-squared or Fisher’s exact test for qualitative variables. A two-sided p-value <0.05 was considered statistically significant. Analyses were performed using STATA^® v15.1 and R v3.6.3.

Results

Phase 1: model derivation in high-risk patients with COVID-19

During the study period, 3,816 patients were assessed for suspected COVID-19, of whom 2,644 met the inclusion criteria (612 with high clinical suspicion and 2,032 with microbiological confirmation).

Triage cut-off values were defined as age ≥55 years, baseline oxygen saturation <96%, or the presence of pulmonary infiltrates on chest radiography, classifying patients as high-risk. A total of 2,436 patients fulfilled these criteria and underwent detailed clinical, laboratory and radiological evaluation (Figure 1). Table 1 summarises the baseline characteristics of these patients.

The final prediction model included age, baseline oxygen saturation, estimated glomerular filtration rate, lactate dehydrogenase (LDH), C-reactive protein (CRP), platelet count, and dementia (Supplementary material-Annex 1). The diagnostic accuracy was very high (apparent AUC of 0.918 [95% CI: 0.902-0.933], optimism corrected of 0.908) (Figure 2a). Calibration after shrinkage showed a strong agreement between predicted and observed risk throughout a wide range of predicted probabilities, as shown by the flexible calibration curve near the diagonal (Figure 2b). The predicted probability distribution is shown in Supplementary material Figure 3.

To enhance clinical applicability, an interactive web-based tool was developed and is available at: https://predicovid.shinyapps.io/RISK_MODEL_COVID/ (Supplementary material Figure 4).

Phase 2: Validation of the model in low-risk patients with COVID-19 pneumonia

A total of 565 patients with SARS-CoV-2 pneumonia were included. Table 2 compares their baseline characteristics with those of the high-risk cohort from phase 1 (n=1,961).

Telephone follow-up lasted a median of 6–15 days. The median lowest oxygen saturation recorded during follow-up was 96% (IQR 95–97). Antibiotics were prescribed in 100 patients (17.7%), mainly cefditoren (54.1%), fluoroquinolones (17.1%), azithromycin (17.1%) and amoxicillin/clavulanic acid (6.9%), with a median of 9 days (IQR 7–12) from symptom onset to prescription.

Thirty-day all-cause mortality was 0. Forty-five patients (8.0%) re-attended the emergency department within 30 days, at a median of 7 days (IQR 2–10) after the initial visit; all were discharged home. New diagnoses were established in 10 patients (22.2%), most frequently radiological progression (n=4). Additional treatment was prescribed in 22 cases (48.9%), mainly antipyretics (39.3%), corticosteroids (10.7%) and antibiotics (10.7%).

Overall, 71 patients (12.6%) required hospitalisation within 30 days, with a median time of 3 days (IQR 2–4) from the initial visit and a median length of stay of 7 days (IQR 6–9). Respiratory failure (76.1%) and radiological progression (15.5%) were the most frequent admission diagnoses. Twelve patients (16.9%) required ICU care, with a median ICU stay of 5 days (IQR 2.8–6.5). All hospitalised patients were eventually discharged home.

Discussion

This study documents the care experience of ED in a tertiary hospital in an urban setting during the first wave of the COVID-19 pandemic in Madrid, a period characterised by high clinical uncertainty and severe pressure on healthcare resources. In this context, the systematic collection of clinical data became crucial, not only to describe the clinical course of patients but also to serve as the basis for developing predictive models to support decision-making in scenarios of high care demand. In addition, the implementation of alternative strategies, such as home-based management in selected cohorts of patients with COVID-19 pneumonia, proved to be an innovative and feasible option that contributed to alleviating hospital overcrowding without compromising patient safety.

This work presents the development of the risk model PrediCOVID, which demonstrated excellent discriminative ability and appropriate calibration for 30-day all-cause mortality among high-risk patients with COVID-19 attending the ED. This predictive capacity was particularly relevant during the first wave, in a context of limited knowledge regarding the clinical progression of the disease and marked variability in its course. The model design, based on minimal routinely collected ED variables, together with its integration into a user-friendly web application, represented an innovative contribution that facilitated implementation at triage and during the initial clinical assessment. Its use as a decision-support tool enabled more efficient resource allocation, prioritisation of patients at higher risk, and safe discharge of those at low-risk, thereby reducing unnecessary hospital admissions, relieving pressure on healthcare services, and secondarily lowering healthcare costs.

Our findings are consistent with those reported in other European regions, where the reorganisation of care pathways and the prioritisation of resources proved essential to cope with hospital overload [30-32].

Accumulated experience has shown that the structured capture of clinical data facilitates the identification of risk factors for mortality and complications, as documented in Spanish multicentre studies that identified advanced age, cardiovascular comorbidities, lymphopenia, and elevated D-dimer levels as independent predictors of poor outcomes. [33]. Such structured information forms the foundation for predictive models capable of stratifying patients at admission and guiding therapeutic decisions [34].

Several predictive models were developed during the pandemic with the aim of improving risk stratification in patients with SARS-CoV-2 infection. Notable among these is the COVID-19 SEIMC score, derived from a retrospective multicentre study in 127 Spanish hospitals, which provided national representativeness but was limited by insufficient characterisation of comorbidities and the exclusive use of 30-day mortality as the outcome [35]. The 4C Mortality Score, developed in the United Kingdom in a large prospective cohort of over 57,000 patients, demonstrated good ability to predict in-hospital mortality, although it did not include relevant comorbidities or critical outcomes such as ICU admission or mechanical ventilation [18]. The Priority Score, proposed by the SEMI-COVID-19 network in Spain from a registry of 132 hospitals, aimed to predict progression to critical illness using easily obtainable variables at ED presentation. Although its simplicity favours immediate applicability, the absence of analytical biomarkers or radiological findings, as well as lack of post-discharge follow-up, limit its scope [36]. Finally, the PANDEMYC score, developed in Madrid, demonstrated good discriminative ability for predicting in-hospital mortality using simple and accessible ED variables [37]. Beyond these specific models, traditional scores for community-acquired pneumonia, such as the Pneumonia Severity Index (PSI) and CURB-65, were also explored during the pandemic. While widely validated in non-pandemic settings, they were not designed for COVID-19 and therefore showed limitations in predicting complex outcomes [38-39]. Systematic reviews have highlighted that many available models suffer from methodological limitations, scarce external validation, and difficulties in applicability outside their development context [40], underscoring the need for multicentre registries and regular updating of predictive tools [41-42].

The evaluation of home-based management for patients with mild to moderate COVID-19 pneumonia in this study represents another relevant contribution. This approach proved feasible and safe in selected cohorts, contributing to hospital decongestion during periods of maximum demand. Similar results have been reported in international series, where structured follow-up and telemedical support were crucial to ensure patient safety [43-44]. However, other authors have stressed the need for strict inclusion criteria and efficient mechanisms for hospital readmission, to avoid complications in patients initially classified as low-risk [45-47]. 

Telemedicine (TM) had already been employed in the follow-up and treatment of various infectious diseases [48-49]. During the SARS-CoV-2 pandemic, faced with the saturation of healthcare systems, its use experienced a notable resurgence. Predictive models found in TM a strategically, taking advantage of its inherent benefits: reduced risk of exposure and transmission by minimising travel and physical contact; decreased consumption of personal protective equipment; support for safe discharges from the ED through structured follow-up; and reduced healthcare burden and costs compared with conventional hospital admission [50-51].

Funding

None to declare. 

Conflicts of Interest

The authors declare no conflict of interest.

References

1. Wang C, Horby PW, Hayden FG, Gao GF. A novel coronavirus outbreak of global health concern. Lancet. 2020;395(10223):470-3. Available from: https://doi.org/10.1016/S0140-6736(20)30185-9
2. Li Q, Guan X, Wu P, Wang X, Zhou L, Tong Y, et al. Early transmission dynamics in Wuhan, China, of novel coronavirus-infected pneumonia. N Engl J Med. 2020;382(13):1199-207. Available from: https://doi.org/10.1056/NEJMoa2001316
3. Phelan AL, Katz R, Gostin LO. The novel coronavirus originating in Wuhan, China: challenges for global health governance. JAMA. 2020;323(8):709-10. Available from: https://doi.org/10.1001/jama.2020.1097
4. Sanche S, Lin YT, Xu C, Romero-Severson E, Hengartner N,
5. Ke R. High contagiousness and rapid spread of severe acute respiratory syndrome coronavirus 2. Emerg Infect Dis. 2020;26(7):1470-7. Available from: https://doi.org/10.3201/eid2607.200282
6. Henríquez J, Gonzalo-Almorox E, García-Goñi M, Paolucci F. The first months of the COVID-19 pandemic in Spain. Health Policy Technol. 2020;9(4):560-74. Available from: https://doi.org/10.1016/j.hlpt.2020.08.013
7. Garcia-Carretero R, Vazquez-Gomez O, Gil-Prieto R, Gil-de-Miguel A. Hospitalization burden and epidemiology of the COVID-19 pandemic in Spain (2020-2021). BMC Infect Dis. 2023;23(1):476. Available from: https://doi.org/10.1186/s12879-023-08454-y
8. Font-Cabrera C, Juvé-Udina ME, Adamuz J, Diaz Membrives M, Fabrellas N, Guix-Comellas EM. Activity, triage levels and impact of the pandemic on hospital emergency departments: a multicentre cross-sectional study. J Adv Nurs. 2025;81(3):1332-42. Available from: https://doi.org/10.1111/jan.16332
9. Font-Cabrera C, Adamuz J, Juvé-Udina ME, Sánchez M,
10. Mateos-Dávila A, Sarria-Guerrero JA, et al. Degree of compliance of hospital emergency departments with the recommended standards and their evolution during the SARS-CoV-2 pandemic. J Nurs Manag. 2025;2025:4228788. Available from: https://doi.org/10.1155/jonm/4228788
11. Quah LJJ, Tan BKK, Fua TP, Wee CPJ, Lim CS, Nadarajan G, et al. Reorganising the emergency department to manage the COVID-19 outbreak. Int J Emerg Med. 2020;13(1):32. Available from: https://doi.org/10.1186/s12245-020-00294-w
12. Rodríguez-Madrid MN, Pastor-Moreno G, Albert-Lopez E, Pastor-Valero M. “You knew you had to be there, it had to be done”: experiences of health professionals who faced the COVID-19 pandemic in one public hospital in Spain. Front Public Health. 2023;11:1089565. Available from: https://doi.org/10.3389/fpubh.2023.1089565
13. Wexner SD, Cortés-Guiral D, Spinelli A, Mayol J. Lessons learned and experiences shared from the front lines: Milan and Madrid. Am Surg. 2020;86(6):577-84. Available from: https://doi.org/10.1177/0003134820924396
14. Ramos-Martínez A, Parra-Ramírez LM, Morrás I, Carnevali M, Jiménez-Ibañez L, Rubio-Rivas M, et al. Frequency, risk factors, and outcomes of hospital readmissions of COVID-19 patients. Sci Rep. 2021;11(1):13733. Available from: https://doi.org/10.1038/s41598-021-93076-0
15. Alfonso Viguria U, Casamitjana N. Early interventions and impact of COVID-19 in Spain. Int J Environ Res Public Health. 2021;18(8):4026. Available from: https://doi.org/10.3390/ijerph18084026
16. González Del Castillo J. Keys to interpreting predictive models for the patient with COVID-19. Emergencias. 2021;33(4):251-3.
17. Chang SC, Grunkemeier GL, Goldman JD, Wang M, McKelvey PA, Hadlock J, et al. A simplified pneumonia severity index (PSI) for clinical outcome prediction in COVID-19. PLoS One. 2024;19(5):e0303899. Available from: https://doi.org/10.1371/journal.pone.0303899
18. Armiñanzas C, Arnaiz de Las Revillas F, Gutiérrez Cuadra M, Arnaiz A, Fernández Sampedro M, González-Rico C. Usefulness of the COVID-GRAM and CURB-65 scores for predicting severity in patients with COVID-19. Int J Infect Dis. 2021;108:282-8. Available from: https://doi.org/10.1016/j.ijid.2021.05.048
19. Berenguer J, Borobia AM, Ryan P, Rodríguez-Baño J, Bellón JM, Jarrín I, et al; COVID-19@Spain and COVID@HULP Study Groups. Development and validation of a prediction model for 30-day mortality in hospitalised patients with COVID-19: the COVID-19 SEIMC score. Thorax. 2021;76(9):920-9. Available from: https://doi.org/10.1136/thoraxjnl-2020-216001
20. Knight SR, Ho A, Pius R, Buchan I, Carson G, Drake TM, et al; ISARIC4C investigators. Risk stratification of patients admitted to hospital with COVID-19 using the ISARIC WHO Clinical Characterisation Protocol: development and validation of the 4C Mortality Score. BMJ. 2020;370:m3339. Available from: https://doi.org/10.1136/bmj.m3339
21. Martínez-Lacalzada M, Viteri-Noël A, Manzano L, Fabregate M, Rubio-Rivas M, García SL, et al; SEMI-COVID-19 Network. Predicting critical illness on initial diagnosis of COVID-19 based on easily obtained clinical variables: development and validation of the PRIORITY model. Clin Microbiol Infect. 2021;27(12):1838-44. Available from: https://doi.org/10.1016/j.cmi.2021.07.006
22. Carriel J, Muñoz-Jaramillo R, Bolaños-Ladinez O, Heredia-
23. Villacreses F, Menéndez-Sanchón J, Martin-Delgado J; en representación del grupo de investigadores COVID-EC. CURB-65 como predictor de mortalidad a 30 días en pacientes hospitalizados con COVID-19 en Ecuador: estudio COVID-EC. Rev Clin Esp. 2022;222(1):37-41. Available from: https://doi.org/10.1016/j.rce.2020.10.001
24. Alberta Health Services. COVID-19 Scientific Advisory Group Rapid Response Report: Patient risk stratification and admission [Internet]. Alberta: Alberta Health Services; 2020 [Consulta: 17 Sep 2025]. Available from: https://www.albertahealthservices.ca/assets/info/ppih/if-ppih-covid-19-rapid-review-patient-risk-stratification-and-admission.pdf
25. Pantazopoulos I, Tsikrika S, Kolokytha S, Manos E, Porpodis K. Management of COVID-19 patients in the Emergency Department. J Pers Med. 2021;11(10):961. Available from: https://doi.org/10.3390/jpm11100961
26. Cherian R. Clinical risk stratification in COVID-19: the need for a revised approach? Pulm Circ. 2021;11(1):2045894020988630. Available from: https://doi.org/10.1177/2045894020988630
27. Martín-Sánchez FJ, Del Toro E, Cardassay E, Valls Carbó A, Cuesta F, Vigara M, et al. Clinical presentation and outcome across age categories among patients with COVID-19 admitted to a Spanish Emergency Department. Eur Geriatr Med. 2020;11(5):829-41. Available from: https://doi.org/10.1007/s41999-020-00359-2
28. Riley RD, Ensor J, Snell KIE, Harrell FE Jr, Martin GP, Reitsma JB, et al. Calculating the sample size required for developing a clinical prediction model. BMJ. 2020;368:m441. Available from: https://doi.org/10.1136/bmj.m441
29. Lopez-Ayala P, Riley RD, Collins GS, Zimmermann T. Dealing with continuous variables and modelling non-linear associations in healthcare data: practical guide. BMJ. 2025;390:e082440. Available from: https://doi.org/10.1136/bmj-2024-082440
30. Harrell FE. Regression modeling strategies. 2nd ed. Springer; 2015. Available from: https://doi.org/10.1007/978-3-319-19425-7
31. Moons KG, Altman DG, Reitsma JB, Ioannidis JP, Macaskill P, Steyerberg EW, et al. Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): explanation and elaboration. Ann Intern Med. 2015;162(1):W1-73. Available from: https://doi.org/10.7326/M14-0698
32. Van Calster B, McLernon DJ, van Smeden M, Wynants L, Steyerberg EW; Topic Group ‘Evaluating diagnostic tests and prediction models’ of the STRATOS initiative. Calibration: the Achilles heel of predictive analytics. BMC Med. 2019;17(1):230. Available from: https://doi.org/10.1186/s12916-019-1466-7
33. Docherty AB, Harrison EM, Green CA, Hardwick HE, Pius R, Norman L, et al; ISARIC4C investigators. Features of 20 133 UK patients in hospital with COVID-19 using the ISARIC WHO Clinical Characterisation Protocol: prospective observational cohort study. BMJ. 2020;369:m1985. Available from: https://doi.org/10.1136/bmj.m1985
34. Excoffier JB, Salaün-Penquer N, Ortala M, Raphaël-Rousseau M, Chouaid C, Jung C. Analysis of COVID-19 inpatients in France during first lockdown of 2020 using explainability methods. Med Biol Eng Comput. 2022;60(6):1647-58. Available from: https://doi.org/10.1007/s11517-022-02540-0
35. Ziebart C, Kfrerer ML, Stanley M, Austin LC. A digital-first health care approach to managing pandemics: scoping review of pandemic self-triage tools. J Med Internet Res. 2023;25:e40983. Available from: https://doi.org/10.2196/40983
36. Vela E, Carot-Sans G, Clèries M, Monterde D, Acebes X, Comella A, et al. Development and validation of a population-based risk stratification model for severe COVID-19 in the general population. Sci Rep. 2022;12(1):3277. Available from: https://doi.org/10.1038/s41598-022-07138-y
37. Tacconelli E, Gorska A, Carrara E, Davis RJ, Bonten M, Friedrich AW, et al. Challenges of data sharing in European COVID-19 projects: a learning opportunity for advancing pandemic preparedness and response. Lancet Reg Health Eur. 2022;21:100467. Available from: https://doi.org/10.1016/j.lanepe.2022.100467
38. Berenguer J, Borobia AM, Ryan P, Rodríguez-Baño J, Bellón JM, Jarrín I, et al; COVID-19@Spain and COVID@HULP Study Groups. Development and validation of a prediction model for 30-day mortality in hospitalised patients with COVID-19: the COVID-19 SEIMC score. Thorax. 2021;76(9):920-9. Available from: https://doi.org/10.1136/thoraxjnl-2020-216001
39. Martínez-Lacalzada M, Viteri-Noël A, Manzano L, Fabregate M, Rubio-Rivas M, García SL, et al; SEMI-COVID-19 Network. Predicting critical illness on initial diagnosis of COVID-19 based on easily obtained clinical variables: development and validation of the PRIORITY model. Clin Microbiol Infect. 2021;27(12):1838-44. Available from: https://doi.org/10.1016/j.cmi.2021.07.006
40. Bermejo-Martin JF, Almansa R, Torres A, González-Rivera M, Kelvin DJ. COVID-19 as a cardiovascular disease: the potential role of chronic endothelial dysfunction. Cardiovasc Res. 2020;116(10):e132-3. Available from: https://doi.org/10.1093/cvr/cvaa140
41. Fine MJ, Auble TE, Yealy DM, Hanusa BH, Weissfeld LA, Singer DE, et al. A prediction rule to identify low-risk patients with community-acquired pneumonia. N Engl J Med. 1997;336(4):243-50. Available from: https://doi.org/10.1056/NEJM199701233360402
42. Lim WS, van der Eerden MM, Laing R, Boersma WG, Karalus N, Town GI, et al. Defining community acquired pneumonia severity on presentation to hospital: an international derivation and validation study. Thorax. 2003;58(5):377-82. Available from: https://doi.org/10.1136/thorax.58.5.377
43. Wynants L, Van Calster B, Collins GS, Riley RD, Heinze G, Schuit E, et al. Prediction models for diagnosis and prognosis of COVID-19: systematic review and critical appraisal. BMJ. 2020;369:m1328. Available from: https://doi.org/10.1136/bmj.m1328 
44. Buttia C, Llanaj E, Raeisi-Dehkordi H, Kastrati L, Amiri M, Meçani R. Prognostic models in COVID-19 infection that predict severity: a systematic review. Eur J Epidemiol. 2023;38(4):355-72. Available from: https://doi.org/10.1007/s10654-023-00973-x
45. Jit M, Ainslie K, Althaus C, Caetano C, Colizza V, Paolotti D, et al. Reflections on epidemiological modeling to inform policy during the COVID-19 pandemic in Western Europe, 2020-23. Health Aff (Millwood). 2023;42(12):1630-6. Available from: https://doi.org/10.1377/hlthaff.2023.00688
46. Jaulmes L, Yordanov Y, Descamps A, Durand-Zaleski I, Dinh A, Jourdain P, et al. Effectiveness and medicoeconomic evaluation of home monitoring of patients with mild COVID-19: Covidom Cohort Study. J Med Internet Res. 2023;25:e43980. Available from: https://doi.org/10.2196/43980
47. Wurzer D, Spielhagen P, Siegmann A, Gercekcioglu A, Gorgass J, Henze S, et al. Remote monitoring of COVID-19 positive high-risk patients in domestic isolation: a feasibility study. PLoS One. 2021;16(9):e0257095. Available from: https://doi.org/10.1371/journal.pone.0257095
48. Douillet D, Penaloza A, Mahieu R, Morin F, Chauvin A, Gennai S, et al; Hospitalization or Outpatient Management of Patients With SARS-CoV-2 Infection Study Group. Outpatient management of patients with COVID-19: multicenter prospective validation of the Hospitalization or Outpatient Management of Patients With SARS-CoV-2 Infection Rule to discharge patients safely. Chest. 2021;160(4):1222-31. Available from: https://doi.org/10.1016/j.chest.2021.05.008
49. Crilly D, Shakeshaft P, Marks M, Logan S, Cutfield T. Evaluation of a remote monitoring service for patients with COVID-19 discharged from University College London Hospital. PLoS One. 2023;18(7):e0284997. Available from: https://doi.org/10.1371/journal.pone.0284997
50. Van den Berg R, Meccanici C, de Graaf N, van Thiel E, Schol-Gelok S. Starting home telemonitoring and oxygen therapy directly after emergency department assessment appears to be safe in COVID-19 patients. J Clin Med. 2022;11(23):7236. Available from: https://doi.org/10.3390/jcm11237236
51. Parmar P, Mackie D, Varghese S, Cooper C. Use of telemedicine technologies in the management of infectious diseases: a review. Clin Infect Dis. 2015;60(7):1084-94. Available from: https://doi.org/10.1093/cid/ciu1143
52. Monkowski D, Rhodes LV, Templer S, Kromer S, Hartner J, Pianucci K, et al. A retrospective cohort study to assess the impact of an inpatient infectious disease telemedicine consultation service on hospital and patient outcomes. Clin Infect Dis. 2020;70(5):763-70. Available from: https://doi.org/10.1093/cid/ciz293
53. Young JD, Abdel-Massih R, Herchline T, McCurdy L, Moyer KJ, Scott JD, et al. Infectious Diseases Society of America position statement on telehealth and telemedicine as applied to the practice of infectious diseases. Clin Infect Dis. 2019;68(9):1437-43. Available from: https://doi.org/10.1093/cid/ciy907
54. Eron L. Telemedicine: the future of outpatient therapy? Clin Infect Dis. 2010;51 Suppl 2:S224-30. Available from: https://doi.org/10.1086/653524