Observer variation in target volume delineation of lung cancer related to radiation oncologist-computer interaction: a 'Big Brother' evaluation.Citation formats

  • External authors:
  • Roel J H M Steenbakkers
  • Joop C Duppen
  • Isabelle Fitton
  • Kirsten E I Deurloo
  • Lambert Zijp
  • Apollonia L J Uitterhoeve
  • Patrick T R Rodrigus
  • Gijsbert W P Kramer
  • Johan Bussink
  • Katrien De Jaeger
  • José S A Belderbos
  • Augustinus A M Hart
  • Peter J C M Nowak
  • Marcel van Herk
  • Coen R N Rasch

Standard

Observer variation in target volume delineation of lung cancer related to radiation oncologist-computer interaction: a 'Big Brother' evaluation. / Van Herk, Marcel; Steenbakkers, Roel J H M; Duppen, Joop C; Fitton, Isabelle; Deurloo, Kirsten E I; Zijp, Lambert; Uitterhoeve, Apollonia L J; Rodrigus, Patrick T R; Kramer, Gijsbert W P; Bussink, Johan; De Jaeger, Katrien; Belderbos, José S A; Hart, Augustinus A M; Nowak, Peter J C M; van Herk, Marcel; Rasch, Coen R N.

In: Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology, Vol. 77, No. 2, 11.2005.

Research output: Contribution to journalArticle

Harvard

Van Herk, M, Steenbakkers, RJHM, Duppen, JC, Fitton, I, Deurloo, KEI, Zijp, L, Uitterhoeve, ALJ, Rodrigus, PTR, Kramer, GWP, Bussink, J, De Jaeger, K, Belderbos, JSA, Hart, AAM, Nowak, PJCM, van Herk, M & Rasch, CRN 2005, 'Observer variation in target volume delineation of lung cancer related to radiation oncologist-computer interaction: a 'Big Brother' evaluation.', Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology, vol. 77, no. 2. https://doi.org/10.1016/j.radonc.2005.09.017

APA

Van Herk, M., Steenbakkers, R. J. H. M., Duppen, J. C., Fitton, I., Deurloo, K. E. I., Zijp, L., ... Rasch, C. R. N. (2005). Observer variation in target volume delineation of lung cancer related to radiation oncologist-computer interaction: a 'Big Brother' evaluation. Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology, 77(2). https://doi.org/10.1016/j.radonc.2005.09.017

Vancouver

Van Herk M, Steenbakkers RJHM, Duppen JC, Fitton I, Deurloo KEI, Zijp L et al. Observer variation in target volume delineation of lung cancer related to radiation oncologist-computer interaction: a 'Big Brother' evaluation. Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology. 2005 Nov;77(2). https://doi.org/10.1016/j.radonc.2005.09.017

Author

Van Herk, Marcel ; Steenbakkers, Roel J H M ; Duppen, Joop C ; Fitton, Isabelle ; Deurloo, Kirsten E I ; Zijp, Lambert ; Uitterhoeve, Apollonia L J ; Rodrigus, Patrick T R ; Kramer, Gijsbert W P ; Bussink, Johan ; De Jaeger, Katrien ; Belderbos, José S A ; Hart, Augustinus A M ; Nowak, Peter J C M ; van Herk, Marcel ; Rasch, Coen R N. / Observer variation in target volume delineation of lung cancer related to radiation oncologist-computer interaction: a 'Big Brother' evaluation. In: Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology. 2005 ; Vol. 77, No. 2.

Bibtex

@article{d65ef6df6ea345969cb46798a1a3e2bb,
title = "Observer variation in target volume delineation of lung cancer related to radiation oncologist-computer interaction: a 'Big Brother' evaluation.",
abstract = "BACKGROUND AND PURPOSE: To evaluate the process of target volume delineation in lung cancer for optimization of imaging, delineation protocol and delineation software. PATIENTS AND METHODS: Eleven radiation oncologists (observers) from five different institutions delineated the Gross Tumor Volume (GTV) including positive lymph nodes of 22 lung cancer patients (stages I-IIIB) on CT only. All radiation oncologist-computer interactions were recorded with a tool called 'Big Brother'. For each radiation oncologist and patient the following issues were analyzed: delineation time, number of delineated points and corrections, zoom levels, level and window (L/W) settings, CT slice changes, use of side windows (coronal and sagittal) and software button use. RESULTS: The mean delineation time per GTV was 16 min (SD 10 min). The mean delineation time for lymph node positive patients was on average 3 min larger (P = 0.02) than for lymph node negative patients. Many corrections (55{\%}) were due to L/W change (e.g. delineating in mediastinum L/W and then correcting in lung L/W). For the lymph node region, a relatively large number of corrections was found (3.7 corr/cm2), indicating that it was difficult to delineate lymph nodes. For the tumor-atelectasis region, a relative small number of corrections was found (1.0 corr/cm2), indicating that including or excluding atelectasis into the GTV was a clinical decision. Inappropriate use of L/W settings was frequently found (e.g. 46{\%} of all delineated points in the tumor-lung region were delineated in mediastinum L/W settings). Despite a large observer variation in cranial and caudal direction of 0.72 cm (1 SD), the coronal and sagittal side windows were not used in 45 and 60{\%} of the cases, respectively. For the more difficult cases, observer variation was smaller when the coronal and sagittal side windows were used. CONCLUSIONS: With the 'Big Brother' tool a method was developed to trace the delineation process. The differences between observers concerning the delineation style were large. This study led to recommendations on how to improve delineation accuracy by adapting the delineation protocol (guidelines for L/W use) and delineation software (double window with lung and mediastinum L/W settings at the same time, enforced use of coronal and sagittal views) and including FDG-PET information (lymph nodes and atelectasis).",
author = "{Van Herk}, Marcel and Steenbakkers, {Roel J H M} and Duppen, {Joop C} and Isabelle Fitton and Deurloo, {Kirsten E I} and Lambert Zijp and Uitterhoeve, {Apollonia L J} and Rodrigus, {Patrick T R} and Kramer, {Gijsbert W P} and Johan Bussink and {De Jaeger}, Katrien and Belderbos, {Jos{\'e} S A} and Hart, {Augustinus A M} and Nowak, {Peter J C M} and {van Herk}, Marcel and Rasch, {Coen R N}",
year = "2005",
month = "11",
doi = "10.1016/j.radonc.2005.09.017",
language = "English",
volume = "77",
journal = "Radiotherapy & Oncology",
issn = "0167-8140",
publisher = "Elsevier BV",
number = "2",

}

RIS

TY - JOUR

T1 - Observer variation in target volume delineation of lung cancer related to radiation oncologist-computer interaction: a 'Big Brother' evaluation.

AU - Van Herk, Marcel

AU - Steenbakkers, Roel J H M

AU - Duppen, Joop C

AU - Fitton, Isabelle

AU - Deurloo, Kirsten E I

AU - Zijp, Lambert

AU - Uitterhoeve, Apollonia L J

AU - Rodrigus, Patrick T R

AU - Kramer, Gijsbert W P

AU - Bussink, Johan

AU - De Jaeger, Katrien

AU - Belderbos, José S A

AU - Hart, Augustinus A M

AU - Nowak, Peter J C M

AU - van Herk, Marcel

AU - Rasch, Coen R N

PY - 2005/11

Y1 - 2005/11

N2 - BACKGROUND AND PURPOSE: To evaluate the process of target volume delineation in lung cancer for optimization of imaging, delineation protocol and delineation software. PATIENTS AND METHODS: Eleven radiation oncologists (observers) from five different institutions delineated the Gross Tumor Volume (GTV) including positive lymph nodes of 22 lung cancer patients (stages I-IIIB) on CT only. All radiation oncologist-computer interactions were recorded with a tool called 'Big Brother'. For each radiation oncologist and patient the following issues were analyzed: delineation time, number of delineated points and corrections, zoom levels, level and window (L/W) settings, CT slice changes, use of side windows (coronal and sagittal) and software button use. RESULTS: The mean delineation time per GTV was 16 min (SD 10 min). The mean delineation time for lymph node positive patients was on average 3 min larger (P = 0.02) than for lymph node negative patients. Many corrections (55%) were due to L/W change (e.g. delineating in mediastinum L/W and then correcting in lung L/W). For the lymph node region, a relatively large number of corrections was found (3.7 corr/cm2), indicating that it was difficult to delineate lymph nodes. For the tumor-atelectasis region, a relative small number of corrections was found (1.0 corr/cm2), indicating that including or excluding atelectasis into the GTV was a clinical decision. Inappropriate use of L/W settings was frequently found (e.g. 46% of all delineated points in the tumor-lung region were delineated in mediastinum L/W settings). Despite a large observer variation in cranial and caudal direction of 0.72 cm (1 SD), the coronal and sagittal side windows were not used in 45 and 60% of the cases, respectively. For the more difficult cases, observer variation was smaller when the coronal and sagittal side windows were used. CONCLUSIONS: With the 'Big Brother' tool a method was developed to trace the delineation process. The differences between observers concerning the delineation style were large. This study led to recommendations on how to improve delineation accuracy by adapting the delineation protocol (guidelines for L/W use) and delineation software (double window with lung and mediastinum L/W settings at the same time, enforced use of coronal and sagittal views) and including FDG-PET information (lymph nodes and atelectasis).

AB - BACKGROUND AND PURPOSE: To evaluate the process of target volume delineation in lung cancer for optimization of imaging, delineation protocol and delineation software. PATIENTS AND METHODS: Eleven radiation oncologists (observers) from five different institutions delineated the Gross Tumor Volume (GTV) including positive lymph nodes of 22 lung cancer patients (stages I-IIIB) on CT only. All radiation oncologist-computer interactions were recorded with a tool called 'Big Brother'. For each radiation oncologist and patient the following issues were analyzed: delineation time, number of delineated points and corrections, zoom levels, level and window (L/W) settings, CT slice changes, use of side windows (coronal and sagittal) and software button use. RESULTS: The mean delineation time per GTV was 16 min (SD 10 min). The mean delineation time for lymph node positive patients was on average 3 min larger (P = 0.02) than for lymph node negative patients. Many corrections (55%) were due to L/W change (e.g. delineating in mediastinum L/W and then correcting in lung L/W). For the lymph node region, a relatively large number of corrections was found (3.7 corr/cm2), indicating that it was difficult to delineate lymph nodes. For the tumor-atelectasis region, a relative small number of corrections was found (1.0 corr/cm2), indicating that including or excluding atelectasis into the GTV was a clinical decision. Inappropriate use of L/W settings was frequently found (e.g. 46% of all delineated points in the tumor-lung region were delineated in mediastinum L/W settings). Despite a large observer variation in cranial and caudal direction of 0.72 cm (1 SD), the coronal and sagittal side windows were not used in 45 and 60% of the cases, respectively. For the more difficult cases, observer variation was smaller when the coronal and sagittal side windows were used. CONCLUSIONS: With the 'Big Brother' tool a method was developed to trace the delineation process. The differences between observers concerning the delineation style were large. This study led to recommendations on how to improve delineation accuracy by adapting the delineation protocol (guidelines for L/W use) and delineation software (double window with lung and mediastinum L/W settings at the same time, enforced use of coronal and sagittal views) and including FDG-PET information (lymph nodes and atelectasis).

U2 - 10.1016/j.radonc.2005.09.017

DO - 10.1016/j.radonc.2005.09.017

M3 - Article

VL - 77

JO - Radiotherapy & Oncology

JF - Radiotherapy & Oncology

SN - 0167-8140

IS - 2

ER -