Kim, Moon, Lee, Moon, Cho, Chang, Han, and Noh: Efficacy of Mastocheck for Screening of Early Breast Cancer: Comparison with Screening Mammography

Abstract

Purpose

This study was designed to verify the effectiveness of Mastocheck, a new diagnostic tool developed with proteomics techniques using plasma proteins. In addition, checked the current state of mammography used for breast cancer screening.

Methods

A total of 305 cases were analyzed (normal 122, breast cancer 183) and used for validation after Mastocheck development. First, screening mammograms of normal patients and pre-diagnosis mammography of cancer patients were reviewed retrospectively. The results were compared with Mastocheck, a newly developed blood test. Imaging tests were blinded and analyzed by dividing the readings of breast specialists and non-breast specialists among radiologists. We confirmed how much better the results would be if only the mammography was used and if both tests were used together.

Results

The sensitivity, specificity, and accuracy of mammography alone, reviewed by non-breast specialists among radiologists, were 63.0%, 85.7%, and 71.3%, respectively. In dense breasts, the values were 59.2%, 84.8% and 69.0%, which were too low to be considered interpretable. The sensitivity, specificity and accuracy of the test was 93.9%, 83.8%, and 90.2% when using mammography and Mastocheck together. From these results, an improvement in sensitivity of about 30% and an improvement in accuracy of about 15% or more in concomitant use than mammography alone can be seen.

Conclusion

Mastocheck can be widely used for screening breast cancer, especially in dense breasts, patients with low accuracy in mammography, and patients with mammography side effects. In addition, it has the advantage of increasing the diagnosis rate when used with mammography, the current screening method of choice.

INTRODUCTION

The incidence of breast cancer is rapidly increasing worldwide, with a 20% increase between 2008 and 2012 [1,2]. In Korea, the incidence and mortality rates of breast cancer are increasing every year. The second most common cancer among women, according to the 2015 Ministry of Health and Welfare of Korea, breast cancer accounts for 18.9% of all cancers [3]. The most effective way to prevent disease is early screening. Breast cancer screening is performed by a clinician, and mammography has been the standard method for the early diagnosis and reduction of mortality. The current screening program in Korea performs breast cancer screening according to methods used in the West. However, Korean women have different characteristics of breast tissue from Western women, and it is important to develop breast cancer screening programs that are suitable for Korean women, based on study findings. Screening mammography is a very important technique for detecting breast cancer and reduces breast cancer mortality by 20–30% [4]. Mammography is known for its inadequate sensitivity to dense breast tissue, and the need for developing individual breast cancer screening guidelines for Asian women has emerged, as the frequency of such dense breasts is significantly higher in the East than in the West [5]. According to various literature, for patient’s age (young age), high density, and growth rate, the sensitivity of mammography is low at 40–50% in Asian women, including Koreans, which frequently leads to false negative results [6,7]. In addition, many disadvantages have been reported in association with the difference between the test results by the device or the inspector. It is therefore important to develop a screening method that compensates for the drawbacks of mammography and that minimizes differences in test results among devices and inspectors. Although there is no officially recommended blood test for the diagnosis of breast cancer, its value as a diagnostic or prognostic factor for various tumor markers based on blood tests is being studied [8-11]. We have developed a blood marker for breast cancer diagnosis using proteomic techniques through numerous experiments over 10 years using tissue, blood, and cell samples. This test is an early breast cancer screening algorithm using logistic regression analysis of three proteins (Carbonic anhydrase 1, CAH 1; Neural cell adhesion molecule L1-like protein, NCHL1; Apo lipoprotein C-1, APOC1) [12,13]. This test method is named Mastocheck and was approved for use in humans by the Korean Ministry of Food and Drug Safety (MFDS) in January 2019. Mastocheck is a technique that screens for the development of breast cancer through algorithmic calculations of three plasma protein levels specific to breast cancer. It can be analyzed in trace amounts of 1 μL (0.001 cc) plasma and has been approved by the MFDS for use as an in vitro diagnostic technology to help diagnose stage 0-2 breast cancer. Liquid chromatography—mass spectrometry (LC-MS) based algorithm is used for quantification of the three plasma proteins specific to breast cancer. The cut-off value is 0.0668. If higher than this, a diagnosis of cancer is made. If lower, it is judged as normal. This study was designed to confirm the current status of mammography, which is used as a screening test for breast cancer, and to verify the effectiveness of the newly developed diagnostic tool Mastocheck.

METHODS

The study analyzed a total of 305 cases (normal 122, breast cancer 183) used for verification after the development of Mastocheck. We reviewed mammography, which was done at screening in the normal group, and initial mammography, which was used for diagnosis in the breast cancer group. Based on the results of this image review, the diagnostic ability of mammography was checked and compared with that of Mastocheck. Next, when mammography and Mastocheck were used together, the degree of change in the performance was checked. To prevent bias of retrospective review of mammography, all imaging reviews were performed blindly, covering patient information, lesion area, etc. The results of the mammography were divided into the following five categories according to the degree of malignancy based on the Breast Imaging Reporting and Data System (BI-RADS). This suggests that Category 1 is a normal finding, Category 2 is an obvious non-malignant lesion, Category 3 is a nonmalignant lesion, Category 4 is an indeterminate and malignant lesion or probable malignancy, and Category 5 means highly suggestive of malignancy. Category 0 means needs additional evaluation [14]. Since each test is meaningful for differentiating malignant lesions, we classified Category 4 and 5 into positive results, that is, when the biopsy resulted in diagnosis of malignant lesions. For Category 3, positive results were included in cases where other immediate or biopsy or surgical interventions were recommended. Breast density is based on the BI-RADS classifications of almost entirely fatty (grade 1), scattered fibroglandular tissue (grade 2), relatively dense (grade 3), and extremely dense (grade 4) [15]. For analysis, relatively dense (grade 3) and extremely dense tissues (grade 4) were defined as dense breasts. Considering the high proportion of dense breasts in Asian women, including Koreans, we have identified how the test results change in high density breasts of grade 3 and grade 4. The review of the imaging test was blindly reviewed by three doctors who were radiologists but not breast specialists and classified as the non-expert group. In addition, an expert group of seven radiologists who specialize in breast imaging was asked to blindly review the same image. A subgroup analysis was also performed on how the degree of breast tissue density changes.
The data was analyzed using SPSS 23.0. The statistical analysis method used included independent t-test, 2-test, kappa system. To assess the diagnostic value of a test, operating characteristic (ROC) curve and the Area under the Curve (AUC) was used. In addition, nonparametric statistical methods such as Fisher’s exact test and Mann-Whitney U-test were performed. Significance was considered as p<0.05, 95% confidence interval. This study was approved by the Institutional Review Board of Seoul National University Hospital (IRB No. 1603-053-748, IRB No. 0512-502-163) and was conducted according to the Declaration of Helsinki.

RESULTS

We retrospectively reviewed the mammography of the 305 patients who were classified into normal (122) and patient (183) groups. The results of the review were compared with the results for Mastocheck, which is a blood test, and how they changed when used together. In the normal group, a total of 105 patients were analyzed, with the exception of 17 patients who were young or did not want to have a mammogram. In the breast cancer group, a total of 181 patients except 2 patients without mammography were analyzed.

Basic Characteristics of the cases

Among the 305 total cases, 122 (40%) were normal cases and 183 (60%) were malignant cases. The normal group consisted of BI-RADS category C1 and C2 patients, with 94 cases of C1 and 28 cases of C2 (benign mass and calcification). Cancer patients included 17 DCIS patients, 91 patients in stage 1, 74 patients in stage 2 and 1 patient in stage 3. According to histologic type, there were 148 IDCa, 11 ILCa, 3 IDCa+ILCa patients, 1 case of mucinonus cancer, and 2 cases of metaplastic cancer. The mean age of the malignancy group was 51.92±10.51 years, and the mean age of normal group was 45.87±9.04 years. (independent t-test, p<0.001) The age distribution showed that the high-risk age group accounted for 37% and 33.1% in the 40–49 and 50–59 age groups, respectively. For patients over age 59, the pattern has been monophasic, with cancer decreasing with age. There were no significant differences in the distribution of breast density between the malignant and normal groups in 284 cases where breast density was identified (Fisher’s exact test, p=0.217). On the other hand, the distribution of breast density decreased significantly with age (trend test, p<0.001) (Table 1).

Mammography alone and Mastocheck combination tests - Diagnostic value comparison

Mammography and Mastocheck were compared with biopsy results. Among 183 malignant cases, Mastocheck showed positive results in 132 cases (72.1%). The mammography showed positive results in 114 (63%) cases in the non-expert group and 145 (79.2%) in the expert group. Looking at the consistency of the two tests, Mastocheck showed a ‘moderate agreement’ with a kappa coefficient of 0.556 (p<0.001). The non-specialist group used “moderate agreement” with a kappa coefficient of 0.441 (p<0.001). The expert group showed a “substantial agreement” with a Kappa coefficient of 0.636 (p<0.001) (Table 2) [16,17].
As seen in Tables 3 and 4, the mammography readings of non-breast specialists have sensitivity, specificity and accuracy rates of 63.0%, 85.7%, and 71.3%, respectively, which are better results than other studies [18,19]. Considering that this research institute is an advanced general hospital, it may be thought that the excellence of the medical staff and the equipment may have influenced these good results. The sensitivity, specificity, and accuracy were 59.2%, 84.8% and 69.0%, respectively, when dense breasts, which are grade 3 or higher, were classified and analyzed. In the same cases who underwent mammography review, the sensitivity, specificity, and accuracy of Mastocheck were 71.6%, 85.2% and 77.0%, respectively, as confirmed by the final experiment. Even if this value is compared with the sensitivity, specificity, and accuracy of mammography, it can be confirmed that the results of the Mastocheck alone are superior to that of mammography alone. When mammography and Mastocheck were used together, the sensitivity, specificity, and accuracy were 93.9%, 75.2%, and 87.1%, and the sensitivity was more than 30% higher than that of the mammography alone. However, the combination resulted in a lower specificity than the use of mammography alone. This is because if one of the two tests results in suspicion of cancer, it is classified as a cancer suspect. This seems to be a problem that can occur with the use of multiple tests. The sensitivity, specificity and accuracy of subgroup analysis on dense breasts are 93.0%, 72.8%, and 85.4%, as shown in Table 4, with a 30% improvement in sensitivity over mammography alone. The accuracy has also been improved by 15%. The results of the breast radiologic specialists, the expert group, reveal that mammography alone satisfies the function of the screening test (Tables 5, 6). However, it is not possible to generalize the level of instrumentation in higher-level hospitals and the image reading level of breast specialists. In addition, the reading level of the breast imaging radiologist who reviewed the mammography of this study was the highest level, and the health examination center would be lower. Therefore, considering the reading level of the general examination center, the actual mammography value is lower than that shown in this study. ROC curves were used to compare diagnostic values of the five test combinations. As a result, mammography+Mastocheck (AUC 0.846) was better than mammography alone (AUC 0.641), and it was statistically significant (p<0.001) (Figure 1).

DISCUSSION

From the results, mammography can identify low levels of sensitivity, specificity and accuracy for use in screening tests. In addition, the results of experts and non-experts showed a sensitivity difference of more than 30%. This confirms that there is a large difference in the results according to the image reader, and thus, it can be seen that there is a problem in the reproducibility of the test and the objectivity of the results. It should also be taken into account that the quality of inspection results may vary depending on the level of inspection equipment. Considering that the data used in this study are the results of tests performed by the excellent equipment of general hospitals, it is possible to expect lower values than the results of this study by conducting a full survey. In fact, according to 2017 National Statistical Office and Disease Headquarters data, more than 80% of the approximately 3,200 mammography devices nationwide were owned by local hospital-level hospitals, and 46% of the total was found to be older equipment more than 10 years old. In addition, according to the data presented by the Korean Society for Breast Screen, the quality reduction rate of mammography was 36.3%, and the 2012 Korean Journal of Breast Examination data showed that 57.4% of all hospitals do not read by radiologists and do not control for quality [20,21]. Based on these data, we can question the objectivity and reliability of the mammogram, the current breast cancer screening test. It can also be expected that the test results will be significantly lower than the values shown in this study. These differences in screening equipment performance, poor quality control, and reading levels make it difficult to accept the value of mammography as a screening test. Therefore, there is much research and development on imaging devices that will replace or supplement mammography. However, there is currently no method for diagnosing or testing breast cancer through blood, and related studies are still unsuccessful. In that sense, the newly developed and licensed Mastocheck is the only blood diagnostic test for breast cancer screening. For proper screening test to be performed, the level of test should be satisfied within a certain range, with little effect from external factors such as place or person. For this reason, blood-based tests are objectively reliable tests without any difference between the tester and the reader. In addition, we confirmed that the performance was higher when used in combination with mammography. Unlike conventional tests, in which the patient has to remove clothes or apply pressure, there is no discomfort and no exposure to radiation. It is a minimally invasive method and can be performed simultaneously with other blood tests.
In conclusion, Mastocheck can be widely used as a tool for screening breast cancer, especially in dense breasts, patients with low accuracy in mammography, and patients with mammography side effects. The limitations of this study, however, were the small number of subjects and the fact that they were conducted in a single institution. In order to overcome these limitations and to ensure a higher level of evidence, we plan to continue experimenting with Mastocheck for its usefulness. In addition, the diagnostic marker will continue to be studied for application beyond diagnosis to become a useful marker for the evaluation and follow-up of cancer status. It is also anticipated that multi-center prospective studies will be initiated to obtain more clinical evidence and be used extensively.

CONFLICT OF INTEREST

The authors declare that they have no competing interests.

ACKNOWLEDGMENTS

The authors thank the following radiologists for their mammography reviews.
Su Hyun Lee, M.D., Soo-Yeon Kim, M.D., Eun Sil Kim, M.D., Su Min Ha, M.D., Jihee Kang, M.D., Ju Gang Nam, M.D., Sae Jin Park, M.D.

Figure 1.
Comparison of diagnostic accuracy when mastocheck alone, mammography (mmg) alone, and both tests combined.
jbd-7-2-59f1.tif
Table 1.
Characteristics of cases
Variable Benign (n = 122) Malignancy (n = 183) Total p-value
No. (%) No. (%) No. (%)
Age (yr)
 20–29 4 (3.7) 1 (0.5) 5 (1.7) < 0.001*
 30–39 25 (22.9) 14 (7.7) 39 (13.4)
 40–49 41 (37.6) 67 (37) 108 (37.2)
 50–59 32 (29.4) 60 (33.1) 92 (31.7)
 60–69 7 (6.4) 24 (13.3) 31 (10.7)
 70– 0 15 (8.3) 15 (7.2)
 Mean ± SD 45.87 ± 9.04 51.92 ± 10.51
Density
 Grade 1 1 (0.1) 5 (2.8) 6 (2.1) 0.217
 Grade 2 12 (11.4) 27 (15.1) 39 (13.7)
 Grade 3 49 (46.7) 94 (52.5) 143 (50.4)
 Grade 4 43 (41.0) 53 (29.6) 96 (33.8)
BI-RADS
 C1 94 (77.0) 122
 C2 28 (23.0)
Stage
 0 17 (9.3) 183
 1 91 (50.0)
 2 74 (40.2)
 3 1 (0.5)

BI-RADS=breast imaging reporting and data system.

* By independent t-test;

Density: Radiographic breast density;

Comparison of distribution of radiographic breast densities by Fisher’s exact test.

Table 2.
Results of Mastocheck and mammography (non-expert and expert group)
Variable Test results No.
Kappa coefficient (p < 0.001)
Normal Cancer
Mastocheck
 Benign 105 17 0.556
 Cancer 51 132
Mammography (non-expert*)
 Benign 90 67 0.441
 Cancer 15 114
Mammography (Expert)
 Benign 101 34 0.636
 Cancer 3 145

* non-breast specialist among the radiologist;

breast specialist among the radiologis.

Table 3.
Result of mammography alone and result of combination of mammography and Mastocheck (non-breast specialists among radiologists; non-expert group)
Radiologist (non-expert) Test result No.
Total
Normal Cancer
Mammography alone
 Benign 90 15 105
 Cancer 67 114 181
Mammography+Mastocheck
 Benign 79 26 105
 Cancer 11 170 181
Dense breast (Grade 3 and 4)
 Mammography alone
  Benign 78 14 92
  Cancer 60 87 147
 Mammography+Mastocheck
  Benign 67 25 92
  Cancer 10 137 147
Table 4.
Diagnostic value of tests (non-expert group)
Variable Sensitivity (%) Specificity (%) PPV (%) NPV (%) Accuracy (%)
Mammography alone 63.0 85.7 88.4 57.3 71.3
Mammography + Mastocheck 93.9 75.2 86.7 87.8 87.1
Dense breast (Grade 3 and 4)
 Mammography alone 59.2 84.8 86.1 56.5 69.0
 Mammography + Mastocheck 93.0 72.8 84.6 87 85.4
Table 5.
Result of mammography alone and result of combination of mammography and Mastocheck (breast specialists among radiologists; expert group)
Radiologist (expert) Test results No.
Total
Normal Cancer
Mammography alone
 Benign 98 7 105
 Cancer 35 146 181
Mammography+Mastocheck
 Benign 88 17 105
 Cancer 11 170 181
Dense breast (Grade 3 and 4)
 Mammography alone
  Benign 85 7 92
  Cancer 32 115 147
 Mammography+Mastocheck
  Benign 75 17 92
  Cancer 11 136 147
Table 6.
Diagnostic value of tests (expert group)
Variable Sensitivity (%) Specificity (%) PPV (%) NPV (%) Accuracy (%)
Mammography alone 81.1 93.3 95.4 73.7 85.6
Mammography + Mastocheck 93.9 83.8 90.9 88.9 90.2
Dense breast (Grade 3 and 4)
 Mammography alone 78.2 92.4 94.3 72.6 83.7
 Mammography + Mastocheck 92.5 81.5 88.9 87.2 88.3

REFERENCES

1. Korean Breast Cancer Society. Breast Cancer Facts & Figures 2018. Seoul: Korean Breast Cancer Society; 2018.

2. Stewart BW, Wild CP. World cancer report 2014. Lyon: International Agency for Research on Cancer; 2014.

3. Jung KW, Won YJ, Kong HJ, Lee ES. Cancer statistics in Korea: incidence, mortality, survival, and prevalence in 2015. Cancer Res Treat 2018;50:303-16.
crossref pmid pmc
4. Kim SH, Kim MH, Oh KK. Analysis and comparison of breast density according to age on mammogram between Korean and western women. J Korean Radiol Soc 2000;42:1009-1.
crossref
5. Hasegawa S, Ohnuki K, Nagakubo J, Kitami A, Oyama K, Koizumi R, et al. Breast cancer imaging by mammography: effects of age and breast composition. J Jpn Assoc Breast Cancer Screen 2003;12:101-7.
crossref
6. Mandelson MT, Oestreicher N, Porter PL, White D, Finder CA, Taplin SH, et al. Breast density as a predictor of mammographic detection: Comparison of interval- and screen-detected cancers. J Natl Cancer Inst 2000;92:1081-7.
crossref pmid
7. Kim EK, Kim MH, Oh KK, Lee SK. Variable causes of false negative mammogram. J Korean Radiol Soc 1999;40:997-1002.
crossref
8. Tang SSK, Gui GPH. Biomarkers in the diagnosis of primary and recurrent breast cancer. Biomark Med 2012;6:567-85.
crossref pmid
9. Kazarian A, Blyuss O, Metodieva G, Gentry-Maharaj A, Ryan A, Kiseleva EM, et al. Testing breast cancer serum biomarkers for early detection and prognosis in pre-diagnosis samples. Br J Cancer 2017;116:501-8.
crossref pmid pmc
10. Wahid B, Wahid K, Wahid SB. An update on breast cancer biomarkers. MOJ Immunol 2017;5:00150.
crossref
11. Dos Anjos Pultz B, da Luz FA, de Faria PR, Oliveira AP, de Araújo RA, Silva MJ. Far beyond the usual biomarkers in breast cancer: a review. J Cancer 2014;5:559-71.
crossref pmid pmc
12. Lee HB, Kang UB, Moon HG, Lee J, Lee KM, Yi M, et al. Development and validation of a novel plasma protein signature for breast cancer diagnosis by using multiple reaction monitoring-based mass spectrometry. Anticancer Res 2015;35:6271-9.
pmid
13. Kang UB, Ahn Y, Lee JW, Kim YH, Kim J, Yu MH, et al. Differential profiling of breast cancer plasma proteome by isotope-coded affinity tagging method reveals biotinidase as a breast cancer biomarker. BMC Cancer 2010;10:114.
crossref pmid pmc
14. Bomalaski JJ, Tabano M, Hooper L, Fiorica J. Mammography. Curr Opin Obstet Gynecol 2001;13:15-23.
crossref pmid
15. Carney PA, Miglioretti DL, Yankaskas BC, Kerlikowske K, Rosenberg R, Rutter CM, et al. Individual and combined effects of age, breast density and hormone replacement therapy use on the accuracy of screening mammography. Ann Intern Med 2003;138:168-75.
crossref pmid
16. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometics 1977;33:159-74.
crossref
17. Lim SY, Lee SJ, Shin YK, Lee SN, Choi JY, Kang DR, et al. Comparison of the diagnostic value between mammography and mammography with breast ultrasonography in diagnosing breast cancer. J Korean Acad Fam Med 2003;24:925-33.

18. Shin HJ, Ko ES, Yi A. Breast cancer screening in Korean woman with dense breast tissue. J Korean Soc Radiol 2015;73:279-86.
crossref
19. Kerlikowske K, Grady D, Barclay J, Sickles EA, Ernster V. Likelihood ratios for modern screening mammography. Risk of breast cancer based on age and mammographic interpretation. JAMA 1996;276:39-43.
crossref pmid
20. Moon WK, Kim TJ, Cha JH, Cho KS, Choi EW, Lee YJ, et al. Clinical image evaluation of mammograms: a national survey. J Korean Radiol Soc 2003;49:507-11.
crossref
21. Lee EH, Jung SE. Current status of radiologists’ part-time services in mammography. J Korean Soc Breast Screening 2012;9:142-7.