{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2023,12,5]],"date-time":"2023-12-05T01:42:22Z","timestamp":1701740542051},"reference-count":66,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2023,1,19]],"date-time":"2023-01-19T00:00:00Z","timestamp":1674086400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,1,19]],"date-time":"2023-01-19T00:00:00Z","timestamp":1674086400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001659","name":"Deutsche Forschungsgemeinschaft","doi-asserted-by":"publisher","award":["SI 2171\/3-1"]},{"DOI":"10.13039\/501100001659","name":"Deutsche Forschungsgemeinschaft","doi-asserted-by":"publisher","award":["SI 2171\/2"]},{"DOI":"10.13039\/501100001659","name":"Deutsche Forschungsgemeinschaft","doi-asserted-by":"publisher","award":["AP 206\/11-1"]},{"DOI":"10.13039\/501100001659","name":"Deutsche Forschungsgemeinschaft","doi-asserted-by":"publisher","award":["389792660"]},{"DOI":"10.13039\/501100008678","name":"Universit\u00e4t Leipzig","doi-asserted-by":"crossref"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Empir Software Eng"],"published-print":{"date-parts":[[2023,3]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Understanding the influence of configuration options on the performance of a software system is key for finding optimal system configurations, system understanding, and performance debugging. In the literature, a number of performance-influence modeling approaches have been proposed, which model a configuration option\u2019s influence and a configuration\u2019s performance as a scalar value. However, these point estimates falsely imply a certainty regarding an option\u2019s influence that neglects several sources of uncertainty within the assessment process, such as (1) measurement bias, choices of model representation and learning process, and incomplete data. This leads to the situation that different approaches and even different learning runs assign different scalar performance values to options and interactions among them. The true influence is uncertain, though. There is no way to quantify this uncertainty with state-of-the-art performance modeling approaches.<\/jats:p><jats:p>We propose a novel approach, P4, which is based on probabilistic programming, that explicitly models uncertainty for option influences and consequently provides a confidence interval for each prediction alongside a scalar. This way, we can explain, for the first time, why predictions may be erroneous and which option\u2019s influence may be unreliable. An evaluation on 13 real-world subject systems shows that P4\u2019s accuracy is in line with the state of the art while providing reliable confidence intervals, in addition to scalar predictions. We qualitatively explain how uncertain influences of individual options and interactions cause inaccurate predictions.<\/jats:p>","DOI":"10.1007\/s10664-022-10250-2","type":"journal-article","created":{"date-parts":[[2023,1,19]],"date-time":"2023-01-19T11:14:46Z","timestamp":1674126886000},"update-policy":"http:\/\/dx.doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Mastering uncertainty in performance estimations of configurable software systems"],"prefix":"10.1007","volume":"28","author":[{"ORCID":"http:\/\/orcid.org\/0000-0002-5562-0894","authenticated-orcid":false,"given":"Johannes","family":"Dorn","sequence":"first","affiliation":[]},{"given":"Sven","family":"Apel","sequence":"additional","affiliation":[]},{"given":"Norbert","family":"Siegmund","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2023,1,19]]},"reference":[{"key":"10250_CR1","doi-asserted-by":"publisher","unstructured":"Aken Dana Van, Pavlo Andrew, Gordon Geoffrey J, Zhang Bohan (2017) Automatic database management system tuning through large-scale machine learning. In: Proceedings of the international conference on management of data (SIGMOD). ACM, pp 1009\u20131024. ISBN 978-1-4503-4197-4. https:\/\/doi.org\/10.1145\/3035918.3064029","DOI":"10.1145\/3035918.3064029"},{"key":"10250_CR2","doi-asserted-by":"publisher","unstructured":"Alan M (2002) Subset selection in regression. CRC Press, https:\/\/doi.org\/10.1201\/9781420035933","DOI":"10.1201\/9781420035933"},{"key":"10250_CR3","doi-asserted-by":"publisher","first-page":"746","DOI":"10.1016\/j.future.2019.09.006","volume":"102","author":"F Antonelli","year":"2020","unstructured":"Antonelli F, Cortellessa V, Gribaudo M, Pinciroli R, Trivedi KS, Trubiani C (2020) Analytical modeling of performance indices under epistemic uncertainty applied to cloud computing systems. Future Gener Comput Syst 102:746\u2013761. ISSN 0167-739X. https:\/\/doi.org\/10.1016\/j.future.2019.09.006","journal-title":"Future Gener Comput Syst"},{"key":"10250_CR4","doi-asserted-by":"publisher","first-page":"106371","DOI":"10.1016\/j.infsof.2020.106371","volume":"127","author":"Paolo Arcaini","year":"2020","unstructured":"Arcaini Paolo, Inverso Omar, Trubiani Catia (2020) Automated model-based performance analysis of software product lines under uncertainty. J Inform Software Technol (IST) 127:106371. ISSN 0950-5849. https:\/\/doi.org\/10.1016\/j.infsof.2020.106371","journal-title":"J Inform Software Technol (IST)"},{"key":"10250_CR5","unstructured":"Bergstra J, Bardenet R, Bengio Y, K\u00e9gl B (2011) Algorithms for hyper-parameter optimization. In: Shawe-Taylor J, Zemel R, Bartlett P, Pereira F, Weinberger KQ (eds) Advances in neural information processing systems, volume 24. Curran Associates, Inc"},{"key":"10250_CR6","doi-asserted-by":"crossref","unstructured":"Cheng J, Gao C, Zheng Z (2022) HINNPerf: Hierarchical Interaction Neural Network for Performance Prediction of Configurable Systems. ACM Transactions on Software Engineering and Methodology","DOI":"10.1145\/3528100"},{"key":"10250_CR7","doi-asserted-by":"publisher","unstructured":"Dorn J, Apel S, Siegmund N (2020) Mastering Uncertainty in Performance Estimations of Configurable Software Systems. In: ASE\u201920. Association for Computing Machinery, New York, https:\/\/doi.org\/10.1145\/3324884.3416620","DOI":"10.1145\/3324884.3416620"},{"key":"10250_CR8","doi-asserted-by":"crossref","unstructured":"Dubslaff C, Weis K, Baier C, Apel S (2022) Causality in Configurable Software Systems, p 13","DOI":"10.1145\/3510003.3510200"},{"key":"10250_CR9","doi-asserted-by":"publisher","unstructured":"Elbaum S, Rosenblum DS (2014) Known Unknowns: Testing in the Presence of Uncertainty. In: Proceedings of the 22nd ACM SIGSOFT international symposium on foundations of software engineering, FSE 2014. Association for Computing Machinery, New York, pp 833\u2013836, https:\/\/doi.org\/10.1145\/2635868.2666608","DOI":"10.1145\/2635868.2666608"},{"key":"10250_CR10","doi-asserted-by":"publisher","first-page":"92","DOI":"10.2307\/1937887","volume":"49","author":"DE Farrar","year":"1967","unstructured":"Farrar DE, Glauber RR (1967) Multicollinearity in Regression Analysis: The Problem Revisited. Rev Econ Stat 49:92\u2013107. https:\/\/doi.org\/10.2307\/1937887","journal-title":"Rev Econ Stat"},{"key":"10250_CR11","doi-asserted-by":"publisher","unstructured":"Gareth J, Witten D, Hastie T, Tibshirani R (2013) An introduction to statistical learning. Springer texts in statistics. Springer. ISBN 978-1-4614-7137-0 978-1-4614-7138-7. https:\/\/doi.org\/10.1007\/978-1-4614-7138-7","DOI":"10.1007\/978-1-4614-7138-7"},{"key":"10250_CR12","doi-asserted-by":"publisher","unstructured":"Gogate V, Dechter R (2006) A New Algorithm for Sampling CSP Solutions Uniformly at Random. In: Principles and Practice of Constraint Programming - CP 2006, Springer, pp 711\u2013715, https:\/\/doi.org\/10.1007\/11889205_56","DOI":"10.1007\/11889205_56"},{"key":"10250_CR13","doi-asserted-by":"publisher","unstructured":"Guo J, Czarnecki K, Apel S, Siegmund N, Wasowski A (2013) Variability-Aware Performance Prediction: A Statistical Learning Approach. In: Proceedings of the International Conference on Automated Software Engineering (ASE), IEEE, pp 301\u2013311, https:\/\/doi.org\/10.1109\/ASE.2013.6693089","DOI":"10.1109\/ASE.2013.6693089"},{"key":"10250_CR14","doi-asserted-by":"publisher","first-page":"1826","DOI":"10.1007\/s10664-017-9573-6","volume":"23","author":"J Guo","year":"2018","unstructured":"Guo J, Yang D, Siegmund N, Apel S, Sarkar A, Valov P, Czarnecki K, Wasowski A, Yu H (2018) Data-Efficient Performance Learning for Configurable Systems. Empir Softw Eng 23:1826\u20131867","journal-title":"Empir Softw Eng"},{"key":"10250_CR15","doi-asserted-by":"publisher","unstructured":"Ha H, Zhang H (2019a) DeepPerf: Performance Prediction for Configurable Software with Deep Sparse Neural Network. In: Proceedings of the International Conference on Software Engineering (ICSE), IEEE, pp 1095\u20131106, https:\/\/doi.org\/10.1109\/ICSE.2019.00113","DOI":"10.1109\/ICSE.2019.00113"},{"key":"10250_CR16","doi-asserted-by":"publisher","unstructured":"Ha H, Zhang H (2019b) Performance- Influence Model for Highly Configurable Software with Fourier Learning and Lasso Regression. In: Proceedings of the International Conference on Software Maintenance and Evolution (ICSME), IEEE, pp 470\u2013480, https:\/\/doi.org\/10.1109\/ICSME.2019.00080","DOI":"10.1109\/ICSME.2019.00080"},{"issue":"1","key":"10250_CR17","doi-asserted-by":"publisher","first-page":"70","DOI":"10.1214\/aos\/1176346577","volume":"13","author":"JA Hartigan","year":"1985","unstructured":"Hartigan JA, Hartigan PM (1985) The Dip Test of Unimodality. Annal Stat 13(1):70\u201384","journal-title":"Annal Stat"},{"key":"10250_CR18","doi-asserted-by":"publisher","unstructured":"Henard C, Papadakis M, Harman M, Traon LE (2015) Combining Multi- Objective Search and Constraint Solving for Configuring Large Software Product Lines. In: Proceedings of the International Conference on Software Engineering (ICSE), IEEE\/ACM, pp 517\u2013528, https:\/\/doi.org\/10.1109\/ICSE.2015.69","DOI":"10.1109\/ICSE.2015.69"},{"key":"10250_CR19","unstructured":"Herodotou H, Lim H, Luo G, Borisov N, Dong L, Cetin FB, Babu S (2011) Starfish: a self-tuning system for big data analytics. In: Proceedings of the Conference on Innovative Data Systems Research (CIDR), pp 261\u2013272. www.cidrdb.org"},{"key":"10250_CR20","doi-asserted-by":"publisher","unstructured":"Hill RC, Adkins LC (2007) Collinearity. In: A Companion to Theoretical Econometrics, chapter 12, Wiley, pp 256\u2013278, https:\/\/doi.org\/10.1002\/9780470996249.ch13","DOI":"10.1002\/9780470996249.ch13"},{"key":"10250_CR21","doi-asserted-by":"publisher","first-page":"55","DOI":"10.1080\/00401706.1970.10488634","volume":"12","author":"AE Hoerl","year":"1970","unstructured":"Hoerl AE, Kennard RW (1970) Ridge Regression: Biased Estimation for Nonorthogonal Problems. Technometrics 12:55\u201367. https:\/\/doi.org\/10.1080\/00401706.1970.10488634","journal-title":"Technometrics"},{"key":"10250_CR22","first-page":"1593","volume":"15","author":"MD Hoffman","year":"2014","unstructured":"Hoffman MD, Gelman A (2014) The No- U- Turn Sampler: Adaptively Setting Path Lengths in Hamiltonian Monte Carlo. J Mach Learn Res 15:1593\u20131623","journal-title":"J Mach Learn Res"},{"key":"10250_CR23","doi-asserted-by":"publisher","unstructured":"Horcas J-M, Galindo JA, Heradio R, Fernandez-Amoros D, Benavides D (2021) Monte Carlo Tree Search for Feature Model Analyses: A General Framework for Decision-Making. In: Proceedings of the 25th ACM international systems and software product line conference- Volume A, SPLC\u201921. Association for Computing Machinery, pp 190\u2013201, New York, https:\/\/doi.org\/10.1145\/3461001.3471146","DOI":"10.1145\/3461001.3471146"},{"key":"10250_CR24","doi-asserted-by":"publisher","unstructured":"Iqbal MdS, Krishna R, Javidian MA, Ray B, Jamshidi P (2022) Unicorn: reasoning about configurable system performance through the lens of causality. In: Proceedings of the seventeenth european conference on computer systems, EuroSys \u201922. Association for Computing Machinery, pp 199\u2013217, New York, https:\/\/doi.org\/10.1145\/3492321.3519575","DOI":"10.1145\/3492321.3519575"},{"key":"10250_CR25","doi-asserted-by":"publisher","unstructured":"Jamshidi P, Casale G (2016) An uncertainty-aware approach to optimal configuration of stream processing systems. In: Proceedings of the international symposium on modeling, analysis and simulation of computer and telecommunication systems (MASCOTS). IEEE, pp 39\u201348, https:\/\/doi.org\/10.1109\/MASCOTS.2016.17","DOI":"10.1109\/MASCOTS.2016.17"},{"key":"10250_CR26","doi-asserted-by":"publisher","unstructured":"Johansen MF, Haugen \u00d8, Fleurey F (2012) An algorithm for generating T-wise covering arrays from large feature models. In: Proceedings of the international software product line conference (SPLC). ACM, p 46, https:\/\/doi.org\/10.1145\/2362536.2362547","DOI":"10.1145\/2362536.2362547"},{"issue":"4","key":"10250_CR27","doi-asserted-by":"publisher","first-page":"58","DOI":"10.1109\/MS.2020.2987024","volume":"37","author":"Christian Kaltenecker","year":"2020","unstructured":"Kaltenecker Christian, Grebhahn Alexander, Siegmund Norbert, Apel Sven (2020) The interplay of sampling and machine learning for software performance prediction. IEEE Softw 37(4):58\u201366. ISSN 0740-7459, 1937-4194. https:\/\/doi.org\/10.1109\/MS.2020.2987024","journal-title":"IEEE Softw"},{"key":"10250_CR28","doi-asserted-by":"publisher","unstructured":"Kaltenecker C, Grebhahn A, Siegmund N, Guo J, Apel S (2019) Distance-based sampling of software configuration spaces. In: Proceedings of the international conference on software engineering (ICSE). IEEE, pp 1084\u20131094, https:\/\/doi.org\/10.1109\/ICSE.2019.00112","DOI":"10.1109\/ICSE.2019.00112"},{"issue":"1-2","key":"10250_CR29","doi-asserted-by":"publisher","first-page":"81","DOI":"10.1093\/biomet\/30.1-2.81","volume":"30","author":"MG Kendall","year":"1938","unstructured":"Kendall MG (1938) A new measure of rank correlation. Biometrika 30(1-2):81\u201393. ISSN 0006-3444. https:\/\/doi.org\/10.1093\/biomet\/30.1-2.81","journal-title":"Biometrika"},{"key":"10250_CR30","unstructured":"Kendall Alex, Gal Yarin (2017) What uncertainties do we need in Bayesian deep learning for computer vision?. In: Proceedings of the international conference on neural information processing systems (NIPS). Curran Associates Inc., pp 5580\u20135590"},{"key":"10250_CR31","first-page":"020","volume":"06","author":"AD Kiureghian","year":"2008","unstructured":"Kiureghian AD, Ditlevsen O (2008) Aleatory or Epistemic? Does It Matter? Structural Safety, 31:105\u2013112, 2009. ISSN 01674730. 10.1016\/j.strusafe. 06:020","journal-title":"Structural Safety, 31:105\u2013112, 2009. ISSN 01674730. 10.1016\/j.strusafe."},{"issue":"4","key":"10250_CR32","doi-asserted-by":"publisher","first-page":"2410","DOI":"10.1007\/s10664-019-09705-w","volume":"24","author":"S Kolesnikov","year":"2019","unstructured":"Kolesnikov S, Siegmund N, K\u00e4stner C., Apel S (2019) On the relation of control-flow and performance feature interactions: a case study. Empirical Software Eng (EMSE) 24(4):2410\u20132437","journal-title":"Empirical Software Eng (EMSE)"},{"key":"10250_CR33","doi-asserted-by":"publisher","unstructured":"Krishna Rahul, Tang Chong, Sullivan Kevin, Ray Baishakhi (2020) Conex: efficient exploration of big-data system configurations for better performance. IEEE Trans Software Eng, pp 1\u20131. https:\/\/doi.org\/10.1109\/TSE.2020.3007560","DOI":"10.1109\/TSE.2020.3007560"},{"key":"10250_CR34","unstructured":"Maaten LVD, Postma E, den Herik JV (2009) Dimensionality reduction: a comparative review. J Mach Learn Res, pp 10"},{"key":"10250_CR35","doi-asserted-by":"publisher","unstructured":"Mandrioli C, Maggio M (2021) Testing self- adaptive software with probabilistic guarantees on performance metrics: extended and comparative results. IEEE Trans Software Eng, pp 1\u20131, https:\/\/doi.org\/10.1109\/TSE.2021.3101130. ISSN 0098-5589, 1939-3520, 2326-3881","DOI":"10.1109\/TSE.2021.3101130"},{"key":"10250_CR36","unstructured":"Murphy KP (2012) Machine learning: a probabilistic perspective. Adaptive computation and machine learning series. MIT Press. ISBN 978-0-262-01802-9"},{"key":"10250_CR37","doi-asserted-by":"publisher","first-page":"247","DOI":"10.1007\/s10515-017-0225-2","volume":"25","author":"Vivek Nair","year":"2017","unstructured":"Nair Vivek, Menzies Tim, Siegmund Norbert, Apel Sven (2017a) Faster discovery of faster system configurations with spectral learning. Autom Softw Eng 25:247\u2013277. https:\/\/doi.org\/10.1007\/s10515-017-0225-2","journal-title":"Autom Softw Eng"},{"key":"10250_CR38","doi-asserted-by":"crossref","unstructured":"Nair Vivek, Menzies Tim, Siegmund Norbert, Apel Sven (2017b) Using bad learners to find good configurations. In: Proceedings of the 2017 11th joint meeting on foundations of software engineering, ESEC\/FSE 2017. ACM, pp 257\u2013267. ISBN 9781450351058","DOI":"10.1145\/3106237.3106238"},{"key":"10250_CR39","doi-asserted-by":"publisher","first-page":"794","DOI":"10.1109\/TSE.2018.2870895","volume":"46","author":"V Nair","year":"2020","unstructured":"Nair V, Zhe Y u, Menzies T, Siegmund N, Apel S (2020) Finding faster configurations using FLASH. Trans Software Eng 46:794\u2013811","journal-title":"Trans Software Eng"},{"key":"10250_CR40","unstructured":"Neal RM (1993) Probabilistic inference using markov chain monte carlo methods. Department Of Computer Science University Of Toronto"},{"key":"10250_CR41","unstructured":"Werner N (2019) Energy and performance evolution of configurable systems case studies and experiments. Master thesis, University of Passau"},{"key":"10250_CR42","doi-asserted-by":"publisher","first-page":"673","DOI":"10.1007\/s11135-006-9018-6","volume":"41","author":"RM O\u2019Brien","year":"2007","unstructured":"O\u2019Brien RM (2007) A caution regarding rules of thumb for variance inflation factors. Quality Quantity 41:673\u2013690. ISSN 0033-5177. https:\/\/doi.org\/10.1007\/s11135-006-9018-6","journal-title":"Quality Quantity"},{"key":"10250_CR43","doi-asserted-by":"publisher","unstructured":"Oh J, Batory D, Myers M, Siegmund N (2017) Finding near-optimal configurations in product lines by random sampling. In: Proceedings of the joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on the foundations of software engineering (ESEC\/FSE). ACM, pp 61\u201371. ISBN 978-1-4503-5105-8. https:\/\/doi.org\/10.1145\/3106237.3106273","DOI":"10.1145\/3106237.3106273"},{"key":"10250_CR44","doi-asserted-by":"publisher","unstructured":"Rice JR (1976) The algorithm selection problem. volume 15 of Advances in Computers, pages 65\u2013118. Elsevier, https:\/\/doi.org\/10.1016\/S0065-2458(08)60520-3","DOI":"10.1016\/S0065-2458(08)60520-3"},{"key":"10250_CR45","doi-asserted-by":"publisher","unstructured":"Robbins HE (1956) An empirical bayes approach to statistics. In: Proceedings of the third berkeley symposium on mathematical statistics and probability, Vol 1: contributions to the theory of statistics. The Regents of the University of California, https:\/\/doi.org\/10.1007\/978-1-4612-0919-5_26","DOI":"10.1007\/978-1-4612-0919-5_26"},{"key":"10250_CR46","unstructured":"Roeder G, Yuhuai W (2017) David k duvenaud. Sticking the landing simple, lower- variance gradient estimators for variational inference. In: Proceedings of the international conference on neural information processing systems (NIPS). Curran Associates Inc., pp 6928\u20136937"},{"key":"10250_CR47","doi-asserted-by":"crossref","unstructured":"Salvatier J, Wiecki TV, Fonnesbeck C (2016) Probabilistic programming in Python using PyMC3. PeerJ computer science, 2:e55. ISSN 2376-5992. 10.7717\/peerj-cs.55","DOI":"10.7717\/peerj-cs.55"},{"key":"10250_CR48","doi-asserted-by":"publisher","first-page":"379","DOI":"10.1002\/j.1538-7305.1948.tb01338.x","volume":"27","author":"CE Shannon","year":"1948","unstructured":"Shannon CE (1948) A mathematical theory of communication. Bell Syst Tech J 27:379\u2013423. ISSN 0005-8580. https:\/\/doi.org\/10.1002\/j.1538-7305.1948.tb01338.x","journal-title":"Bell Syst Tech J"},{"issue":"3-4","key":"10250_CR49","doi-asserted-by":"publisher","first-page":"591","DOI":"10.1093\/biomet\/52.3-4.591","volume":"52","author":"SS Shapiro","year":"1965","unstructured":"Shapiro SS , Wilk MB (1965) An analysis of variance test for normality (complete samples)\u2021. Biometrika 52(3-4):591\u2013611. ISSN 0006-3444. https:\/\/doi.org\/10.1093\/biomet\/52.3-4.591","journal-title":"Biometrika"},{"key":"10250_CR50","doi-asserted-by":"publisher","unstructured":"Siegmund Norbert, Grebhahn Alexander, Apel Sven, K\u00e4stner Christian (2015) Performance- influence models for highly configurable systems. In: Proceedings of the joint meeting of the european software engineering conference and the ACM SIGSOFT symposium on the foundations of software engineering (ESEC\/FSE). ACM, pp 284\u2013294. ISBN 978-1-4503-3675-8. https:\/\/doi.org\/10.1145\/2786805.2786845","DOI":"10.1145\/2786805.2786845"},{"key":"10250_CR51","doi-asserted-by":"publisher","unstructured":"Siegmund Norbert, Kolesnikov Sergiy S, Kastner Christian, Apel Sven, Batory Don, Rosenmuller Marko, Saake Gunter (2012a) Predicting performance via automated feature-interaction detection. In: Proceedings of the international conference on software engineering (ICSE). IEEE, pp 167\u2013177. ISBN 978-1-4673-1066-6 978-1-4673-1067-3. https:\/\/doi.org\/10.1109\/ICSE.2012.6227196","DOI":"10.1109\/ICSE.2012.6227196"},{"key":"10250_CR52","doi-asserted-by":"publisher","first-page":"487","DOI":"10.1007\/s11219-011-9152-9","volume":"20","author":"Norbert Siegmund","year":"2012","unstructured":"Siegmund Norbert, Rosenm\u00fcller M, Kuhlemann Martin, K\u00e4stner C, Apel Sven, Saake Gunter (2012b) SPL conqueror: toward optimization of non-functional properties in software product lines. Software Qual J 20:487\u2013517. ISSN 0963-9314, 1573-1367. https:\/\/doi.org\/10.1007\/s11219-011-9152-9","journal-title":"Software Qual J"},{"key":"10250_CR53","doi-asserted-by":"crossref","unstructured":"Smith R (2013) Uncertainty quantification: theory, implementation, and applications. Soc Indust Appl Math. ISBN 978-1-61197-321-1","DOI":"10.1137\/1.9781611973228"},{"key":"10250_CR54","doi-asserted-by":"publisher","unstructured":"Taylor JR (1997) An introduction to error analysis: the study of uncertainties in physical measurements University Science Books, 2nd edn. https:\/\/doi.org\/10.1063\/1.882103","DOI":"10.1063\/1.882103"},{"key":"10250_CR55","first-page":"267","volume":"58","author":"R Tibshirani","year":"1996","unstructured":"Tibshirani R (1996) Regression Shrinkage and selection via the lasso. J Royal Stat Soc . Series B (Methodol) 58:267\u2013288. ISSN 0035-9246","journal-title":"J Royal Stat Soc . Series B (Methodol)"},{"key":"10250_CR56","doi-asserted-by":"publisher","unstructured":"Trubiani Catia, Apel Sven (2019) PLUS: performance learning for uncertainty of software. In: Proceedings of the international conference on software engineering: new ideas and emerging results. IEEE, pp 77\u201380, https:\/\/doi.org\/10.1109\/ICSE-NIER.2019.00028","DOI":"10.1109\/ICSE-NIER.2019.00028"},{"issue":"3","key":"10250_CR57","doi-asserted-by":"publisher","first-page":"265","DOI":"10.1007\/s10515-020-00273-8","volume":"27","author":"M Velez","year":"2020","unstructured":"Velez M, Jamshidi P, Sattler F, Siegmund N, Apel S, Christian K\u00e4stner (2020) Configcrusher: towards white-box performance analysis for configurable systems. Autom Softw Eng 27(3):265\u2013300","journal-title":"Autom Softw Eng"},{"key":"10250_CR58","doi-asserted-by":"publisher","unstructured":"Velez Miguel, Jamshidi Pooyan, Siegmund Norbert, Apel Sven, K\u00e4stner C (2021) White-box analysis over machine learning: modeling performance of configurable systems. In: 2021 IEEE\/ACM 43rd international conference on software engineering (ICSE), pp 1072\u20131084. https:\/\/doi.org\/10.1109\/ICSE43902.2021.00100","DOI":"10.1109\/ICSE43902.2021.00100"},{"key":"10250_CR59","doi-asserted-by":"publisher","first-page":"261","DOI":"10.1038\/s41592-019-0686-2","volume":"17","author":"P Virtanen","year":"2020","unstructured":"Virtanen P, Gommers R, Oliphant TE, Haberland M, Reddy T, Cournapeau D, Burovski E, Peterson P, Weckesser W, Bright J, Walt SJ, Brett M, Wilson J, Millman JK, Mayorov N, Nelson ARJ, Jones E, Kern R, Larson E, Carey CJ, Polat I, Feng Y u, Moore EW, erPlas JV, Laxalde D, Perktold J, Cimrman R, Henriksen I, Quintero EA, Harris CR, Archibald AM, Ribeiro AH, Pedregosa F, Mulbregt Pv (2020) Scipy 1. 0 Contributors. SciPy 1.0: fundamental algorithms for scientific computing in python. Nat Methods 17:261\u2013272","journal-title":"Nat Methods"},{"key":"10250_CR60","doi-asserted-by":"publisher","unstructured":"Weber Max, Apel Sven, Siegmund Norbert (2021) White-box performance-influence models: a profiling and learning approach. In: 2021 IEEE\/ACM 43rd international conference on software engineering (ICSE), pp-1059\u20131071. https:\/\/doi.org\/10.1109\/ICSE43902.2021.00099","DOI":"10.1109\/ICSE43902.2021.00099"},{"key":"10250_CR61","unstructured":"Wooldridge Jeffrey (2012) Introductory econometrics: a modern approach. South-Western College Pub, 5 edn. ISBN 978-1-111-53104-1"},{"key":"10250_CR62","doi-asserted-by":"publisher","unstructured":"Xu Tianyin, Jin Long, Fan Xuepeng, Zhou Yuanyuan, Pasupathy Shankar, Talwadker Rukma (2015) Hey, You Have given Me Too Many Knobs!: understanding and dealing with over-designed configuration in system software. In: Proceedings of the joint meeting of the european software engineering conference and the ACM SIGSOFT symposium on the foundations of software engineering (ESEC\/FSE). ACM, pp 307\u2013319. ISBN 978-1-4503-3675-8. https:\/\/doi.org\/10.1145\/2786805.2786852","DOI":"10.1145\/2786805.2786852"},{"key":"10250_CR63","doi-asserted-by":"publisher","unstructured":"Zhang Yi, Guo Jianmei, Blais Eric, Czarnecki Krzysztof (2015) Performance prediction of configurable software systems by fourier learning. In: Proceedings of the international conference on automated software engineering (ASE). IEEE, pp 365\u2013373, https:\/\/doi.org\/10.1109\/ASE.2015.15","DOI":"10.1109\/ASE.2015.15"},{"key":"10250_CR64","doi-asserted-by":"publisher","unstructured":"Zhu Y, Liu J, Guo M, Bao Y, Ma W, Liu Z, Song K, Yang Y (2017) BestConfig: tapping the performance potential of systems via automatic configuration tuning. In: Proceedings of the symposium on cloud computing (SoCC). ACM, pp 338\u2013350. ISBN 9781450350280. https:\/\/doi.org\/10.1145\/3127479.3128605","DOI":"10.1145\/3127479.3128605"},{"key":"10250_CR65","doi-asserted-by":"publisher","first-page":"301","DOI":"10.1111\/j.1467-9868.2005.00503.x","volume":"67","author":"H Zou","year":"2017","unstructured":"Zou H, Hastie T (2017) Regularization and variable selection via the elastic net. J Royal Stat Soc: Series B (Stat Method) 67:301\u2013320. ISSN 1369-7412. https:\/\/doi.org\/10.1111\/j.1467-9868.2005.00503.x","journal-title":"J Royal Stat Soc: Series B (Stat Method)"},{"key":"10250_CR66","doi-asserted-by":"crossref","unstructured":"Zwillinger D, Kokoska S (1999) CRC standard probability and statistics tables and formulae CRC Press","DOI":"10.1201\/9780367802417"}],"container-title":["Empirical Software Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-022-10250-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10664-022-10250-2\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-022-10250-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,12,5]],"date-time":"2023-12-05T01:18:03Z","timestamp":1701739083000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10664-022-10250-2"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,1,19]]},"references-count":66,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2023,3]]}},"alternative-id":["10250"],"URL":"http:\/\/dx.doi.org\/10.1007\/s10664-022-10250-2","relation":{},"ISSN":["1382-3256","1573-7616"],"issn-type":[{"value":"1382-3256","type":"print"},{"value":"1573-7616","type":"electronic"}],"subject":["Software"],"published":{"date-parts":[[2023,1,19]]},"assertion":[{"value":"9 October 2022","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"19 January 2023","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors have no competing interests to declare that are relevant to the content of this article.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"<!--Emphasis Type='Bold' removed-->Conflict of Interests\/Competing Interests"}}],"article-number":"33"}}