Workload Prediction using Apache Spark and R in a Mainframe Operating System

  • This master thesis presents a machine learning approach using the distributed data- processing framework Apache Spark and the programming language R. The aim is to predict the time series of the Appl Percentage (ApplPerc) from the workload manager of the z/OS mainframe system using SMF 72.3 and SMF 70.1 records as input. System Management Facility (SMF) data are binary log files that are used to collect system performance data and information about system behaviour. The SMFs show how Apache Spark can be used for the pre-processing. This includes data collection, data extraction, and data selection. The machine learning algorithms are implemented in R. The input SMF datasets will be split into training and test data, and thereby applied to different machine learning and deep learning models such as random forest regression, recurrent neural network, and k-nearest neighbor regression to predict the ApplPerc. The validation of the prediction models will be proved by using cross-validation techniques to evaluate the best applied parameters for each model and therefore to locate the model with the best performance. The process of data analysis is followed by a data-mining methodology called Cross-Industry Standard Process for Data Mining (CRISP-DM), which outlines the steps involved in performing the analysis.

Export metadata

Additional Services

Share in Twitter Search Google Scholar
Metadaten
Author:Daniel Röchert
Examiner:Frank Zimmer, Andreas Henicke
Document Type:Master Thesis
Language:English
Date of Publication (online):2017/12/05
Year of first Publication:2017
Publishing Institution:Hochschule Rhein-Waal
Degree-granting institution:Hochschule Rhein-Waal
Date of final exam:2017/10/27
Release Date:2017/12/08
Tag:Apache Spark; Deep learning; Machine learning; Prediction; Workload management
Faculties and Institutes:Fakultät Kommunikation und Umwelt
Dewey Decimal Classification:0 Informatik, Informationswissenschaft, allgemeine Werke / 00 Informatik, Wissen, Systeme / 004 Datenverarbeitung; Informatik
Licence (German):License LogoCC BY-NC 3.0 DE - Namensnennung-Nicht kommerziell