(self, load_script=False, variant="predict")
Make sequential predictions using a trained model. 
        This function is built for sequence and time series predictions. For standard ML simply use the predict function.
        For sequence prediction we expect using lag periods as features. 
        So previous samples, and possibly predictions, need to be fed as input for the next prediction. 
        Therefore, ensure that the number of features and historical data passed to get a prediction matches the model's requirements.
        
        If the previous target is being included in the lag observations, i.e. the lag_target parameter was set to True,
        include historical target values in n_features in the same order as the feature definitions provided during model training. 
        The target can be empty or 0 for future periods.
        For multi-step predictions, the features can be empty or 0 for future periods as well.

        If variant='predict_proba', we return the predicted probabilties for each sample. Otherwise, we return the predictions.
        If variant='internal', the call can be made from within this class.
        
        This method can be called from a chart expression or the load script in Qlik. 
        The load_script flag needs to be set accordingly for the correct response.
Make sequential predictions using a trained model. 
        This function is built for sequence and time series predictions. For standard ML simply use the predict function.
        For sequence prediction we expect using lag periods as features. 
        So previous samples, and possibly predictions, need to be fed as input for the next prediction. 
        Therefore, ensure that the number of features and historical data passed to get a prediction matches the model's requirements.
        
        If the previous target is being included in the lag observations, i.e. the lag_target parameter was set to True,
        include historical target values in n_features in the same order as the feature definitions provided during model training. 
        The target can be empty or 0 for future periods.
        For multi-step predictions, the features can be empty or 0 for future periods as well.
Make sequential predictions using a trained model . This function is built for sequence and time series predictions . For standard ML simply use the predict function . For sequence prediction we expect using lag periods as features . So previous samples and possibly predictions need to be fed as input for the next prediction . Therefore ensure that the number of features and historical data passed to get a prediction matches the model s requirements . If the previous target is being included in the lag observations i . e . the lag_target parameter was set to True include historical target values in n_features in the same order as the feature definitions provided during model training . The target can be empty or 0 for future periods . For multi - step predictions the features can be empty or 0 for future periods as well .
def sequence_predict(self, load_script=False, variant="predict"):
        """
        Make sequential predictions using a trained model. 
        This function is built for sequence and time series predictions. For standard ML simply use the predict function.
        For sequence prediction we expect using lag periods as features. 
        So previous samples, and possibly predictions, need to be fed as input for the next prediction. 
        Therefore, ensure that the number of features and historical data passed to get a prediction matches the model's requirements.
        
        If the previous target is being included in the lag observations, i.e. the lag_target parameter was set to True,
        include historical target values in n_features in the same order as the feature definitions provided during model training. 
        The target can be empty or 0 for future periods.
        For multi-step predictions, the features can be empty or 0 for future periods as well.

        If variant='predict_proba', we return the predicted probabilties for each sample. Otherwise, we return the predictions.
        If variant='internal', the call can be made from within this class.
        
        This method can be called from a chart expression or the load script in Qlik. 
        The load_script flag needs to be set accordingly for the correct response.
        """

        if variant != 'internal':
            # Open an existing model and get the input dataset. 
            # Target for historical data are expected if using previous targets as a feature.
            request_data = self._get_model_and_data(ordered_data=True) 
            if type(request_data) == list:
                X, y = request_data
            else:
                X = request_data
        else:
            X = self.X_test.copy()
            y = self.y_test.copy()

       # Scale the targets and increase stationarity if required
        if variant != 'internal' and self.model.lag_target and (self.model.scale_target or self.model.make_stationary):
            # If using differencing, we retain original y values for inversing the transformation later
            y_orig = y.values.ravel() if self.model.make_stationary=='difference' else None
            # Apply the transformer to the targets
            y = self.model.target_transformer.transform(y)
            # Drop samples where y cannot be transformed due to insufficient lags
            X = X.iloc[len(X)-len(y):]

        # Set the number of periods to be predicted
        prediction_periods = self.model.prediction_periods
        # Set the number of rows required for one prediction
        self.rows_per_pred = 1
        self.diff_lags = max(self.model.stationarity_lags) if self.model.lag_target and self.model.make_stationary=='difference' else 0
        # Set property depending on whether the current sample will be included as an input, or if we only use lag observations for predictions
        self.first_pred_modifier = 1 if self.model.current_sample_as_input else 0  

        # Check that the input data includes history to meet any lag calculation requirements
        if self.model.lags:
            # An additional lag observation is needed if previous targets are being added to the features
            self.rows_per_pred = self.model.lags+self.first_pred_modifier+1 if self.model.lag_target else self.model.lags+self.first_pred_modifier
            # If the target is being lagged and made stationary through differencing additional lag periods are required
            if self.model.lag_target and self.model.make_stationary=='difference':
                extra_msg = " plus an additional {} periods for making the target stationary using differencing".format(self.diff_lags)
            # For multi-step predictions we only expect lag values, not the current period's values
            # self.rows_per_pred = self.rows_per_pred-1 if prediction_periods > 1 else self.rows_per_pred
            assert len(X) >= self.rows_per_pred + self.diff_lags, "Insufficient input data as the model requires {} lag periods for each prediction".format(self.rows_per_pred) + extra_msg

        if variant != 'internal':
            # Prepare the response DataFrame
            # Initially set up with the 'model_name' and 'key' columns and the same index as request_df
            self.response = self.request_df.drop(columns=['n_features'])
        
        # Set up a list to contain predictions and probabilities if required
        predictions = []
        get_proba =  False
        if variant == 'predict_proba':
            get_proba =  True
            probabilities = []     

        # Refresh the keras model to avoid tensorflow errors
        if self.model.using_keras:
            self._keras_refresh()

        if prediction_periods > 1:
            if not self.model.lag_target:
                y = None

            # Check that we can generate 1 or more predictions of prediction_periods each
            n_samples = len(X)
            assert (n_samples - self.rows_per_pred) >= prediction_periods, \
                "Cannot generate predictions for {} periods with {} rows, with {} rows required for lag observations. You may need to provide more historical data or sufficient placeholder rows for future periods."\
                .format(prediction_periods, n_samples, self.rows_per_pred)
            
            # For multi-step predictions we can add lag observations up front as we only use actual values
            # i.e. We don't use predicted y values for further predictions    
            if self.model.lags or self.model.lag_target:
                X = self._add_lags(X, y=y, extrapolate=self.first_pred_modifier)   

            # We start generating predictions from the first row as lags will already have been added to each sample
            start = 0
        else:
            # We start generating predictions from the point where we will have sufficient lag observations
            start = self.rows_per_pred
        
        if self.model.lag_target or prediction_periods > 1:
            # Get the predictions by walking forward over the data
            for i in range(start, len(X) + self.first_pred_modifier, prediction_periods):      
                # For multi-step predictions we take in self.rows_per_pred rows of X to generate predictions for prediction_periods
                if prediction_periods > 1:
                    batch_X = X.iloc[[i]]
                    
                    if not get_proba:
                        # Get the prediction. 
                        pred = self.model.pipe.predict(batch_X)
                        # Flatten the predictions for multi-step outputs and add to the list
                        pred = pred.ravel().tolist()
                        predictions += pred
                    else:
                        # Get the predicted probability for each sample 
                        proba = self.model.pipe.predict_proba(batch_X)
                        proba = proba.reshape(-1, len(self.model.pipe.named_steps['estimator'].classes_))
                        probabilities += proba.tolist()
                # For walk forward predictions with lag targets we use each prediction as input to the next prediction, with X values avaialble for future periods.
                else:
                    batch_X = X.iloc[i-self.rows_per_pred : i] 
                    # Add lag observations
                    batch_y = y.iloc[i-self.rows_per_pred : i]
                    batch_X = self._add_lags(batch_X, y=batch_y, extrapolate=self.first_pred_modifier)

                    # Get the prediction. We only get a prediction for the last sample in the batch, the remaining samples only being used to add lags.
                    pred = self.model.pipe.predict(batch_X.iloc[[-1],:])

                    # Add the prediction to the list. 
                    predictions.append(pred)
                                
                    # Add the prediction to y to be used as a lag target for the next prediction
                    y.iloc[i - self.first_pred_modifier, 0] = pred

                    # If probabilities need to be returned
                    if get_proba:
                        # Get the predicted probability for each sample 
                        probabilities.append(self.model.pipe.predict_proba(batch_X.iloc[[-1],:]))
        else:
            # Add lag observations to the samples if required
            if self.model.lags:
                X = self._add_lags(X, extrapolate=self.first_pred_modifier)

            # Get prediction for X
            predictions = self.model.pipe.predict(X)

            # If probabilities need to be returned
            if get_proba:
                # Get the predicted probability for each sample 
                probabilities = self.model.pipe.predict_proba(X)
        
        # Set the number of placeholders needed in the response
        # These are samples for which predictions were not generated due to insufficient lag periods or for meeting multi-step prediction period requirements
        self.placeholders = self.rows_per_pred + self.diff_lags - self.first_pred_modifier

       # Transform probabilities to a readable string
        if get_proba:
            # Add the required number of placeholders at the start of the response list
            y = ["\x00"] * self.placeholders
            
            # Truncate multi-step predictions if the (number of samples - self.rows_per_pred) is not a multiple of prediction_periods
            if prediction_periods > 1 and ((n_samples-self.rows_per_pred) % prediction_periods) > 0:              
                probabilities = probabilities[:-len(probabilities)+(n_samples-self.rows_per_pred)]
            
            for a in probabilities:
                s = ""
                i = 0
                for b in a:
                    s = s + ", {0}: {1:.3f}".format(self.model.pipe.named_steps['estimator'].classes_[i], b)
                    i += 1
                y.append(s[2:])

        # Prepare predictions
        else:
            if prediction_periods > 1:
                # Set the value to use for nulls
                null = np.NaN if is_numeric_dtype(np.array(predictions)) else "\x00"

                # Truncate multi-step predictions if the (number of samples - self.placeholders) is not a multiple of prediction_periods
                if (n_samples-self.rows_per_pred) % prediction_periods > 0:
                    predictions = predictions[:-len(predictions)+(n_samples-self.rows_per_pred)]

                # Add null values at the start of the response list to match the cardinality of the input from Qlik
                y = np.array(([null] * (self.rows_per_pred - self.first_pred_modifier)) + predictions)
            elif self.model.lag_target: 
                # Remove actual values for which we did not generate predictions due to insufficient lags
                if is_numeric_dtype(y.iloc[:, 0].dtype):
                    y.iloc[:self.placeholders - self.first_pred_modifier, 0] = np.NaN
                else:
                    y.iloc[:self.placeholders - self.first_pred_modifier, 0] = "\x00"
                # Flatten y to the expected 1D shape
                y = y.values.ravel()
            else:
                y = np.array(predictions)
            
            # Inverse transformations on the targets if required  
            if variant != 'internal' and (self.model.scale_target or self.model.make_stationary):
                # Take out placeholder values before inverse transform of targets
                null_values = y[:self.rows_per_pred - self.first_pred_modifier] if prediction_periods > 1 or self.model.lag_target else []
                # Add placeholders for samples removed during differencing
                if self.model.make_stationary=='difference':
                    null_values = np.append(null_values, np.repeat(null_values[0], self.diff_lags))
                y = y if len(null_values) == 0 else y[-len(predictions):]
                # Add untransformed lag values for differencing if required
                end = self.placeholders
                start = end - self.diff_lags
                y = y if y_orig is None else np.append(y_orig[start : end], y)

                # Apply the transformer to the test targets
                y = self.model.target_transformer.inverse_transform(y) 

                # Remove lags used for making the series stationary in case of differencing
                if self.model.make_stationary == 'difference':
                    y = y[self.diff_lags:]

                # Replace lags used for making the series stationary with nulls in case of differencing
                # if self.model.make_stationary == 'difference':
                    #null = np.NaN if is_numeric_dtype(np.array(predictions)) else "\x00"
                    # y = np.append(np.array([null]*self.diff_lags), y[self.diff_lags:])
                
                # Add back the placeholders for lag values
                if len(null_values) > 0:
                    y = np.append(null_values, y)
        
        if variant == 'internal':
            return y

        # Add predictions / probabilities to the response
        self.response['result'] = y

        # Reindex the response to reset to the original sort order
        self.response = self.response.reindex(self.original_index)
        
        if load_script:
            # If the function was called through the load script we return a Data Frame
            self._send_table_description("predict")
            
            # Debug information is printed to the terminal and logs if the paramater debug = true
            if self.model.debug:
                self._print_log(4)
            
            return self.response
            
        # If the function was called through a chart expression we return a Series
        else:
            # Debug information is printed to the terminal and logs if the paramater debug = true
            if self.model.debug:
                self._print_log(4)
            
            return self.response.loc[:,'result']
def sequence_predict ( self , load_script = False , variant = "predict" ) : if variant != 'internal' : # Open an existing model and get the input dataset.  # Target for historical data are expected if using previous targets as a feature. request_data = self . _get_model_and_data ( ordered_data = True ) if type ( request_data ) == list : X , y = request_data else : X = request_data else : X = self . X_test . copy ( ) y = self . y_test . copy ( ) # Scale the targets and increase stationarity if required if variant != 'internal' and self . model . lag_target and ( self . model . scale_target or self . model . make_stationary ) : # If using differencing, we retain original y values for inversing the transformation later y_orig = y . values . ravel ( ) if self . model . make_stationary == 'difference' else None # Apply the transformer to the targets y = self . model . target_transformer . transform ( y ) # Drop samples where y cannot be transformed due to insufficient lags X = X . iloc [ len ( X ) - len ( y ) : ] # Set the number of periods to be predicted prediction_periods = self . model . prediction_periods # Set the number of rows required for one prediction self . rows_per_pred = 1 self . diff_lags = max ( self . model . stationarity_lags ) if self . model . lag_target and self . model . make_stationary == 'difference' else 0 # Set property depending on whether the current sample will be included as an input, or if we only use lag observations for predictions self . first_pred_modifier = 1 if self . model . current_sample_as_input else 0 # Check that the input data includes history to meet any lag calculation requirements if self . model . lags : # An additional lag observation is needed if previous targets are being added to the features self . rows_per_pred = self . model . lags + self . first_pred_modifier + 1 if self . model . lag_target else self . model . lags + self . first_pred_modifier # If the target is being lagged and made stationary through differencing additional lag periods are required if self . model . lag_target and self . model . make_stationary == 'difference' : extra_msg = " plus an additional {} periods for making the target stationary using differencing" . format ( self . diff_lags ) # For multi-step predictions we only expect lag values, not the current period's values # self.rows_per_pred = self.rows_per_pred-1 if prediction_periods > 1 else self.rows_per_pred assert len ( X ) >= self . rows_per_pred + self . diff_lags , "Insufficient input data as the model requires {} lag periods for each prediction" . format ( self . rows_per_pred ) + extra_msg if variant != 'internal' : # Prepare the response DataFrame # Initially set up with the 'model_name' and 'key' columns and the same index as request_df self . response = self . request_df . drop ( columns = [ 'n_features' ] ) # Set up a list to contain predictions and probabilities if required predictions = [ ] get_proba = False if variant == 'predict_proba' : get_proba = True probabilities = [ ] # Refresh the keras model to avoid tensorflow errors if self . model . using_keras : self . _keras_refresh ( ) if prediction_periods > 1 : if not self . model . lag_target : y = None # Check that we can generate 1 or more predictions of prediction_periods each n_samples = len ( X ) assert ( n_samples - self . rows_per_pred ) >= prediction_periods , "Cannot generate predictions for {} periods with {} rows, with {} rows required for lag observations. You may need to provide more historical data or sufficient placeholder rows for future periods." . format ( prediction_periods , n_samples , self . rows_per_pred ) # For multi-step predictions we can add lag observations up front as we only use actual values # i.e. We don't use predicted y values for further predictions     if self . model . lags or self . model . lag_target : X = self . _add_lags ( X , y = y , extrapolate = self . first_pred_modifier ) # We start generating predictions from the first row as lags will already have been added to each sample start = 0 else : # We start generating predictions from the point where we will have sufficient lag observations start = self . rows_per_pred if self . model . lag_target or prediction_periods > 1 : # Get the predictions by walking forward over the data for i in range ( start , len ( X ) + self . first_pred_modifier , prediction_periods ) : # For multi-step predictions we take in self.rows_per_pred rows of X to generate predictions for prediction_periods if prediction_periods > 1 : batch_X = X . iloc [ [ i ] ] if not get_proba : # Get the prediction.  pred = self . model . pipe . predict ( batch_X ) # Flatten the predictions for multi-step outputs and add to the list pred = pred . ravel ( ) . tolist ( ) predictions += pred else : # Get the predicted probability for each sample  proba = self . model . pipe . predict_proba ( batch_X ) proba = proba . reshape ( - 1 , len ( self . model . pipe . named_steps [ 'estimator' ] . classes_ ) ) probabilities += proba . tolist ( ) # For walk forward predictions with lag targets we use each prediction as input to the next prediction, with X values avaialble for future periods. else : batch_X = X . iloc [ i - self . rows_per_pred : i ] # Add lag observations batch_y = y . iloc [ i - self . rows_per_pred : i ] batch_X = self . _add_lags ( batch_X , y = batch_y , extrapolate = self . first_pred_modifier ) # Get the prediction. We only get a prediction for the last sample in the batch, the remaining samples only being used to add lags. pred = self . model . pipe . predict ( batch_X . iloc [ [ - 1 ] , : ] ) # Add the prediction to the list.  predictions . append ( pred ) # Add the prediction to y to be used as a lag target for the next prediction y . iloc [ i - self . first_pred_modifier , 0 ] = pred # If probabilities need to be returned if get_proba : # Get the predicted probability for each sample  probabilities . append ( self . model . pipe . predict_proba ( batch_X . iloc [ [ - 1 ] , : ] ) ) else : # Add lag observations to the samples if required if self . model . lags : X = self . _add_lags ( X , extrapolate = self . first_pred_modifier ) # Get prediction for X predictions = self . model . pipe . predict ( X ) # If probabilities need to be returned if get_proba : # Get the predicted probability for each sample  probabilities = self . model . pipe . predict_proba ( X ) # Set the number of placeholders needed in the response # These are samples for which predictions were not generated due to insufficient lag periods or for meeting multi-step prediction period requirements self . placeholders = self . rows_per_pred + self . diff_lags - self . first_pred_modifier # Transform probabilities to a readable string if get_proba : # Add the required number of placeholders at the start of the response list y = [ "\x00" ] * self . placeholders # Truncate multi-step predictions if the (number of samples - self.rows_per_pred) is not a multiple of prediction_periods if prediction_periods > 1 and ( ( n_samples - self . rows_per_pred ) % prediction_periods ) > 0 : probabilities = probabilities [ : - len ( probabilities ) + ( n_samples - self . rows_per_pred ) ] for a in probabilities : s = "" i = 0 for b in a : s = s + ", {0}: {1:.3f}" . format ( self . model . pipe . named_steps [ 'estimator' ] . classes_ [ i ] , b ) i += 1 y . append ( s [ 2 : ] ) # Prepare predictions else : if prediction_periods > 1 : # Set the value to use for nulls null = np . NaN if is_numeric_dtype ( np . array ( predictions ) ) else "\x00" # Truncate multi-step predictions if the (number of samples - self.placeholders) is not a multiple of prediction_periods if ( n_samples - self . rows_per_pred ) % prediction_periods > 0 : predictions = predictions [ : - len ( predictions ) + ( n_samples - self . rows_per_pred ) ] # Add null values at the start of the response list to match the cardinality of the input from Qlik y = np . array ( ( [ null ] * ( self . rows_per_pred - self . first_pred_modifier ) ) + predictions ) elif self . model . lag_target : # Remove actual values for which we did not generate predictions due to insufficient lags if is_numeric_dtype ( y . iloc [ : , 0 ] . dtype ) : y . iloc [ : self . placeholders - self . first_pred_modifier , 0 ] = np . NaN else : y . iloc [ : self . placeholders - self . first_pred_modifier , 0 ] = "\x00" # Flatten y to the expected 1D shape y = y . values . ravel ( ) else : y = np . array ( predictions ) # Inverse transformations on the targets if required   if variant != 'internal' and ( self . model . scale_target or self . model . make_stationary ) : # Take out placeholder values before inverse transform of targets null_values = y [ : self . rows_per_pred - self . first_pred_modifier ] if prediction_periods > 1 or self . model . lag_target else [ ] # Add placeholders for samples removed during differencing if self . model . make_stationary == 'difference' : null_values = np . append ( null_values , np . repeat ( null_values [ 0 ] , self . diff_lags ) ) y = y if len ( null_values ) == 0 else y [ - len ( predictions ) : ] # Add untransformed lag values for differencing if required end = self . placeholders start = end - self . diff_lags y = y if y_orig is None else np . append ( y_orig [ start : end ] , y ) # Apply the transformer to the test targets y = self . model . target_transformer . inverse_transform ( y ) # Remove lags used for making the series stationary in case of differencing if self . model . make_stationary == 'difference' : y = y [ self . diff_lags : ] # Replace lags used for making the series stationary with nulls in case of differencing # if self.model.make_stationary == 'difference': #null = np.NaN if is_numeric_dtype(np.array(predictions)) else "\x00" # y = np.append(np.array([null]*self.diff_lags), y[self.diff_lags:]) # Add back the placeholders for lag values if len ( null_values ) > 0 : y = np . append ( null_values , y ) if variant == 'internal' : return y # Add predictions / probabilities to the response self . response [ 'result' ] = y # Reindex the response to reset to the original sort order self . response = self . response . reindex ( self . original_index ) if load_script : # If the function was called through the load script we return a Data Frame self . _send_table_description ( "predict" ) # Debug information is printed to the terminal and logs if the paramater debug = true if self . model . debug : self . _print_log ( 4 ) return self . response # If the function was called through a chart expression we return a Series else : # Debug information is printed to the terminal and logs if the paramater debug = true if self . model . debug : self . _print_log ( 4 ) return self . response . loc [ : , 'result' ]