Custom Inference handler.py: FileNotFoundError
Hi all,I am trying to deploy my own custom model usinghandler.py. Here is my code so far:class EndpointHandler():

    def __init__(self, path=""):

        """ Initialize the model and required parameters."""

        # model_path = "SRx4_EDTB_Div2kFlickr2K.pth"

        config_root = os.path.join(os.path.dirname(os.path.abspath(__file__)), 'configs')
        config_files = [
            os.path.splitext(os.path.basename(v))[0] for v in scandir(config_root)
            if v.endswith('.py')
        ]
        config_file = os.path.basename("./configs/SRx4_EDTB_Div2kFlickr2K.py").split('.')[0]
        assert config_file in config_files, 'Illegal config!'
        module_reg = importlib.import_module(f'configs.{config_file}')
        self.config = getattr(module_reg, 'Config', None)

        self.model = Network(self.config)
        if torch.cuda.is_available():
            self.device = torch.device('cuda')
        else:
            self.device = torch.device('cpu')
        self.model = self.model.to(self.device)

        weights = "SRx4_EDTB_Div2kFlickr2K.pth"

        load_model_filter_list(self.model, weights, filter_list=[])
        self.model.eval()


    def __call__(self, data: Dict[str, Any]) -> List[Dict[str, Any]]:

        """
        Data Args:
            inputs (:obj: `str` | `PIL.Image` | `np.array`)
            kwargs
        Return:
            A :obj:`list` | `dict`: will be serialized and returned
        """

        input_data = data.pop("inputs", data)

        img = torch.from_numpy(np.array(input_data))
        img = img.permute(2, 0, 1).unsqueeze(0) / 255.0
        img = img.to(self.device)
        # if model.parameters().dtype == 'fp16':
        #     input_data = input_data.half()

        scales = []
        for s in self.config.VAL.SCALES:
            scales.append(s)

        lqs, _, _ = preprocess_images(img, scales[0], self.config)

        with torch.no_grad():

            start_time = time.time()
            preds = self.model(lqs)
            torch.cuda.synchronize()
            end_time = time.time()
            print("Inference time: ", end_time - start_time, "seconds")

            result = np.array(preds[0][0,:,:,:].cpu())

        return  {"result" : np.transpose(result, (1,2,0))}Which gives me this error when I initialize the endpoint:FileNotFoundError: [Errno 2] No such file or directory: 'SRx4_EDTB_Div2kFlickr2K.pth'It works on my PC in a container, the file exists, I tried many things already and it doesn’t see my model. What is going on? I don’t know how to solve this issue at this point.
2023-10-04T09:45:45Z
Hi,You are not using thepathparameter of the__init__function of the handler.It tells you where your model repository is pulled.So you probably want to construct the path to your weights using thispathargument.
2023-10-04T14:46:55Z
Tried. Doesn’t work. Works with /repository/your_model.pth, but doesn’t work with ONNX models. It just doesn’t load it, it is not there even though it exists in the repo.
2023-10-04T15:35:02Z
Yes I have the same problem. In the logs there is a line that indicates that some extensions are ignored when fetching the repository.INFO | Ignore regex pattern for files, which are not downloaded:onnx,tar.gz, rust, *mlmodel,tflite, tf,openvino,ckpt, flax, *safetensorsI don’t know why these file extensions are ignored…One potential workaround is maybe to save the weights file under another extension in the repo, and then at the start of__init__change it back to its original extension but haven’t tried it yet.
2023-10-16T19:56:04Z
I am encountering the same issue as I have a single model.safetensors in my repo that I am trying to load using from_single_file and although everything works perfectly fine testing the EndpointHandler locally once I deploy it I get an error that the provided path is not a valid model file or repository.
2023-12-21T20:15:48Z
Facing some problem with onnx model, anyone have a solution?
2024-01-29T13:44:10Z
Same here. Lost a day before pointing out that pulling ignoring my onnx files from a huggingface model. What we should do?
2024-01-29T15:49:18Z
I don’t know guys. I quit and became a dancer.
2024-04-08T16:18:50Z
For anyone struggling with this issue, it seems to come from the “Framework” setting in the “Advanced configuration” panel when creating an inference endpoint. By default, it will select “PyTorch”, which will ignore the mentioned file formats. If you don’t want these file formats to be ignored, just select “Custom” for this field.