Using Persistent Disk and External DB with Flowise Space
I’ve tried to add a persistance storage to my space.I’m running Flowise using a dockerfile, but i cannot get the storage to persist my sqlite setup.
2023-10-23T20:33:10Z
hi@Academondo,Do you know to customize the working dir with Flowise? on HF the persistent data is always mounted as  volume/dataIf you can point Flowise to write and read the volume that would probably make it work.
2023-10-23T20:45:17Z
Thank you for your reply.i use this dockerfile. but nothing is persisted it seems.FROM node:18-alpineUSER rootArguments that can be passed at build timeARG FLOWISE_PATH=/usr/local/lib/node_modules/flowiseARG BASE_PATH=/data/.flowiseARG DATABASE_PATH=$BASE_PATHARG APIKEY_PATH=$BASE_PATHARG SECRETKEY_PATH=$BASE_PATHARG LOG_PATH=$BASE_PATH/logsARG OVERRIDE_DATABASE=falseInstall dependenciesRUN apk add --no-cache git python3 py3-pip make g++ build-base cairo-dev pango-dev chromiumENV PUPPETEER_SKIP_DOWNLOAD=trueENV PUPPETEER_EXECUTABLE_PATH=/usr/bin/chromium-browserENV HF_HOME=/data/.huggingfaceInstall Flowise globallyRUN npm install -g flowiseConfigure Flowise directories using the ARGRUN mkdir -p $LOG_PATH $FLOWISE_PATH/uploads && chmod -R 777 $LOG_PATH $FLOWISE_PATHWORKDIR /dataCMD [“npx”, “flowise”, “start”]
2023-10-23T22:16:42Z
hi@Academondo, I made a couple of changes herehttps://huggingface.co/spaces/radames/Flowise/blob/main/Dockerfilethis seems to be working nowI tested building and rebooting, the Space, the sqlite file is now saved and reload correctlyFROM node:18-alpine
USER root

# Arguments that can be passed at build time
ARG FLOWISE_PATH=/usr/local/lib/node_modules/flowise
ARG BASE_PATH=/data/.flowise
ARG DATABASE_PATH=$BASE_PATH
ARG APIKEY_PATH=$BASE_PATH
ARG SECRETKEY_PATH=$BASE_PATH
ARG LOG_PATH=$BASE_PATH/logs

# Set the ARGs as ENV
ENV FLOWISE_PATH=$FLOWISE_PATH
ENV BASE_PATH=$BASE_PATH
ENV DATABASE_PATH=$DATABASE_PATH
ENV APIKEY_PATH=$APIKEY_PATH
ENV SECRETKEY_PATH=$SECRETKEY_PATH
ENV LOG_PATH=$LOG_PATH

# Install dependencies
RUN apk add --no-cache git python3 py3-pip make g++ build-base cairo-dev pango-dev chromium

ENV PUPPETEER_SKIP_DOWNLOAD=true
ENV PUPPETEER_EXECUTABLE_PATH=/usr/bin/chromium-browser

# Install Flowise globally
RUN npm install -g flowise

# # Configure Flowise directories using the ARG
RUN mkdir -p $FLOWISE_PATH/uploads && chmod -R 777 $FLOWISE_PATH

CMD ["sh", "-c", "mkdir -p $LOG_PATH && chmod -R 777 $LOG_PATH && npx flowise start"]
2023-10-31T15:12:39Z
Hi@radames, this seems to be working! Care to elaborate why the last part where you create the $LOG_PATH directory only works when using sh?
2023-10-31T17:49:53Z
Hi@station-dev, of course, Flowise was looking for$LOG_PATHin order to initialize. This is where it stores the persistent data. As I mentioned before, the/data/volume is only mounted at runtime. Therefore, before running Flowise, it creates the$LOG_PATHfolder at runtime.
2023-12-13T10:14:42Z
hi, since huggingface space doesn’t have persistent storage in free tier, is it possible to add our own 3rd party database for flowise?
2023-12-14T03:40:46Z
hi@ryansmith69, if you can have flowise remotely connect to the external database, then it’s possible, We have many Spaces, talking to external SQL databases.
2023-12-19T13:43:33Z
@radamesHi, I tried to do it like they have written in the Flowise Docs but, I can’t connect my database with Flowise. Every time I restart space  the chatflows, credentials, tools get deleted!@radamesCan you please show me how to do that?@ryansmith69Have you done it successfully?
2023-12-19T19:19:12Z
hi@sarahcarlson, I’m not familiar with Flowise, but reading their docshereit looks like this option is enabled, so on every launch or Space restart your DB is recreated?Synchronize in ProductionFlowise usesTypeormto configure database connection. By default, synchronize is set to true. This indicates if database schema should be auto created on every application launch.However, we have to be careful with this option and don’t use this in production - otherwise you canloseproduction data. This option is useful during debug and development.To override the value, set the following env variable
2023-12-19T20:39:28Z
@radameswell, I set all database credentials but, it still doesn’t work. do I need to change database path, apikey path, secretkey path and all these path from   /root/.flowise   to something else? if so, what do I set them as?
2023-12-19T20:44:46Z
@radamesHow bout you just show the steps like you showed earlier about the hugging face persistent disk. you’re an experienced man so you can help me. the flowise support team is crap so, there is nowhere to seek for support.
2023-12-19T22:39:17Z
hi@sarahcarlson,I did a fresh new duplication and set up a PostgreSQL withhttps://supabase.com/. After reboot it successfully reload the last state from the external DB.You can modify your Dockerfile to includeENV DATABASE_TYPE=postgres
ENV DATABASE_PORT=5432
ENV OVERRIDE_DATABASE=falseAnd add these as SecretsDATABASE_TYPE=postgres
DATABASE_PORT=5432
DATABASE_HOST=localhost
DATABASE_NAME=flowise
DATABASE_USER=user
DATABASE_PASSWORD=123
FLOWISE_SECRETKEY_OVERWRITE= #Random encryption key to recover tokens from the DBAdditionally, you can duplicate this new exampleFlowise DB Example - a Hugging Face Space by SpacesExamples. When you use this example, you will be prompted to enter the required variables and secrets.image1006×982 56.9 KB
2023-12-20T05:28:54Z
Oh, thank you so much@radames. That’s so nice of you. I really appreciate it.
2023-12-20T05:33:26Z
By the way, do I need to change the database path and also I have to paste these in dockerfile. right?ENV DATABASE_TYPE=postgresENV DATABASE_PORT=5432ENV OVERRIDE_DATABASE=false
2023-12-20T05:45:56Z
So sorry to bother you but, where do i get   FLOWISE_SECRETKEY_OVERWRITE   or can i do i create it in my own way like, whatever i like.
2023-12-20T19:34:27Z
sarahcarlson:FLOWISE_SECRETKEY_OVERWRITEYou can generate a random string. This is used to encrypt the secret tokens inside the database, and it’s required when you reboot your Space.
2023-12-20T19:35:12Z
please change the type and port in case you’re using MySQL, or your DB port is not the standard ports
2023-12-21T15:16:06Z
@radamesLook what’s showing-Screenshot 2023-12-21 at 10.12.40 AM3104×1820 690 KB
2023-12-21T15:16:44Z
@radamesScreenshot 2023-12-21 at 10.12.43 AM3104×1820 1.05 MB
No date available
No reply text available