How to run two LLMs in series for inference?
Hi there,I am building an application that accepts the user query and feeds it to a BART LLM for zero-shot-classification. The output of the BART model will be fed to another LLM (LLAMA-2).I am looking for an efficient way to load and query both LLMs while keeping GPU memory usage to the minimum.Here is what I do in steps:1- Load the BART model using transformersZeroShotClassificationPipeline2- Load the LLAMA-2-13B model using transformersTextGenerationPipeline3- for each received user’s query:(a) Send the query to the model loaded in “1”(b) Send the result from (a) to the model loaded in “2”