text
After the announcements yesterday, I got a chance to try the new gemini-1.5-flash model from 
After the announcements yesterday, I got a chance to try the new gemini-1.5-flash model from 
mention
@goog1e
goog1e
text
, it is almost as good as gpt-4o on the StaticAnalaysisEval (
, it is almost as good as gpt-4o on the StaticAnalaysisEval (
resource
https://huggingface.co/datasets/patched-codes/static-analysis-eval
dataset
patched-codes/static-analysis-eval
https://huggingface.co/datasets/patched-codes/static-analysis-eval
text
) It is also a bit faster than gpt-4o and much cheaper. 
) It is also a bit faster than gpt-4o and much cheaper. 
new_line
new_line
text
I did run into a recitation flag with an example in the dataset where the api refused to fix the vulnerability and flagged the input as using copyrighted content. This is something you cannot unset even with the safety filters and seems to be an existing bug 
I did run into a recitation flag with an example in the dataset where the api refused to fix the vulnerability and flagged the input as using copyrighted content. This is something you cannot unset even with the safety filters and seems to be an existing bug 
link
https://issuetracker.google.com/issues/331677495
https://issuetracker.google.com/issues/331677495
new_line
new_line
text
But overall you get gpt-4o level performance for 7% the price, we are thinking of making it default in patchwork - 
But overall you get gpt-4o level performance for 7% the price, we are thinking of making it default in patchwork - 
link
https://github.com/patched-codes/patchwork
https://github.com/patched-codes/patchwork
text
 You can use the 
 You can use the 
inline_code
`google_api_key`
google_api_key
text
 and 
 and 
inline_code
`model`
model
text
 options to choose 
 options to choose 
inline_code
`gemini-1.5-flash-latest`
gemini-1.5-flash-latest
text
 to run it with patchwork.
 to run it with patchwork.
After the announcements yesterday, I got a chance to try the new gemini-1.5-flash model from @goog1e, it is almost as good as gpt-4o on the StaticAnalaysisEval (https://huggingface.co/datasets/patched-codes/static-analysis-eval) It is also a bit faster than gpt-4o and much cheaper. 

I did run into a recitation flag with an example in the dataset where the api refused to fix the vulnerability and flagged the input as using copyrighted content. This is something you cannot unset even with the safety filters and seems to be an existing bug https://issuetracker.google.com/issues/331677495

But overall you get gpt-4o level performance for 7% the price, we are thinking of making it default in patchwork - https://github.com/patched-codes/patchwork You can use the `google_api_key` and `model` options to choose `gemini-1.5-flash-latest` to run it with patchwork.
🔥
codelion lunarflu victor alias8797 osanseviero
🚀
codelion lunarflu alias8797
🤯
codelion lunarflu alias8797
https://cdn-avatars.huggingface.co/v1/production/uploads/6340651b388c3fa40f9a5bc0/av1C4_S7bHGxAzOu8lOmG.jpeg
Adam Molnar
lunarflu
user
https://cdn-avatars.huggingface.co/v1/production/uploads/1677134945205-62f32eab52ad88c930bb3f3b.png
Asankhaya Sharma
codelion
user