text
The new Claude Sonnet 3.5 model from Anthropic AI has been getting good reviews on since last night. It is quite good at coding related tasks. We tried it on the Static Analysis Eval benchmark (
The new Claude Sonnet 3.5 model from Anthropic AI has been getting good reviews on since last night. It is quite good at coding related tasks. We tried it on the Static Analysis Eval benchmark (
resource
https://huggingface.co/datasets/patched-codes/static-analysis-eval
dataset
patched-codes/static-analysis-eval
https://huggingface.co/datasets/patched-codes/static-analysis-eval
text
) which measures the ability of a LLM to fix vulnerabilities. The model scores 59.21% which is good but not better than other frontier models (like GPT-4, Gemini-1.5 and LLama-3).
) which measures the ability of a LLM to fix vulnerabilities. The model scores 59.21% which is good but not better than other frontier models (like GPT-4, Gemini-1.5 and LLama-3).
The new Claude Sonnet 3.5 model from Anthropic AI has been getting good reviews on since last night. It is quite good at coding related tasks. We tried it on the Static Analysis Eval benchmark (https://huggingface.co/datasets/patched-codes/static-analysis-eval) which measures the ability of a LLM to fix vulnerabilities. The model scores 59.21% which is good but not better than other frontier models (like GPT-4, Gemini-1.5 and LLama-3).
👀
codelion John6666 sbarman25 lemon-mint den0620 sroecker
🧠
codelion phanerozoic pduf xi0v
👍
codelion joseEjmendez
🚀
codelion
/avatars/066a250e5f73c4f37c9436e60aa8c861.svg
Minoru Tanaka 
Minorutanaka14052005
user
https://cdn-avatars.huggingface.co/v1/production/uploads/1671634255137-noauth.png
diego lopoez
D1360VR
user
/avatars/8ee8dc056adbf361e79f577ce801e163.svg
JacketJacket
Jacket1119
user
/avatars/0fbd78ac044ee1fa6248ab1119021956.svg
Fuji Santoso
FujiAILearner
user
/avatars/cbd018187d5909d2173357f46bff030f.svg
امیر مرادی
Ettdghh
user
/avatars/2b3abd4d9045b68fc213946950bdf4cd.svg
Rich'Art Dely 
Richartvrai
user
/avatars/f3fc9a8fa17c76dfc4d5c3d7a89a6956.svg
MEDDEB
NOBODY204
user