Pretrain model not accepting optimizer
For this code,model = TFAutoModelForSequenceClassification.from_pretrained(“bert-base-cased”, num_labels=3)model.compile(optimizer = tf.keras.optimizers.Adam(learning_rate=5e-5))This gives me this error ValueError: Could not interpret optimizer identifier: <keras.src.optimizers.adam.Adam object at 0x7e0d28e55fc0>what to do?I am using google colab
2024-03-08T01:21:35Z
I’m having the same issue, but using this code:model = TFBertForSequenceClassification.from_pretrained(‘neuralmind/bert-base-portuguese-cased’, num_labels=2)optimizer = tf.keras.optimizers.Adam(learning_rate=5e-5)loss = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)model.compile(optimizer=optimizer, loss=loss, metrics=[‘accuracy’])
2024-03-10T01:47:26Z
Same problem. Tried switching optimizers but not working for any (SGD or Adam)
2024-03-11T01:26:17Z
This is also a case for me when I’m using tf on 2.15, and when I change it to 2.16 versions and try to “from transformers import TFBertForSequenceClassification” I’m having the issue below:RuntimeError: Failed to import transformers.models.bert.modeling_tf_bert because of the following error (look up to see its traceback):module ‘tensorflow._api.v2.compat.v2.internal’ has no attribute ‘register_load_context_function’
2024-03-11T10:08:09Z
Same error here. I am using keras 2.15 and this code is failing now (two months ago it was working fine):import tensorflow as tf
from transformers import TFBertForSequenceClassification

REDACTED_TOKEN_name = "dccuchile/bert-base-spanish-wwm-cased"
model = TFBertForSequenceClassification.from_pretrained(REDACTED_TOKEN_name)

optimizer = tf.keras.optimizers.Adam(learning_rate=3e-5, epsilon=1e-08, clipnorm=1.0)
loss = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
metric = tf.keras.metrics.SparseCategoricalAccuracy('accuracy')

model.compile(optimizer=optimizer, loss=loss, metrics=[metric])Error:ValueError: Could not interpret optimizer identifier: <keras.src.optimizers.adam.Adam object at 0x7e3ac5f13d60>
2024-03-11T18:23:31Z
Hi all, Matt from Hugging Face here! The cause is that TensorFlow has switched to Keras 3 as the ‘default’ Keras as of TF 2.16, and Keras 3 is often installed alongside TF 2.15 as well. The errors in this thread are because Keras 3 objects are being passed to Keras 2 model objects and code.The quickest solution is topip install tf-kerasand then set the environment variableTF_USE_LEGACY_KERAS=1. This will maketf.keraspoint to Keras 2, and your code should work as before.We’re also pushing a fix totransformersto do this by defaulthere. If you want to try the fixedtransformersbranch instead, you can install it withpip install git+https://github.com/huggingface/transformers.git@keras_3_compat_fixThis is an area of active work for us - please let us know if these approaches fixed your problem, and if you’re encountering any other issues with the Keras 3 transition!
2024-03-12T09:42:53Z
Thanks for the solution,@Rocketknight1It works correctly for me.Looking forward to seeing this fix by default in thetransformerlibrary.Regards
2024-03-21T02:05:50Z
Hi Matt,both solutions did not work for me in Google Colab.Solution 1 produced the same error for me!pip install tf-keras
import os
os.environ['TF_USE_LEGACY_KERAS'] = '1'Solution 2!pip install git+https://github.com/huggingface/transformers.git@keras_3_compat_fixresulted in the following message:Collecting git+https://github.com/huggingface/transformers.git@keras_3_compat_fix
  Cloning https://github.com/huggingface/transformers.git (to revision keras_3_compat_fix) to /tmp/pip-req-build-u6t3ei27
  Running command git clone --filter=blob:none --quiet https://github.com/huggingface/transformers.git /tmp/pip-req-build-u6t3ei27
  WARNING: Did not find branch or tag 'keras_3_compat_fix', assuming revision or ref.
  Running command git checkout -q keras_3_compat_fix
  error: pathspec 'keras_3_compat_fix' did not match any file(s) known to git
  error: subprocess-exited-with-error
  
  × git checkout -q keras_3_compat_fix did not run successfully.
  │ exit code: 1
  ╰─> See above for output.
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× git checkout -q keras_3_compat_fix did not run successfully.
│ exit code: 1
╰─> See above for output.What can I do?
2024-03-21T09:06:29Z
try !pip install tf-keras==2.15.0
2024-03-21T12:01:44Z
Hi@leon-hecht, thekeras_3_compat_fixbranch has been deleted since it has now been merged into transformers, and included in version 4.39 which released today. You can now just runpip install --upgrade transformersto get the fix instead ofpip install git+https://github.com/huggingface/transformers.git@keras_3_compat_fix. Let me know if you’re still encountering problems!
2024-03-21T18:25:11Z
I am facing the same issue, and I followed along the entire thread and still not able to resolve it.
2024-03-21T20:56:29Z
I just installed the latest version of the Transformers library (v4.39.0) on my Google Colab, and it worked for me. Thanks@Rocketknight1Code:!pip install --upgrade transformers
import transformers
print(transformers.__version__)
2024-03-22T15:49:51Z
Hi@mahreenfatima, can you confirm that you’re using transformers 4.39, and if so, can you paste me some sample code that shows your issue?
2024-03-22T19:41:09Z
This is the code that is raising a ValueError, I am using transformers version 4.39.0 and tensorflow 2.15.0, I tried restarting the session on google colab to see the changes and stil the same error.opt_new = Adam(learning_rate = learning_rate_scheduler)model = TFAutoModelForSequenceClassification.from_pretrained(checkpoint,num_labels=2)loss = SparseCategoricalCrossentropy(from_logits = True)model.compile(optimizer=opt_new, loss=loss, metrics=[“accuracy”])ValueError Traceback (most recent call last)in <cell line: 1>() ----> 1 model.compile(optimizer=opt_new, loss=loss, metrics=[“accuracy”])2 frames/usr/local/lib/python3.10/dist-packages/transformers/modeling_tf_utils.pyin compile(self, optimizer, loss, metrics, loss_weights, weighted_metrics, run_eagerly, steps_per_execution, **kwargs) 1502 # This argument got renamed, we need to support both versions1503         if “steps_per_execution” in parent_args:→ 1504             super().compile(1505                 optimizer=optimizer,1506                 loss=loss,/usr/local/lib/python3.10/dist-packages/tf_keras/src/utils/traceback_utils.py in error_handler(*args, **kwargs)68             # To get the full stack trace, call:69             #tf.debugging.disable_traceback_filtering()—> 70             raise e.with_traceback(filtered_tb) from None71         finally:72             del filtered_tb/usr/local/lib/python3.10/dist-packages/tf_keras/src/optimizers/init.py in get(identifier, **kwargs)332         )333     else:→ 334         raise ValueError(335             f"Could not interpret optimizer identifier: {identifier}"336         )ValueError: Could not interpret optimizer identifier: <keras.src.optimizers.adam.Adam object at 0x7aa3c841a830>Also while importing,from tensorflow.keras.losses import SparseCategoricalCrossentropyfrom tensorflow.keras.optimizers.schedules import PolynomialDecay,as I hover over the tensorflow.keras it is underlined and shows the following message--------Import “tensorflow.keras.optimizers.schedules” could not be resolved(reportMissingImports)-------
2024-03-28T13:16:12Z
I tried all the solutions suggested in the discussion but still facing the same issue.
2024-04-03T06:57:00Z
I am using Colab, I did all that this thread says:!pip install --upgrade transformers → Upgraded to 4.39.2 version!pip install tf-kerasos.environ[‘TF_USE_LEGACY_KERAS’] = ‘1’ → I’ve set up the legacy versionAfter all, I restarted the colab session and it worked, thanks
2024-04-04T18:18:26Z
I have also tried everything mentioned above. My transformer version is up to date and I have done what others have done but to no avail. What is the current recommendation that works?
2024-04-08T03:32:58Z
I have the same problem and still getting same error. I tried everything, but it doesn’t  work. I am working on a project and I am short on time. Please help.
2024-04-09T07:16:49Z
I’ve also tried all the steps as mentioned above but nothing is working. Attached is the SS showing the error I’m gettingScreenshot 2024-04-09 at 12.46.38 PM3268×1192 377 KBEDIT: The problem solved when I restarted the kernel. Do not refresh the google colab (after doing all the steps), rather restart it
2024-04-10T20:57:21Z
Hello@Rocketknight1, I have followed all of the above recommendations:!pip install --upgrade transformers
!pip install tf-keras
import os
os.environ['TF_USE_LEGACY_KERAS'] = '1'AND restarted the colab session after installation, but I still get the errorValueError: Could not interpret optimizer identifier: <keras.src.optimizers.adam.Adam object at 0x7f3e884e2b60>I am usingcheckpoint = "bert-base-uncased" 
model = TFAutoModelForSequenceClassification.from_pretrained(checkpoint, num_labels=2)Any recommendations why this may still not be working for some?
No date available
No reply text available