Devices deleted on NSO UI gets deleted but displays back again in sometime: Devices deleted on NSO UI gets deleted but displays back again in sometimeNSO version 6.1 engg build#2, TSDN 5.0-M5Steps:1. Onboard few devices onto CW - DLM.2. Devices will be added to NSO UI and in-sync.3. Delete few of the devices from CW - DLM and then NSO UI. Devices will be deleted successfully.4. When we reopen NSO we can see devices list is still displayed in NSO.
 Hi, would you be able to provide the exact NSO version you were using for this ticket? In CLI you can simply do<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"><pre class="code-java">ncs --version6.1_xxxx.xxxxx.xxxxx</pre></div></div>In order to reproduce, we need the NSO build hash as shown above.Thanks,Isaac KimHi Isaac,Please find the requested information for ncs version.admin@nso-1:~$ ncs --version6.1_221212.150815390.4b4cc38b0df7Thanks,Gomathi.Hi Gomanthi,I have tested device delete on multiple devices using NSO 6.1_221212 Web GUI and waited 30 minutes and I do not see the devices coming back up on the UI. As we do not have access to CNC Setup and FPs like DLM, I cannot comment if this is a CNC issue. But on the NSO side, things seem to be working fine.Thanks,Isaac KimHi Isaac,I can provide my nso and CW VM credentials if that helps to troubleshoot the issue.Let me know and I can unicast the details.Thanks,gomathi.Hi Gomathi, there seems to be some general NSO UI issues with NSO 6.1 which may or may not be related to this ticket.I have a few questions to help with RCA.<ol>Why are we using NSO UI to delete devices when CW provides similar function?Has there been any services deployed from NSO UI?</ol>Thanks,Isaac KimHi Isaac,Please find answers inline:1. Why are we using NSO UI to delete devices when CW provides similar function?: As mentioned in step#3 in original issue description, I used CW -DLM UI to delete devices first. Expectation is that devices should get deleted on NSO side as well. But, seems like it didn't get deleted.Note: I can see the devices on ncs cli as well. So, issue is not only with NSO UI but also with ncs cli.2. Has there been any services deployed from NSO UI?: I have been uploading service payloads via NSO UI and services get provisioned after that. Didn't perform any explicit operation to deploy services from NSO UI.Hi Gomanthi, seems like NSO UI shows some unexpected behavior when services are deployed through the GUI. To confirm if this is really a core NSO platform issue, we can try to delete the devices from NSO CLI and see if it goes through. On my side, happy path device deletion seems to be working without any issue.Either I can do it if you provide your VM credentials, or you could try it out on NSO CLI using<ol>enter NSO CLIenter config modedelete devices device <DEVICE>commit</ol>Let me know which option works for you.Thanks,Isaac KimHi Isaac,Devices were successfully deleted from ncs_cli and don't see them again on UI and ncs_cli after commit from cli.NSO UI also don't show devices so far. Please find screenshots attached.In this case, the NSO UI inconsistent behavior will be fixed for device manager?Another question: I tried to onboard two new NCS devices from CW -DLM with different credential profile. Later I updated it on CW but on NSO I had to update it manually. To update ssh keys for these NCS devices I had add the devices via IP and then updated ssh fetch host keys. Adding ssh host keys via device hostname gives connection time out error.Can you confirm what is expected for the above scenario as well.Thanks,Gomathi.Devices were successfully deleted from ncs_cli and don't see them again on UI and ncs_cli after commit from cli.NSO UI also don't show deleted devices so far. Please find screenshots attached.Hi Gomathi, thats great to hear that NSO CLI device deletion went through!"In this case, the NSO UI inconsistent behavior will be fixed for device manager?"There is active investigation going on with NSO 6.1 GUI related issues. It also seems like this may impact NB interfaces like jsonrpc and RESTCONF. We are not totally sure of what the root cause is, but it does seem like deploying services through NSO GUI may be one of the causes. I will provide updates here as we gain more clarity.In the meantime, I would suggest to avoid using NSO GUI if possible. As you may probably know, the NSO 6.1 builds we are using are still engineering builds, so there may be unexpected behavior."I tried to onboard two new NCS devices from CW -DLM with different credential profile. Later I updated it on CW but on NSO I had to update it manually. To update ssh keys for these NCS devices I had add the devices via IP and then updated ssh fetch host keys. Adding ssh host keys via device hostname gives connection time out error.Can you confirm what is expected for the above scenario as well."Connection timeout could be due to a variety of issues. Most likely device is unresponsive. Im not sure what the device onboarding procedure is for CW, but to update ssh keys for all devices from NSO CLI you can do:<ol>enter NSO CLIrequest devices fetch-ssh-host-keys</ol>Thanks,Isaac KimHi Gomanthi,We just had a call today and were able to resolve all doubts. As agreed, I will close this ticket.Thanks,Isaac Kimnot an issue