Hi Attila,
It is not clear what your exact requirement is, from what you have mentioned. But I would say this is not a normal scenario that we come across.
From what you have mentioned, If you want to just close the window in the new tab, then you should put the window.close in the view in which you have the button,but not the standard navigate back button. You should create a new button in the view, and put the javascript inside the .htm page of the corresponding view under the window(window here means not the internet explorer, but the window in the workbench)
I would again suggest not to open a window in a new tab unless you have a specific requirement. Why because, in Web client, there should be only one active window at a time. If you open a window in a new tab, then you have 2 active windows which is different from opening a popup. Also if you need to communicate data between these 2 windows while navigating back, this is not the correct way. Rather you should go for a popup.
Thanks & Best Regards,
Leon