When you are not on the "new tab" page, there's no clear visual distinction between a regular and private window (at least that I can see). I think it's important there is one.
Firefox adds a small icon/motif to the upper right of the window's chrome, Safari 15 seems to show a "tab group" label of "Private", etc.
It may also be worth labelling the Window name (as seen in the Window menu) too with something like (Private) as a suffix?