Hello I have similar experiences and had a few minutes to create some videos demonstrating the issue.
Here is a video of opening a common website in a new window during an Orion session I had running since Saturday (4 days):
In addition to some sluggish, notice the flicker at the end when closing the window. It's like it's working through it's list of open and closed windows.
Now here is a video of doing the same after restarting the Orion (using auto-open all windows from last session startup option to restore the same session of 1 window of 24 tabs of mostly text):
A bit more responsive and no flicker at the end.
Then here is a video doing the same from a clean session(no other window/tabs open in Orion):
A little faster still.
Finally, for comparison with the last, here is doing the same in Brave (1.69.168):
This is all under Catalina on a MacBook Air (Intel i3 w/16GB). Obviously this is a lower-end machine (deliberately so at the time) and an M1/2/3/4 Pro/Max/etc would be much faster but It would be great if it make the best use of the available hardware as possible. I have snapshots of Activity Monitor from before I restarted Orion the first time if helpful to share those as well.
Orion remains my favorite browser from a privacy and UI/UX experience but it doesn't feel as light and snappy as when I first trialed the beta a few years ago.
FYI for testing purposes my sessions typically include opening and closing hundreds of windows/tabs per day for sessions that would ideally go on for weeks. Ideally during that time Orion would have the same memory footprint and responsiveness at the end of the session as at the beginning.