Hmm, it's now working for me in all configurations. GitHub must have fixed this one themselves.
@deseven the back/forward behavior does happen in Chrome as well, the first navigation. After that it works. Likely has to do with ?tab=readme-ov-file being added/removed from the URL.
When I was experiencing this problem, I notice that it was related to how GitHub uses the History API to set ?tab=readme-ov-file, and that was causing issues in Orion but not in Safari.