Yes yes you csn.

The old may was we would run these direct like an app on the pi.

People then came up with containers and isolated things a bit made them easy to share and configure. - dockers

Have a look for people with blogs about all the software they are running on their pi plenty of info on how.

Pi isnt the best at jellyfin server as it isnt the best at transcoding files but for direct playing without having to convert the format its enough.