Well it seems that there's a difference between using the X3 as the sound interface or the Babyface (and running Pod Farm on either of them).

Turns out that me finding the sound of the Pod Farm subpar with the Babyface had more to do with the gain levels on the Babyface's Hi-Z guitar input and the POD Farm input itself.. adjusting them seemed to make things sound more even again. Although (and probably obviously) using the X3 as the sound interface needed no adjustements between it's direct monitoring sound and Pod Farm because it's the same hardware.. it just sounded the same.

Changing sample rates in the X3 didn't seemt o affect the sound at all (i.e. it sounded the same between 44100 and 88200 Khz). However the same procedure using the Babyface causes a sonic change which can be seen in here https://soundcloud.com/mindwalker/44100-vs-88200-pod-farm-in

PS: the recording of this clip was done in Reaper using Pod Farm VST

So in short here's the summary of my findings:

a) If running POD Farm using the POD X3 as the sound interface, there is no difference if I set the sampling rate in Reaper from 44100 to 88200. Also using the X3 as the sound interface seems to make Pod Farm sound identical to the POD itself (this is easy to check by alternating between switching between direct monitoring or monitoring through Reaper and POD Farm)

b) Using the Babyface as the sound interface things aren't so clear. There must some difference in the gain levels that make it sound slightly different to the POD. Ultimately I was able to find a setting that yielded the most similar results (by lowering the guitar input gain on the babyface to +9db) and slightly adjusting the Pod Farm input + output levels. Although this is a bit of trial and error and comparing to the sound of the POD X3 isn't easy since it requires switching cables around and the brain can easily be fooled

c) Both the Babyface and the X3 used as sound interfaces running at 44100 sounds almost identical between each other.

d) Changing the sampling rate in the Babyface from 44100 to 48000 or higher (88200) produces noticeable sonic change (to my ears, better). This didn't happen if I used the X3 as the sound interface

I still don't know what to make of all of this.. or why the Babyface changes the sound in higher sample rates whereas the X3 doesn't. Anyhow.. worst case scenario the Babyface sounds pretty similar to the X3 and can sound better with higher sampling rates. Now I dunno if I want to start using higher sampling rates

In fact the most dramatic difference with the Babyface is heard from 44100 to 48000 Khz. From 48000 Khz to 88200 it's pretty much the same.