Fix the filesystem library check on GCC8 and Clang8

Summary:
Using the linker flags is not good for every version as it really is
an optional library. This diff makes this more correct and avoid build
failures/warnings for some older compiler versions (especially GCC8 on
Debian Buster).

Test Plan:
With GCC8 and Clang8 (on Debian Buster):

ninja all check

Note: for the clang-8 build to succeed you need D13803.

Reviewers: #bitcoin_abc, PiRK

Reviewed By: #bitcoin_abc, PiRK

Differential Revision: https://reviews.bitcoinabc.org/D13804