$_SERVER['REMOTE_ADDR'] is the IP address the TCP connection came in on. While it is technically possible to bidirectionally spoof IP addresses on the Internet (by announcing foul routes via BGP), such attacks are likely to be spotted and not available to the typical attacker - basically, your attacker must have control over an ISP or carrier. There are no feasible unidirectional spoofing attacks against TCP (yet). Bidirectional IP spoofing is trivial on a LAN though.
Also be aware that it may be not be an IPv4, but an IPv6 address. Your current check is fine in that regard, but if you would check that
18.104.22.168 only occurs anywhere within
$_SERVER['REMOTE_ADDR'], an attacker could simply connect from
Summarily, for anything other than critical (banking/military/potential damage >50.000‚ā¨) applications, you can use the remote IP address if you can exclude attackers in your local network.