Depends on where its leaking from, if its the mounting thread, then re-installing it with thread sealer should take care of it, but if the oil is coming from the electrical switch section, then a replacement is your only option.
If there's minor leakage at the mounting thread, its not going to give you false low pressure warnings, but if its leaking into the electrical switch it could be giving false indications.
Until the actual oil pressure is confirmed, I'd not drive the vehicle.