Well, while open source does not mean it's secure, this is still a weird thing to do.

I would simply recommend to stop using Signal and start using XMPP with OMEMO encryption, since this is the gold standard of instant messengers, at least for me. You should especially stop using Signal because it requires your phone number, which immediately disqualifies it for a private messenger.